If you stop, you’re dead

Hello, friends. It’s not snowing, but it is windy, and I thought I’m going to record one of these again.

Because I have the feeling that a lot of software engineers are not aware of the—I call them tectonic forces—or these big forces at play right now in AI and software. I think there are a lot of misconceptions—or maybe, I don’t know, I don’t want to say underinformed—but there are a lot of ideas about inference and model houses that I don’t think truly capture what’s going on.

Here’s how I see the situation, okay, for what it’s worth.

First of all, the one thing that you cannot ignore is that these model houses are in a race with each other. And that must fucking suck. They are in a race because models are a commodity.

If you use GPT-5.6, you can switch to Fable. If you use Fable, you can switch to GPT-5.6. Yes, there are some differences in, you know, however they—blah, blah, blah—but they’re pretty good. I think for 95% of use cases, you can switch between them. And that’s been true for the last three years. Models are a commodity. There’s nothing locking you into one model. So whenever you become the best model, people will switch over to you. That is a race, because there’s no prize for second place, right?

Because the other thing here at play is that they all believe they’re racing towards AGI. At some point, you will have a model so good that you have exponential takeoff; the model can train itself. And then, you know, whoever gets to that line probably wins this whole thing and, I don’t know, has such an advantage that the others can’t catch up, right? But until then, it is a race.

And if you’re Anthropic, you work on Fable for however many months, right? And you think, this is the best model we’ve ever had. This is a Mythos-class model. And you release it. Then six weeks later, GPT-5.6 comes out. And the day after, people write, ‘Yeah, yeah, Fable is good. I saw somebody say it’s a wise owl. But GPT-5.6? That’s actually the workhorse.’

So imagine this: how many millions or billions have been spent on this model—and, oh, there’s a deer—and then six weeks later your competitor comes out with another thing, and people can just switch over. They already have all of the APIs implemented. The APIs are pretty much the same. You can switch over. So that’s a race. Every time you do something, your competitor catches up.

But now listen: since last year, or one and a half years ago, you’re in this race and you look in the rearview mirror. What’s there? Chinese companies that are releasing the models as open source. So that means when you stop, they will catch up and destroy all of the margins that you might have.

And there are margins. I think there are margins in inference. Dario Amodei has said this in an interview, right? And I think on a Dwarkesh Patel podcast, he said that if you take a single model and treat it as a separate, standalone company, then it will be profitable. You put money into training, but it will make the money back on inference because there are positive margins in inference.

Other inference providers have said this too—even independent inference providers have said there are margins in running inference. How much margin? Nobody really knows. The numbers go from 20% to 85%. But there is money in there.

The problem is, if you’re a model house, you need to take all of the profits you make on inference and put them into the next model, because right now your model is already outdated. So you’ve got to take all the money you make and reinvest it into training the next model.

It usually has to be bigger, right? So it’s like, okay, there might be economies of scale. I do not know. But so far, it’s been the case that training a new model has been more expensive than training the previous model. So you’re in a race. All the money you make, you have to reinvest. If you stop and do nothing, the Chinese will catch up with you and kill your margins.

And now, as of last week—or the last two weeks—you have Elon Musk and Zuckerberg smelling your margins and deciding, ‘Hey, maybe we should get in the game. Maybe there are margins. Maybe we sell compute and inference.’ And then they’re going to kill your margins. If they can catch up and bring out a frontier model with the same capabilities as you, or even close, they can kill your margins.

And so, you’re a model house. Every day you wake up, everybody’s out to kill you. Your customers are this close to jumping ship. And you have to make money on inference. And you have to train a new model.

But here’s another thing going on if you’re a model house. I think Anthropic said this publicly, even—or others have said this. These model houses: Anthropic went from, what was it last year, 10 million ARR to, what is it now, 50, 60 billion? They said if they had more compute, they could probably double their ARR.

It’s just the demand. They cannot keep up with demand. There are rumors going around that they were close to having to shut off signups because they couldn’t serve new users. All that Fable stuff and Mythos stuff—it’s because they cannot serve these models. At least that’s my opinion, right? I don’t have confirmation for this. But look, they said, ‘Oh, we’re going to take Fable away,’ and now they’ve moved it two or three times, right?

But the problem is, your users want to use your models. So you’ve got to serve them compute. But you have a finite amount of compute, and all the compute you give to your users to run inference on is compute you don’t have for training. But you need to train, because if you make money now by serving inference, you’re losing out on training the next model. So in four months, the party is over, because your competitor has trained the next best model or the Chinese have caught up with you and killed your margin. That is crazy.

So a lot of the stuff that you see—the rate limits, the subscriptions, the subsidizing, the, you know, buying users, let’s call it this, right?—all of that is going on because this is a race. And this is a race where the margins are constantly shrinking. In the rearview mirror, there’s Musk and Zuckerberg and the Chinese companies now, trying to take your profits. You’ve got to go as fast as possible: train the next biggest model and model and model.

At the same time, you’re now thinking, well, if you’re a model house, the models are a commodity. So we’re going to go vertical, right? We’re going to build applications, get users in there, so then we’re not a commodity anymore. Now they’re all trying to do this. They’re all trying to get you to stay in their applications and spend money there, and not switch to other model providers.

But obviously, as a user, the best thing to do is to have applications where you can switch models. Because, as we’ve seen in even just the last year, the number-one spot always goes back and forth between these model houses. And, you know, everybody else is catching up.

So I think this is crazy. I think this is crazy. And people who think, ‘Oh, the companies aren’t profitable’—I don’t think they understand all of the forces at play. But I do think it’s important to know them.

Because imagine—and some of you don’t have to imagine; some of you can remember—but imagine it’s 1998, right? You have, like, what, a Pentium III or whatever, 900 megahertz, I don’t know. And you’re working on a game engine, a AAA game engine, and you know the development is going to take two or three years.

Are you going to target the hardware constraints of now, or are you going to target the hardware that will be out in one year or two years, or whatever? And back then, AAA gaming studios—I think they still do it—they were always developing for the next generation.

So there is a correlation: how hardware develops has had, and should have—and right now has—an influence on what you should do in software. And I think it’s important, and for that reason I think it’s important, to know all of these forces at play and what’s going on. And, you know, what it means when people say, ‘Oh, inference isn’t profitable,’ which it isn’t—sorry, it is profitable.

But when people talk about rate limits and whatnot, you need to know that they need data centers. They need energy. There aren’t enough data centers coming—and not enough copper coming out of the ground right now—to serve all of the users. They’re in a race. They need to train the next model. It’s always bigger models. All the profits you make are going to be eaten up. It’s a commodity. You have to fight against being commoditized. They know this. Hopefully we all know this.

And it’s pretty wild. So I thought I’d get this out there on my post-lunch walk. And, yeah, I’ll talk to you.