Two things happened. Open-weight models got close enough to matter, with permissively licensed releases scoring competitively on coding benchmarks, which removed the "it is not good enough" objection for a large class of ordinary tasks. And the economics got loud: as we wrote in the gross-margin piece, AI-first products run twenty to sixty percent gross margins against seventy to ninety for classic SaaS, and the difference is a per-token bill that scales with success.
Put those together and self-hosting looks like the obvious lever. You have a cost that grows with usage, and an alternative where the cost is fixed. That is a real and legitimate thing to investigate. It is also where most of the analysis stops.
Self-hosting does not remove the cost. It converts a variable cost into a fixed one, plus a permanent engineering commitment. Whether that trade is good depends almost entirely on your utilisation curve, which most teams have not measured.
The arithmetic people actually do
The spreadsheet is usually: monthly API spend, versus monthly cost of enough GPU to serve our traffic. If the second number is smaller, self-host. This comparison is not wrong so much as incomplete in a specific way that matters.
An API bill is proportional to use. A GPU bill is proportional to time, and it does not care whether you are using it. Your API cost at three in the morning on a quiet Sunday is close to zero; your GPU cost is exactly what it is at peak on Tuesday. So the comparison is only favourable at high, and crucially steady, utilisation.
Most products do not have steady utilisation. They have business-hours traffic in one or two time zones, weekday peaks, and long troughs. If your load varies by five times between peak and trough, you must provision for the peak and pay for it through the trough, and the effective cost per token during the quiet hours is dreadful. Teams that model this properly frequently discover the crossover point sits at a volume several times higher than their current traffic, which turns "we should self-host" into "we should revisit this at ten times the scale."
The costs that never make the spreadsheet
Somebody now operates an inference service. That means capacity planning, batching and queueing under load, monitoring, upgrades, GPU driver problems at unsociable hours, and a paging rotation. For a small team this is not a background task; it is a meaningful fraction of one engineer, permanently. Priced at what that engineer costs, it often exceeds the API bill it was meant to replace.
Model updates become your project. On an API, a better model appears and you change a string. Self-hosted, every upgrade is an evaluation, a migration, a redeployment, and a rollback plan. Given how quickly this field moves, that is not a one-off; it is a recurring tax on staying current, and the alternative is quietly falling behind while your competitors change a string.
You lose elasticity exactly when you need it. The day a launch goes well is the day API-based inference simply absorbs the traffic and self-hosted inference queues. The failure mode of the cheaper option shows up precisely at your best moment.
The capability gap is real but narrow
It is worth being precise here rather than tribal. For a large class of work, classification, extraction, summarisation, routine generation, straightforward code, open-weight models are genuinely good enough, and anyone claiming otherwise has not tested recently.
The gap shows up at the hard end: long-horizon agentic work, complex multi-step reasoning, the tasks where the frontier models have been improving fastest. It also shows up in the surrounding platform, which is easy to forget when comparing raw model quality. Prompt caching, batch processing, structured outputs, server-side tools, and the operational machinery around them are part of what you are buying, and reimplementing that stack is a larger project than serving the weights.
Three cases where it wins
One: you cannot send the data out. This is the strongest case and the one that actually decides it, because it is not a cost question. If regulation, contract, or client requirement means the data cannot leave your infrastructure, self-hosting is not the cheaper option, it is the only option. Note this is the same class of constraint we hit in choosing between hosted models: data handling rules decide architecture more often than capability does.
Two: high, steady, predictable volume on a narrow task. If you run one well-defined workload continuously at scale, classification over a firehose, embedding a large corpus on a schedule, the utilisation curve is flat, a smaller specialised model is sufficient, and the arithmetic genuinely works. Note how specific this is: continuous, narrow, and well-defined.
Three: latency or locality requirements an API cannot meet. On-premise deployments, edge inference, air-gapped environments, or hard latency budgets that a network round trip cannot satisfy. Again, not a cost argument.
Notice that only one of the three is about money, and it is the most constrained. That is the actual finding: self-hosting is usually the right answer for a reason other than cost, and when cost is the only reason it is usually the wrong answer.
The middle path most teams should take
Before rebuilding your inference layer, exhaust the levers that do not require operating one. Route by difficulty: send the easy majority of requests to a smaller, cheaper hosted model and reserve the expensive one for the fraction that needs it. This alone typically removes more cost than self-hosting would, at a fraction of the effort.
Then cache aggressively, cap runaway agent loops, and trim context that is padding rather than signal. Tune the effort or reasoning settings your provider exposes, which on current models moves cost substantially and costs nothing to try. Do all of that, measure again, and if the bill is still the dominant line in your cost of goods sold, you now have a real case for self-hosting rather than a hunch, and you will make the decision with a properly measured utilisation curve instead of a peak-hour extrapolation.
A hybrid is also legitimate and underrated: self-host the one narrow high-volume workload where the arithmetic works, keep everything else on an API. That gets most of the saving without making inference operations your whole engineering identity.
The bottom line
For most teams the answer is no, and the reason is not that open models are bad. It is that self-hosting trades a variable cost you understand for a fixed cost plus an operational commitment you probably have not priced, and the trade only pays at steady utilisation most products do not have. Do the cheaper optimisations first, because they usually recover more margin than the rebuild would. Reserve self-hosting for the cases where something other than cost is forcing it: data that cannot leave, a narrow workload running flat out, or a latency requirement no network call can meet. Those are good reasons. "Our API bill is large" is a reason to go and measure, not a reason to go and build.