Introduction
"Self-hosting is cheaper" is one of the most expensive assumptions in AI infrastructure. It starts from a real number — an H200-class GPU rents for around $3-4 an hour, so surely running your own model beats paying an API markup? — and then ignores everything else that self-hosting actually requires. Sometimes self-hosting genuinely wins. But the honest answer for most teams in 2026 is: a managed API is cheaper and faster to ship until your volume gets very large or your privacy requirements leave you no choice.
This guide gives you the real cost model, the break-even points that matter, a worked example, and the three reasons to self-host that have nothing to do with saving money. Whether you're weighing running an open-weight model like DeepSeek V4 or GLM-5.2 yourself versus calling an API, the framework below tells you where the line is.
The number that misleads everyone
The GPU rental rate is the seductive number. As of 2026, an NVIDIA H200 rents for roughly $3.10-3.80 per GPU-hour on-demand from the cheaper providers (more like $10+ on AWS or Azure). Multiply that out and a single GPU running full-time is around $2,200-2,700 a month, which can look like a bargain against a large API bill.
The problem is that GPU rental is the visible tip of the cost. The rate assumes 100% utilization, flawless operations, and zero human involvement — none of which is true. Budgeting from the GPU price alone is how self-hosting projects end up costing multiples of what the spreadsheet predicted.
The real cost of self-hosting
Once you account for everything, self-hosting costs roughly 3 to 5 times the raw GPU rental price. The additions:
- Engineering time. A self-hosted deployment conservatively needs 10-20 hours per month of a senior DevOps or ML engineer for maintenance, monitoring, and troubleshooting. At $75-150/hour that's $750-3,000/month in labor alone — often more than the GPU bill.
- Idle capacity. Real traffic is bursty. You provision for peak, so you pay for GPU-hours that sit idle between bursts. Effective cost per token is always higher than the utilization-100% math.
- Reliability and ops. Load balancing, failover, model updates, security patching, and the on-call burden when inference breaks at 2 a.m. API providers absorb all of this; self-hosting means you own it.
None of this appears on a GPU pricing page, which is exactly why it's so consistently underestimated.
Where the break-even actually sits
The crossover point depends entirely on what you compare against:
- Versus frontier closed models (the premium API tier), self-hosting on a reserved GPU can break even around 2-5M tokens per day. Frontier APIs carry high margins, so it doesn't take enormous volume to justify your own hardware.
- Versus cheap open-model API providers — the services that already host DeepSeek, Qwen, GLM, and friends on optimized infrastructure at thin margins — the break-even shifts dramatically higher, often to 50M+ tokens per day. They've done the scaling work you'd be duplicating, and they've done it more efficiently than you can at small scale.
That second point is the one most teams miss. You're rarely choosing between self-hosting and the expensive frontier API. You're usually choosing between self-hosting and a cheap open-weight API that runs the exact same weights you would — and beating them on cost is genuinely hard until you're operating at their scale. Published 2026 analyses put the premium-model crossover anywhere from roughly 100M to 256M tokens per month, and considerably higher against budget models.
A worked example
Consider a serious owned-hardware setup: two H100-class GPUs bought at $18,000 each, amortized over 36 months. The hardware component is about $1,000/month. Add electricity ($131/month for both at 500W average) and colocation plus power fees (~$700/month), and your fixed cost is roughly $1,800/month before labor — call it $2,500-4,000 all-in with the ops time.
Against a frontier API at the volume those GPUs can serve, the equivalent API spend might run $7,500-15,000/month, giving a break-even on the $36,000 hardware in roughly 6-7 months. That's a clear win — if you sustain the volume and if you're comparing against premium pricing. Swap in a cheap open-weight API as the comparison and the same setup may never break even. The example is real; the conclusion depends entirely on your two variables: sustained volume and what you benchmark against.
The three reasons to self-host anyway
Cost is often the wrong reason to self-host. These three are frequently the right ones:
- Privacy and data residency. The data never leaves your infrastructure or jurisdiction. For healthcare, finance, legal, and government workloads, this can be a hard requirement that an API cannot satisfy at any price. With permissive licenses like the MIT terms on DeepSeek V4 and GLM-5.2, you can run frontier-class models entirely inside your own boundary.
- Control and stability. No rate limits, no surprise deprecations, no model swapped out from under your evals, no vendor outage taking down your product. You pin a version and it stays pinned.
- Predictable economics at true scale. Above the break-even, fixed-cost hardware turns a variable, volume-scaling API bill into a flat line — which finance teams often prefer even when the average cost is similar.
A decision checklist
Self-host if you can answer yes to most of these; otherwise use an API:
- Is your volume high and sustained (not bursty, not occasional)?
- Are you comparing against a frontier API rather than a cheap open-weight one?
- Do you have in-house ML/DevOps capacity to own inference reliability?
- Do privacy or control requirements make an API a non-starter regardless of cost?
- Can you quantize to fit the model on hardware you can actually afford? (Quantization is the main lever for shrinking the GPU footprint.)
If you're not at high sustained volume and privacy isn't forcing your hand, the API almost always wins on total cost and time-to-ship. If you do want to experiment with local models first, tools like Ollama and LM Studio make it easy to try before you build — see our walkthrough of running Kimi locally with GGUF quantization.
Conclusion
The raw GPU price says self-hosting is cheap; the real cost model says it's usually 3-5x that, and the break-even is far higher than the naive math suggests — especially against the cheap open-weight APIs that most workloads should actually be compared to. Self-host when your volume is genuinely large and sustained, when you have the ops muscle to run inference reliably, or when privacy leaves you no alternative. Otherwise, a managed API is cheaper, faster, and less distracting. Decide on the honest numbers, not the GPU sticker price — and if you're choosing which open-weight model to run, start with our guide to the best open-weight LLMs for agentic coding.