There is no single cheaper option: self-hosting is usually cheaper only at sustained, high utilization, while APIs win for spiky or low-volume use. The break-even is the monthly token volume at which your fixed GPU cost divided by your cost per token under self-hosting equals the API's per-token price. In practice, self-hosting tends to break even in the range of tens of millions of tokens per month for a mid-sized open model on a rented GPU—an estimate that swings widely with model size, hardware price, and how busy the GPU actually stays.
The Two Cost Structures: API Tokens vs. Owned GPUs
An API (application programming interface—a hosted service you call over the network) charges per token: a token is a chunk of text, roughly three-quarters of a word in English, and you pay separately for input tokens (your prompt) and output tokens (the model's reply), with output usually priced higher. The cost is purely variable: zero traffic means zero spend, and someone else owns the hardware, the operations, and the scaling. Your bill scales linearly with usage, and the per-token price is fixed by the provider.
Self-hosting means running the model on hardware you control—either a GPU you bought or a GPU you rent by the hour from a cloud provider. A GPU (graphics processing unit) is the chip that does the matrix math behind an LLM. Here the cost structure flips: most of the cost is fixed (the GPU, its power draw, cooling, and the machine around it), and the variable part is small (electricity per hour, plus your time). You pay whether or not a request arrives, so the economics depend on spreading that fixed cost over as many tokens as possible.
The two structures cross over because one is linear from zero and the other starts high but flattens per token. At low volume the API's line is below the self-hosting line; at high volume the self-hosting line's shallow slope wins. Utilization—the fraction of time the GPU is actually doing useful work—is the variable that decides where that crossing sits.
Building the Break-Even Formula
Start with the self-hosted cost per token. If a GPU costs C dollars per month all-in (rental or amortized purchase plus power and cooling) and serves T tokens per month, then:
self_hosted_cost_per_token = C / T
The API cost per token is a single price P. Break-even is where the two are equal:
C / T = P => T_breakeven = C / P
So the break-even token volume is just your fixed monthly cost divided by the API's blended per-token price. "Blended" matters: if a workload is, say, one part output to three parts input and output is priced higher, compute P as the weighted average for your actual input/output mix rather than picking one number.
Two refinements make the formula honest. First, capacity: a GPU can only produce so many tokens per hour, so T cannot exceed throughput × hours × utilization. If T_breakeven exceeds what the GPU can physically serve, self-hosting never breaks even at that utilization. Second, add the API's non-token costs (none, typically) versus self-hosting's variable extras—electricity per hour and engineering time—as an add-on V per month:
T_breakeven = (C + V) / P
The formula's power is that it exposes the real lever: C is fixed, so break-even is entirely about pushing T (and therefore utilization) high enough.
Worked Example: Monthly Tokens vs. GPU Hours
Assume you serve a 7B-parameter model (7 billion weights — small enough to fit on a single 24 GB GPU when quantized) and your workload is 150 million input tokens plus 50 million output tokens per month. For a rough estimate, assume self-hosting throughput of about 1,500 output tokens per second at batch size 8 on one rented GPU at $1.50/hour (a reasoned estimate for a mid-tier cloud GPU; check current rates).
Compute time, using output tokens as the bottleneck and input prefill folded in at roughly 10% extra:
output_seconds = 50,000,000 / 1,500 = 33,333 s
prefill_seconds = 0.1 * 33,333 ≈ 3,333 s
total_hours = (33,333 + 3,333) / 3,600 ≈ 10.2 h
At $1.50/hour, that is about $15/month of raw GPU time — which looks absurdly cheap. But you cannot rent 10 hours and stop; you pay for wall-clock uptime. If the service must be available 24/7, you pay 730 hours:
730 h * $1.50 = $1,095/month
So the honest comparison is $1,095 (always-on) vs. your API bill. If a hosted API charges, say, $0.50 per million input tokens and $1.50 per million output tokens (illustrative), your monthly API cost is:
(150 * 0.50) + (50 * 1.50) = 75 + 75 = $150/month
Here the API wins by roughly 7x. Self-hosting only beats it if utilization is high enough that the fixed GPU cost is spread thin — or if you already own the hardware.
Hidden Costs and When Self-Hosting Wins
Raw GPU rental is the easy number. The costs that flip the decision:
- Idle time. You pay for uptime, not tokens. Low-traffic services almost always lose to APIs.
- Engineering labor. Deployment, quantization, batching, monitoring, and upgrades are ongoing work. Even a few hours a week at a loaded rate can exceed the API bill.
- Cold starts and scaling. Autoscaling GPUs is slow; keeping a warm spare doubles fixed cost.
- Memory and context limits. Long contexts consume VRAM, forcing bigger GPUs or fewer concurrent requests.
- Redundancy and ops. Backups, failover, logging, and security are yours to build.
- Egress and storage. Model weights and logs cost money to move and keep.
A quick break-even check:
break_even_tokens = fixed_monthly_cost / api_cost_per_token
Self-hosting tends to win when: utilization is steady and high, data cannot leave your infrastructure, you already own GPUs, or you run many small models where per-token API pricing adds up. It usually loses for spiky, low-volume, or fast-changing workloads — where the API's flexibility is itself the cheaper feature.
Frequently Asked Questions
What is the break-even point between self-hosting an LLM and using an API?
The break-even point is the monthly token volume at which your total self-hosting cost equals your API bill for the same workload. In practice it is a range, not a single number, and it shifts with GPU rental or purchase price, electricity, utilization, and how efficiently you serve the model. As a rough estimate: a rented GPU at a few dollars per hour serving a small or quantized model can beat per-token API pricing somewhere in the hundreds-of-millions-of-tokens-per-month band, while large models push the crossover much higher — sometimes beyond what most teams ever consume. Do the division yourself with current quotes rather than trusting any single figure.
Which costs are easy to forget when self-hosting an LLM?
People remember the GPU but forget almost everything around it: idle capacity you pay for during nights and weekends, the engineering time to deploy, monitor, patch, and debug the serving stack, storage for model weights and logs, egress bandwidth, redundancy if you need a second replica for uptime, and the cost of underused memory when a model barely fits. Also easy to miss: the API you still pay for fallbacks, evaluation, or burst traffic, and the opportunity cost of your team maintaining infrastructure instead of building product. A realistic estimate adds 30–100% on top of raw GPU cost, depending on how much of this you already have in place.
How does GPU utilization change the break-even calculation?
Self-hosting costs are mostly fixed per hour, while API costs scale with tokens, so utilization is the variable that decides everything. If a GPU sits idle 80% of the day, its effective cost per token is roughly five times higher than at full load — meaning your break-even volume climbs sharply. High, steady utilization (batching many requests, running background jobs on the same hardware, or sharing across teams) pushes the effective per-token cost down and makes self-hosting attractive at lower volumes. Estimate your realistic average utilization, not your peak, before comparing.
Does model size affect whether self-hosting is cheaper?
Yes, strongly. Bigger models need more GPU memory and more compute per token, which raises both the hardware you must rent or buy and the energy per request, so self-hosting them costs more per token and the crossover volume rises. Small or quantized models (weights stored in lower precision to save memory) can run on modest, cheaper GPUs and often reach break-even far sooner. Meanwhile, APIs typically price large models at a premium too, so the comparison isn't automatic — but as a rule, the smaller the model and the more concentrated your traffic, the more likely self-hosting wins.
Can I estimate API costs before committing to self-hosting?
Yes — start by measuring tokens per request and requests per month from logs or a small pilot, then multiply by the provider's per-input and per-output token prices (output tokens usually cost more). Add a buffer for retries, longer prompts, and future growth, and run the same workload through a test batch to validate your token counts. That estimate becomes the "API side" of your break-even formula; pair it with a self-hosting estimate built from GPU hours, utilization, and the hidden costs above. If the two numbers are within roughly 2x of each other, treat it as too close to call without a short trial.