Self-Hosted LLM Cost: A Break-Even Model You Can Run in a Spreadsheet

Why are both confident answers wrong?

Because neither is an answer to a question that was stated precisely.

"Self-hosting is cheaper at scale." Only if the GPUs are busy. A cluster sized for peak load and idle at night charges you for capacity that serves no tokens, and the cost per token you actually served can be several times the price on the invoice.

"APIs always win." Only if you ignore everything except the token price. Data residency, per-tenant rate limits, weight-level control, quantisation, latency floors and the ability to pin a version for a regulated audit are not features an API can sell you at any price.

The break-even is not a volume. It is a utilisation curve: the moment your GPUs serve enough tokens per hour to cover their own cost plus the labour of operating them.

I have run inference both ways — rented endpoints and GPUs I had to keep alive — and the arithmetic below is the one I would put in front of a CTO. Every input is a placeholder, because your duty cycle and your loaded rates are yours; mine would only mislead you.

What are the six lines?

The token price is one of them and rarely the decisive one.

Two of these deserve emphasis. Throughput on your workload is not a benchmark number: batching behaviour, context length, output length distribution and quantisation move it by multiples, and a doubled throughput halves the cost per served token. Operations labour is the line that settles the decision in most organisations, and it is usually between a fifth and a half of a full-time engineer once re-qualification is counted.

What is the equation?

self_hosted_cost_per_million_tokens =
    ( GPU hours per month × blended hourly cost
    + ops hours per month × loaded rate
    + amortised hardware, if owned )
    ÷ ( millions of tokens actually served per month )

And the comparison is against the API's blended price for your input-to-output ratio — which is rarely the headline price, because input tokens are cheaper and your workflow's ratio is specific.

Worked structurally, with placeholders:

The structure is what matters. The GPU line is hours × rate, and the divisor is tokens served, not tokens provisioned — so every hour of idle capacity raises the unit cost of the hours that worked. The ops line is fixed, which means it dominates at low volume and becomes irrelevant at high volume; that asymmetry is why the same organisation can be right to self-host one workload and wrong to self-host another.

Where does the break-even actually sit?

Set the two unit costs equal and solve for volume:

break_even_tokens = fixed_monthly_cost
                    ÷ ( API price per million − marginal cost per million )

Three properties are worth carrying into a meeting.

It is a volume and a utilisation statement. Fixed cost includes the GPU hours you pay for whether or not they serve traffic. Raise the duty cycle and the break-even volume falls; halve it and the break-even may move beyond any volume you will ever reach.

The marginal cost per million is not zero. It is the GPU hours consumed by those tokens at your realised throughput — which is why "the hardware is already bought" is not an argument. Sunk cost does not reduce the cost of the next million tokens; it just hides it.

A spiky workload pays for its peak. If traffic is ten times higher for two hours a day, you provision for those two hours and average over twenty-four, and the effective unit cost can be several times the flat-traffic case. Autoscaling reduces this and does not remove the cold-start penalty that lands on your latency SLO. Where the workload is genuinely batch — overnight document processing, bulk classification — the duty cycle can be driven close to one, and that is where self-hosting earns its keep.

What decides it before the arithmetic?

Four constraints can end the discussion before a spreadsheet opens, in both directions.

The fourth row is where the argument most often ends, and it is worth being honest about it: if the task needs frontier-level reasoning, the choice is not self-hosting against an API, it is a smaller open model doing less well. The economics are irrelevant if the system does not pass the evaluation set.

What does operating it actually require?

The capability list is short and non-negotiable. In an organisation where none of these exist today, self-hosting is not a cost decision — it is a hiring decision wearing a cost decision's clothes.

If the fourth row cannot be staffed, the model should account for it anyway: an inference incident at peak is the moment a self-hosted decision is judged, and the judgement happens in the incident channel rather than the spreadsheet.

When is self-hosting clearly right?

Four cases where I would not argue against it.

Residency is a hard requirement. Data that cannot leave a jurisdiction or a private network makes the API option unavailable rather than expensive, and the comparison becomes self-hosting against not doing the work at all.

Sustained, batchable, high-volume utilisation. Overnight processing that can keep GPUs genuinely busy gives you a duty cycle near one, and with a sufficient open model this is the canonical case where the arithmetic works.

Weight-level control matters. Fine-tuned or quantised weights you own, a version you must pin for audit, or a distillation pipeline are capabilities an API cannot sell.

The infrastructure already exists. A team already running GPU workloads for training or rendering adds marginal inference capacity cheaply; a team with no GPU operations is starting a new function, which is a different proposal.

When is it clearly wrong?

Low or spiky volume. If utilisation is a fraction of provisioned capacity, the unit cost lands above the API price before the ops line is counted.

No owner. Self-hosting without a named person who owns the serving stack is a decision that will be reversed by an incident rather than a review.

To save money you cannot measure. "Cheaper" without a tokens-served denominator is an assertion. Build the per-token model first, then the break-even.

Because the data "feels" sensitive. Sensitivity is answered by a subprocessor list, a retention schedule, a region and a contract. If those are adequate, the residency argument dissolves — and if they are not adequate for an API, they are probably not adequate for your own hosting either.

The spreadsheet to keep

Five columns, twelve months, so a decision made once can be revisited with evidence rather than opinion: tokens served · GPU hours provisioned · duty cycle · ops hours · fully loaded cost per million served. Re-run it quarterly next to cost per completed task, which is the number the business actually feels.

One asymmetry is worth writing into the decision record. Moving from self-hosted to an API is a configuration change and can be done in a week; moving from an API to self-hosted capacity is a project with a hiring dependency. When the arithmetic is close, that asymmetry is a legitimate reason to buy the reversible option — and the same reasoning applies to the rest of the AI stack, where the cheapest answer is usually to own the boundary and rent the capacity.