How to Read an LLM Price List

Every inference provider publishes one, and at first glance they all look the same: a model name, an input price, an output price, per million tokens. It reads like a spec sheet, and teams routinely treat it like one — sort by the headline number, pick a winner, move on.

That is how budgets go wrong, because the headline number is the beginning of the price list, not the price. The real bill depends on which row of the list your traffic actually lands on, and modern price lists have more rows than they used to. Cached tokens, batch tiers, context-length surcharges — each is a different price for the same model, and the difference between them is larger than the difference between most providers.

This post is a walking tour of the rows. It is deliberately provider-neutral: every example pattern below exists on multiple real price lists, including ours. The goal is that the next time you open one — ours included — you read it like a contract instead of a menu.

A token is not a word. Models read text as subword pieces — common words whole, rare words chopped into fragments, punctuation and whitespace counted too. English prose averages out to something well under a token per word, but code, non-English text, and formatting-heavy content all tokenize denser, sometimes much denser.

The catch for comparison shopping: tokenizers differ across model families. The same paragraph does not produce the same token count on every model, so "per million tokens" is not quite the same unit on two different price lists. For rough budgeting this hardly matters. For a close pricing decision between two providers, run your actual prompts through both tokenizers before you trust the arithmetic.

The practical habit: whenever a price list says a number per million tokens, translate it into your own units — per ticket resolved, per document summarized, per page extracted — before comparing anything.

Because they are different workloads wearing the same API. Reading your prompt is parallel work: the model processes the whole input at once, and the hardware chews through it efficiently. Writing the response is sequential: every output token is generated one at a time, each depending on all the ones before it. Sequential work keeps expensive hardware busy for longer per token, so output costs more — typically several times the input rate, on nearly every price list in the industry.

The design implication is real: your response-length discipline is a cost lever that your prompt-engineering discipline is not. A system prompt trimmed to half its length saves the input rate. A response format that answers in half the tokens saves the output rate. On a well-designed workload the output row of the price list matters more than the input row, and teams shopping by the input price are optimizing the cheaper half of the bill.

When you model a workload, estimate input and output volumes separately. A chat product and an extraction pipeline can have identical total token counts and wildly different bills, purely because of which direction the tokens flow.

Some price lists carry a discounted row for cached input tokens, and it is the most under-read row in the industry. The idea: if the beginning of your prompt is byte-identical across many requests — a system prompt, a policy document, a knowledge base extract — the provider can reuse the computed state for that prefix instead of reprocessing it. Work avoided is priced lower, so cache reads bill at a fraction of the standard input rate where the tier exists.

Whether you benefit is an architecture question, not a pricing question. Workloads with a long, stable prefix and a short variable tail — document Q&A against a fixed corpus, a support bot with a large policy prompt — are cache-shaped. Workloads where every request is novel text gain nothing.

Two things to check on any price list: whether a cached tier exists at all, and what the design constraints are. Prefix caching only helps if the identical content comes first, which makes prompt layout a billing decision. That rabbit hole is deep enough to deserve its own post; for now, when you read a price list, find the cache row before you find the headline.

Real-time pricing buys immediacy: your request jumps the queue and the answer starts streaming in a heartbeat. Batch and flex tiers sell the same computation with the urgency removed — submit work now, collect results later, pay less for the privilege of being interruptible.

The discount exists because deferrable work is worth more to the provider than to you: it fills idle capacity that would otherwise earn nothing. A support report generated at midnight and read at nine in the morning has no business paying real-time rates, yet a surprising share of production traffic is exactly that — batch-shaped work on an interactive price tier, because nobody asked the deadline question when the pipeline was built.

When you read a price list, mark which models offer a deferred tier and file it next to your workload inventory. Anything in your product with a deadline measured in hours rather than heartbeats is a candidate.

Long context is not free for the provider: attention over a very large prompt costs memory and time that short prompts do not. Some price lists reflect this directly, with different rates for the same model depending on how large the context window is, or surcharges that kick in past a threshold. Others bake it into separate model SKUs — the standard version and the long-context version, at different rates.

The trap is paying long-context prices by accident. If your integration defaults to the biggest available window "just in case", you may be buying the expensive row of the price list for traffic that never uses it. Check what context size your requests actually need, check what the tier boundaries are, and make sure the model id you call maps to the row you think you are paying.

This row of the price list is also the one most likely to change as techniques improve, so a decision made a year ago deserves a re-read.

The free tier is the most honest row on any price list, because its purpose is evaluation. Use it that way: run your real prompts against the models you are actually considering, measure what you would have paid, and note the rate limits you hit. A free tier tells you nearly everything about the API surface and nothing about the production experience — throughput, priority, and support are all different above the waterline.

Credits are the same instrument with a clock attached. They are excellent for funding a proper evaluation and terrible as a selection criterion: a large credit balance with a short expiry manufactures urgency, and urgency is how checklists get skipped. Note the expiry, run the trial at your own pace, and treat the credit as consumed learning rather than a discount on the decision.

The one thing free tiers cannot tell you is the production bill, because free-tier traffic is rarely shaped like production traffic. That translation — your volumes, your input-output split, real rates — is again a job for the calculator rather than the price list alone.

Three rules, all learned from watching comparisons go wrong.

Compare model by model, never blend by blend. An average across a catalog mixes flagship and economy tiers into a number no customer is ever charged. The only honest comparison is the same model, or the same capability tier, on both lists.

Weight by your workload, not by the provider's defaults. A price list that wins on input rate loses if your bill is mostly output; a cheap standard tier is irrelevant if your traffic needs the long-context row. The unit of comparison is your monthly volume split by direction, tier, and cache eligibility — run through both lists.

Date everything. Price lists move, and a comparison without an as-of date is a rumor. Our pricing calculator exists precisely because this arithmetic is tedious to do by hand: your volumes in, a model-by-model breakdown out, from the same published rates shown on the model library.

When your spend is small, the price list is a distraction. The difference between two reasonable providers on a modest bill is lunch money; the difference between a clean architecture and a wasteful one is not. Retries you never noticed, prompts padded with context nobody reads, non-production traffic on production tiers — these dwarf any rate-card optimization, and none of them appear on a price list.

The price list is also the wrong tool for the quality question. A cheaper model that fails a fifth of your requests costs more per finished task than the expensive one that passes first try, and no amount of rate-card reading will tell you which is which. Measure on your workload; buy on the measurement.

Read the price list last, in other words — after you know your volumes, your shape, and your quality floor. Then it is a fifteen-minute exercise instead of a leap of faith, and the calculator will do the arithmetic for you.