Serverless, Adaptive, or Dedicated: Picking How Your Models Run

Every inference workload on Run BiOS runs in one of three modes, and the whole decision fits in a paragraph: serverless is shared capacity billed per token, with nothing to provision; BiOS Adaptive is one endpoint that answers each request at the depth the task demands, billed inside a published band; dedicated is your own GPU serving your own model — stock or fine-tuned — billed per second of capacity.

Same OpenAI-compatible API across all three. Same model catalog behind the first two. The choice is not about API shape or capability — it is about how you want to buy compute, and how much of the sizing question you want to own.

This post is the orientation we give every new team, written down: what each mode is for, what it costs you in the currencies that are not money, and the order in which most products graduate through them.

Serverless is where every workload should start. No capacity planning, no idle cost, no minimum: the bill is a pure function of tokens served, and the pricing calculator turns any traffic estimate into a monthly figure directly. The full catalog is available behind one API shape, which is what makes a two-line provider migration real rather than aspirational.

What you give up is control over the hardware story. Shared capacity has shared rhythms — the latency and rate-limit posts cover the honest edges — and at high steady volume the per-token rate stops being the cheapest way to buy the same compute.

The failure mode to avoid is treating serverless as a phase you are embarrassed to still be in. Plenty of serious products run on it permanently, because their traffic is spiky, their team is small, or their time is worth more than the savings from owning the next layer down.

Adaptive exists for the workload almost every product actually has: a mix of trivial prompts and genuinely hard ones, where the right price-quality trade-off differs request by request. You call one endpoint with one model id; each request is answered at the depth the work demands; the rate you pay varies inside a published band, so your bill tracks the difficulty of your traffic instead of the strength of your default.

The full pitch, the band, and the honest fork between pinning and Adaptive are on the Adaptive page — the short version is that you stop pinning, and the price of each answer reflects what that answer took.

The honest boundary, from the same page: if your workload is uniform, or a contract needs one guaranteed per-token rate, or you are benchmarking one specific model, pin instead. Adaptive is the answer to mixed traffic, not to all traffic.

A dedicated endpoint is your model on your GPUs: predictable latency, throughput you sized yourself, a private serving of the exact weights you chose — including a model you fine-tuned with us, whose weights remain yours. Billing is per second of allocated capacity, which changes the shape of every decision: utilization is your problem and your lever, and the marginal request is free.

The crossover from serverless is arithmetic, not vibes — steady utilization, the hidden second workloads, and the inflection math are worked through in the break-even post. The operational setup lives in the deployments docs.

Dedicated is also the compliance answer. When the policy question is "whose hardware touched this data", a dedicated endpoint is the sentence that ends the meeting.

The path we see working, over and over: start serverless, because nothing else respects an unproven workload. Add Adaptive when the traffic is visibly mixed and the default model is clearly wrong for part of it — the hard-coding post is the diagnostic. Graduate specific workloads to dedicated when their utilization curve, not their prestige, justifies it. And fine-tune when the behavior gap is the thing holding quality back — at which point dedicated serving of your own weights stops being an extravagance and starts being the point of the platform.

The modes compose; they do not compete. A typical mature account runs its bulk traffic through Adaptive, a validated workload on a dedicated endpoint, and experiments on serverless — each workload bought in the currency that suits it.

What we would discourage is the skip: provisioning dedicated capacity for a workload you have not measured yet. Measure on serverless, prove the shape, then buy the machine.

If you want the short version, it fits in four questions, answerable from a week of traffic logs:

• **Is the workload proven?** No: serverless. Unproven workloads deserve pay-per-use, not capacity.
• **Is the traffic mixed?** Easy prompts beside genuinely hard ones, with no clean line you can code yourself: Adaptive. That is the workload it exists for.
• **Is the utilization curve steady and yours?** High, flat, and predictable: dedicated, and the crossover math in the break-even post will confirm it.
• **Is there a compliance sentence you must be able to say?** "Our data is served on our GPUs" ends a category of meeting, and only dedicated says it.

Most workloads answer one question clearly and stop there. If yours answers two, split it — the modes compose, and the API does not change between them.

Rarely, which is the point of offering three. If you cannot articulate why a workload needs dedicated capacity, it does not. If you cannot describe the mix in your traffic, you have not logged it yet, and the logging comes before the mode decision.

The genuinely hard case is the boundary one: steady enough that serverless margins sting, spiky enough that dedicated utilization would sag. That is exactly where Adaptive or a hybrid split earns its keep — urgent traffic stays interactive, the patient mass waits, and the batch post has the split playbook.

And if you get it wrong, the same API shape means the correction is configuration, not a rewrite. Choose cheaply, measure honestly, and let the workload promote itself.