What BiOS Adaptive Actually Decides

Every request to BiOS Adaptive is actually two decisions: which model should answer, and what the answer should cost. The caller sends one request to one endpoint; the router chooses the model, and the choice is invisible unless you look at the response metadata.

The promise is the one the hard-coding post argued for from the other side: stop pinning a model ID, and let the routing layer pick the best model for each request. The caller's code stops being a model catalog and starts being a task description.

This post is the inside of that decision: what the router weighs, how the choice is made, and where the router's knowledge ends. It is the platform post for the routing layer — the same honesty standard as the rest of the library, applied to the decision engine.

Three things, in one pass. The model: which of the served models answers this request. The routing is the headline — the same task can land on different models as the request mix changes. The parameters: the settings the request runs with, chosen to match the model and the task. And the price: the effective rate for this request, which follows from the model choice and is published in the model library.

The decision is per-request, not per-account: two requests from the same caller can route differently, because the requests are different. That is the entire point — the routing granularity is the request, not the subscription.

And the decision is observable: the response carries which model answered, so the caller can audit the routing against its own expectations. The router is a black box only if you never look at the metadata.

The router weighs three things. The task: what the request is asking for — the router reads the request and matches it against the strengths of the served models, so a reasoning-heavy request and a formatting-heavy request do not land on the same model by default. The cost: the per-token rates of the candidates, because the routing optimizes for cost as well as quality — the same cost-per-task discipline this blog applies everywhere, built into the routing layer. And the catalog: the current served models, their versions, their availability — the router routes within what the library actually serves today.

The balance between quality and cost is the product decision, and it is tuned, not fixed: the routing favors the model that fits the task at the rate that fits the workload. The tuning is the difference between a router that saves money and a router that downgrades answers.

And the catalog input is why the router stays honest: when a model leaves the library, the router stops routing to it the same day — the versioning post's lifecycle, enforced by the routing layer.

You can — the hard-coding post's whole argument is that the model choice is yours to make, and a hand-built router is a legitimate architecture. The question is what the hand-built version costs to keep current: the model catalog changes, the prices move, the strengths shift — and the hand-built router's rules are only as fresh as the last time someone updated them.

The managed router's advantage is the maintenance: the routing logic tracks the catalog and the pricing automatically, so the caller's code does not. The eval post's discipline — re-evaluate when the models change — is built into the routing layer instead of the team's calendar.

And the honest boundary: the managed router is a default, not a mandate. The caller that wants a specific model for a specific task can pin it — the routing layer is the baseline, and the pin is the override. The router earns its keep on the requests where the caller has no strong opinion.

The router reads the request; it does not read the product. It cannot know your quality bar, your brand voice, your user's history — the context that lives in your application, not in the request text. The routing optimizes for the task as stated; the product's preferences are the caller's to enforce.

It also cannot know the future: the request that looks simple and turns out to need deep reasoning is routed before the depth reveals itself. The routing is a bet made on the request's opening, and the bet is sometimes wrong — which is why the metadata matters, and why the pin exists.

And it cannot know your budget the way you do. The routing optimizes cost per request within the catalog's rates; the budget ceilings, the spend alerts, the workspace limits — those are the guardrails layer's job, and they apply on top of the routing, not instead of it.

When the model choice is the product. A team that has evaluated, fine-tuned, and pinned a specific model for a specific reason should pin it — the router's generality is a downgrade for a decision that was made deliberately. The routing layer is for the requests where the choice is not the point.

It is also the wrong tool when the workload needs a guarantee the routing cannot make: a fixed model version for compliance, a fixed price for a contract, a fixed behavior for a certification. The router optimizes; it does not guarantee — and the workloads that need guarantees need pins.

And it is the wrong tool when the caller wants to learn the models themselves. The eval post's whole discipline is knowing your models by testing them; a team still building that knowledge should route manually and watch where the requests land. The router is for after the learning, not instead of it.