from openai import OpenAI
client = OpenAI(
base_url="https://api.usbios.ai/v1", api_key="sk-bios-...",
)
resp = client.chat.completions.create(
model="<model-id>",
messages=[{"role": "user", "content": "Hello!"}],
stream=True,
)Inference is serverless by default. Scaling, traffic handling, and cost optimization happen automatically, including scaling to zero.
Inference runs serverless by default, with automatic scaling and request batching.
Dedicated GPU clusters ensure stable throughput under sustained load.
Scale from API-based inference to full GPU clusters without re-architecting your stack.
Custom models
Fine-tuning produces weights you own. A finished checkpoint can go straight onto a dedicated endpoint without leaving the platform.
Reached through the same OpenAI-compatible API as everything else. This is part of fine-tuning rather than the serverless product — different billing, and a different reason to reach for it.
A LoRA or QLoRA adapter is served with its base model automatically, with no manual merge step.
Chat, completion, embedding or reranker — each exposes the matching OpenAI route.
Light fits on the minimum that will hold the model; heavy adds GPUs for concurrency.
bf16 or fp8, with KV cache compression to fit more concurrent requests on the same card.
What it costs
BiOS saves you on deepseek-v4-pro
25%
vs Fireworks
25%
vs Together AI
Pick the model you use most
Choose a period, then a volume — or type your own
What share of your tokens are input rather than output
Based on 1 billion tokens/day · 700 million input, 300 million output
| Input / 1M | Output / 1M | Daily | |
|---|---|---|---|
| BiOS | $1.30 | $2.60 | $1,690.00 |
| Fireworks | $1.74 | $3.48 | $2,262.00 |
| Together AI | $1.74 | $3.48 | $2,262.00 |
Over an average month BiOS is $51,404 — $17,398 saved against Fireworks, and $17,398 saved against Together AI.
Fireworks Standard tier, as published 26 July 2026 · Together AI serverless, as published 26 July 2026.
How you pay
Serverless inference is metered per million tokens. Fine-tuning and dedicated endpoints are metered per second of GPU time. You are never charged both ways for the same work.
Rates vary by model — price yours in the calculator above. The rate for the exact model you are about to call is shown in your dashboard before you send the request.
The same rate applies whether you are running SFT, RLHF, VLM or continued pre-training. A stopped endpoint costs nothing while it is stopped.
Neither product can run up a debt. Both draw from one pre-paid wallet. If the balance reaches zero you get a grace warning, and a running endpoint pauses rather than accruing charges. Top up and resume.
Neither is automatically cheaper. It depends on your traffic shape and whether you need your own model.
We would rather say this plainly than have you discover it on an invoice. A dedicated GPU bills while it is idle, so stop endpoints you are not using.
Which one you need
Fine-tuning costs more than a serverless call and takes longer to get right. Plenty of workloads should start on serverless and stay there. Here is how to tell which side you are on.
A general-purpose model is already doing the job — or you do not yet know exactly what the job is.
A general-purpose model gets close, but is consistently wrong in a way you can describe.
To add facts the model should look up. Knowledge that changes belongs in retrieval, not in weights — bake it in and you retrain every time it moves. To rescue a prompt you have not iterated on. Prompt work costs minutes and nothing per attempt; training costs neither of those things. To win a public benchmark. A model that scores well on a leaderboard and badly on your own traffic has cost you money. Measure on your data, not on someone else's.
The API
The API is OpenAI-compatible. Point any OpenAI SDK at the base URL, pass your key as the bearer token, and set the model name. Switching models, or moving from serverless to your own dedicated endpoint, is a change of model id — not a rewrite. The serving engine covers four task shapes:
| Model type | Route | Best for | Streaming |
|---|---|---|---|
| Chat | /v1/chat/completions | Assistants, multi-turn conversation | Yes |
| Completion | /v1/completions | Base or completion-style models | Yes |
| Embedding | /v1/embeddings | Retrieval and similarity search | No |
| Reranker | /v1/rerank | Cross-encoder document scoring | No |
Tool support is read from the model's own chat template, not its name, so a custom fine-tune that speaks a known format gets it automatically. Once a tool request is accepted you always get an answer — if the output cannot be parsed as tool calls you get the raw text and an honest finish reason, never an error.
Structured output works on every chat model, independent of tool support. Pass a response_format and the engine constrains decoding, so you get valid JSON — or an exact match to your schema when you supply one.
Dedicated endpoints
When you do run your own dedicated endpoint, you get the operational surface that decides whether you keep it: what it costs, how fast it is, and why it restarted at 3am.
Requests, prompt and generated tokens, cache-hit rate, average time-to-first-token, end-to-end latency, aborted requests, uptime and spend — cumulative forever, carried across engine and pod restarts.
Requests per minute, tokens per second in and out, TTFT and latency percentiles, running and queued requests, cache-hit rate, GPU utilisation and memory. 1h, 24h, 7d and 30d windows.
A crashed engine restarts with backoff. A crash loop is detected and the pod is stopped so it stops charging you, with the cause classified rather than hidden.
Chat with or prompt a running endpoint in the dashboard, with streaming, before you write any client code. It adapts to the model type and uses the endpoint key for you.
Provisioning, downloading weights, loading model, ready — with a byte-accurate download bar. The provisioning deadline scales with model size, so a large model is never killed for being slow to start.
Create, stop, resume, restart, reconfigure and rotate keys over a REST management API, with idempotency keys on create so a timeout never leaves you with two endpoints.
Control when you want it
Every serving field has a working default — you can deploy without touching any of them. When you do need to tune for cost or throughput, the levers are real, and the platform tells you what each one costs you in GPU memory before you commit.
A live VRAM bar shows the weights / KV-cache / overhead split as you change context length, quantization or GPU count. The minimum count that fits is enforced, the recommended count is starred, and cards that cannot run your chosen precision are disabled with the reason.
| Model type | Chat, completion, embedding or reranker — sets the route exposed |
| Expected traffic | Light, steady or heavy — drives the recommended GPU count |
| Context length | Defaults to min(native, 32K), hard-capped at the model native max |
| Quantization | bf16 full precision, or fp8 for roughly half the memory |
| KV cache | auto, FP8, or TurboQuant 4/3/2-bit to fit more concurrent requests |
| GPU type & count | Live VRAM fit and hourly price, up to 8 GPUs in a single pod |
Product 02 — Fine-tuning
A separate product with its own pricing. Bring your data, pick a method and an algorithm, and train on dedicated GPUs billed by the second.
LoRA, QLoRA, AdaLoRA, LoHa, LoKr, BOFT, OFT, VeRA, ReFT and full parameter training.
DPO, SimPO, CPO, ORPO, KTO and Reward Modeling, without building an RL pipeline.
InternVL, Qwen-VL, LLaVA, DeepSeek-VL, MiniCPM-V, Phi Vision and more.
Extend a foundation model with proprietary data — medical, legal, financial, code.
This is not a positioning statement, it is in the contract. Our Terms state that the datasets you upload and the fine-tuned model weights your jobs produce are yours, and that we claim no rights over them beyond storing and processing them on your behalf. Base models keep their own licences, and complying with those remains your responsibility.