Integrations · LiteLLM

One gateway. Every Run BiOS model.

LiteLLM Proxy sits between your developers and the API: they get one OpenAI-compatible endpoint, you keep the provider credentials and access control on the server.

Get an API key See the setup

config.yaml OpenAI-compatible

model_list:
  - model_name: deepseek-v4-flash
    litellm_params:
      model: openai/deepseek-v4-flash
      api_base: https://api.runbios.ai/v1
      api_key: os.environ/RUNBIOS_API_KEY

  - model_name: kimi-k3
    litellm_params:
      model: openai/kimi-k3
      api_base: https://api.runbios.ai/v1
      api_key: os.environ/RUNBIOS_API_KEY

One entry per model. The openai/ provider plus the Run BiOS base URL — no custom provider needed.

The setup

Four steps to a shared gateway.

The proxy runs wherever you host it — a dev box, a container, your infra. Once it is up, developers stop touching provider credentials entirely.

Prerequisites

A Run BiOS API key from the platform console, and LiteLLM installed:

pip install "litellm[proxy]"

Configure your models

One entry per model you want to expose. Use the openai/ provider prefix with the Run BiOS base URL — any served model id works, including your own fine-tuned checkpoint on a dedicated endpoint.

model_list:
  - model_name: deepseek-v4-flash
    litellm_params:
      model: openai/deepseek-v4-flash
      api_base: https://api.runbios.ai/v1
      api_key: os.environ/RUNBIOS_API_KEY

  - model_name: kimi-k3
    litellm_params:
      model: openai/kimi-k3
      api_base: https://api.runbios.ai/v1
      api_key: os.environ/RUNBIOS_API_KEY

Start the proxy

Export the Run BiOS key where the proxy runs, then start LiteLLM with your config:

export RUNBIOS_API_KEY="sk-bios-..."
litellm --config config.yaml

Call a model

Developers authenticate to the proxy with a virtual key — never the Run BiOS key itself:

curl http://localhost:4000/chat/completions \
  -H "Authorization: Bearer $LITELLM_VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [{"role": "user", "content": "Say pong in one word."}]
  }'

Key layout

Two keys, two jobs.

The Run BiOS key never reaches a developer machine. Virtual keys do the per-person work, and can be revoked without touching the upstream credential.

Key

Who holds it

Used for

Run BiOS API key (sk-bios-...)

The LiteLLM server (env or secret store)

Upstream Run BiOS inference

LiteLLM virtual key

Each developer or service

Proxy authentication and spend tracking

A common pattern: one Run BiOS service-account key on the proxy, virtual keys per developer for access control and attribution.

Why a gateway

Credentials in one place. Developers everywhere else.

One API for every model

Developers call one local endpoint while the proxy maps model names to the Run BiOS catalog. Swap a model behind the gateway and no client code changes.

Per-developer keys, per-developer spend

One Run BiOS key lives on the server. Each developer gets a virtual key, so usage and cost are attributed per person or per service without sharing credentials.

Access control on the server

Which models each team can call, rate limits, and budgets are enforced at the gateway — the same zero-log, zero-retention guarantee applies to every request behind it.

Questions teams ask before adding a gateway

What is LiteLLM Proxy?+

A gateway that exposes a single OpenAI-compatible API in front of one or more model providers. Your developers call the proxy; the proxy holds the provider credentials, issues per-developer virtual keys, and tracks spend per key.

Which Run BiOS models can I expose through it?+

Any served model id from the model library, plus your own fine-tuned checkpoint on a dedicated endpoint — both speak the same OpenAI-compatible API, so both work with the same openai/ provider config.

How are the keys laid out?+

The Run BiOS API key lives only on the LiteLLM server, in its environment or secret store. Developers and services authenticate to the proxy with virtual keys, which is what LiteLLM uses for access control and spend attribution.

Does the gateway change how traffic is billed?+

No. Requests still bill per million tokens at the model library rates. LiteLLM adds its own per-virtual-key spend tracking on top, so you can see which team or service drove which cost.

Put every Run BiOS model behind one endpoint tonight.

New accounts start with a $10 free credit — enough to stand the proxy up and watch the gateway route real traffic.

Get an API key Agent integrations

Per-million-token billing · Zero data retention · Your weights stay yours