LiteLLM gateway

If your team already runs a LiteLLM proxy, you do not need a second gateway to use sference. Add sference as an upstream and keep LiteLLM as the control point: virtual keys, budgets, spend logs, fallbacks, and per-team routing stay where they are, and sference serves the models.

The two shapes line up:

Requests flow client → LiteLLM → sference. LiteLLM authenticates to sference with a sk_... key; clients authenticate to LiteLLM with a LiteLLM virtual key and never see the sference key.

OpenAI-shaped upstream

Point LiteLLM's OpenAI provider at sference's OpenAI-compatible base URL. The base URL includes /v1; LiteLLM appends /chat/completions.

model_list:
  - model_name: "glm-5.3"
    litellm_params:
      model: "openai/zai-org/GLM-5.3"
      api_base: https://api.sference.com/v1
      api_key: os.environ/SFERENCE_API_KEY

  - model_name: "kimi-k3"
    litellm_params:
      model: "openai/moonshotai/Kimi-K3"
      api_base: https://api.sference.com/v1
      api_key: os.environ/SFERENCE_API_KEY

model_name is what your clients ask for; model is the provider plus the sference catalog id, which LiteLLM sends as the request's model. Clients call glm-5.3 through the proxy exactly as they would call any LiteLLM model.

To avoid enumerating the catalog, use a prefix wildcard. Only names under the prefix reach sference, so the entry is safe on a shared proxy:

  - model_name: "sference/*"
    litellm_params:
      model: "openai/*"
      api_base: https://api.sference.com/v1
      api_key: os.environ/SFERENCE_API_KEY

Clients then ask for sference/zai-org/GLM-5.3. A bare model_name: "*" also works, but it captures every unmatched model name for every key — only do that on a proxy that exists for sference alone.

Anthropic-shaped upstream (Claude Code)

sference also speaks the Anthropic Messages API, so Claude Code and the anthropic SDKs can run on open models through the same proxy. Use LiteLLM's Anthropic provider with sference's host as the base URL — no /v1 suffix, because LiteLLM appends /v1/messages:

model_list:
  - model_name: "sference-glm-5.3"
    litellm_params:
      model: "anthropic/zai-org/GLM-5.3"
      api_base: https://api.sference.com
      api_key: os.environ/SFERENCE_API_KEY

Claude Code sends its own tier names (claude-opus-5-5, claude-sonnet-5-5, claude-haiku-5-5, claude-fable-5-1). Map them to sference models in one of two ways.

Aliases on the virtual key — Claude Code's settings stay untouched and the key decides routing:

{
  "aliases": {
    "claude-opus-5-5": "sference-glm-5.3",
    "claude-sonnet-5-5": "sference-glm-5.3",
    "claude-haiku-5-5": "sference-glm-5.3",
    "claude-fable-5-1": "sference-glm-5.3"
  }
}

An alias matches the exact model name Claude Code sends. If a newer Claude Code release renames a tier, the LiteLLM request log shows the name it now sends; add an alias for it.

Tier variables — set the model names in a managed Claude Code settings file instead:

{
  "env": {
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "sference-glm-5.3",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "sference-glm-5.3",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "sference-glm-5.3",
    "ANTHROPIC_DEFAULT_FABLE_MODEL": "sference-glm-5.3"
  }
}

Then point Claude Code at the proxy. ANTHROPIC_BASE_URL is the LiteLLM host without /v1 (Claude Code appends /v1/messages), and ANTHROPIC_AUTH_TOKEN is the LiteLLM virtual key:

export ANTHROPIC_BASE_URL=https://litellm.example.com
export ANTHROPIC_AUTH_TOKEN=sk-litellm-virtual-key
claude

Streaming, tool use, extended thinking, and vision all work on this path — sference's /v1/messages supports them — with one limit: the Messages surface is realtime only. Flex, priority, background responses, and batches live on the OpenAI-shaped endpoints.

Forwarding client headers

LiteLLM makes its own upstream request, so headers your client sets are dropped at the proxy unless you forward them. Two matter on this path:

• anthropic-beta — gates beta features on the Messages surface, including extended thinking.
• Any x--prefixed header your clients send.

Enable forwarding globally:

general_settings:
  forward_client_headers_to_llm_api: true

Only x-prefixed headers and anthropic-beta are forwarded. Verify on the first request that the feature you rely on still works (for example, that extended thinking still streams); if it does not, the header is not arriving.

Virtual keys

Clients should authenticate to LiteLLM, not to sference:

• Create a LiteLLM virtual key allowed to call the sference model entries above (admin UI or API).
• Give that key to the client as its bearer token (ANTHROPIC_AUTH_TOKEN for Claude Code).
• Keep the sference sk_... key in LiteLLM's environment as SFERENCE_API_KEY; LiteLLM uses it upstream.

The sference key's usage, spend, and per-request logs still attribute every call, so the audit trail on the sference side is intact even though the client never holds the key.

Service tiers through the proxy

sference's priority and flex tiers are per-request fields on the OpenAI-shaped endpoints. LiteLLM forwards service_tier on OpenAI-shaped requests, so you can set it per request:

resp = client.chat.completions.create(
    model="glm-5.3",
    service_tier="priority",
    messages=[{"role": "user", "content": "Summarize this thread."}],
)

or as a default for a model entry:

    litellm_params:
      service_tier: "priority"

Three things to know before you route tiers through the proxy:

• Priority is org-entitled. Standard organizations are entitled by default; reseller organizations are not, and an unentitled priority request fails with a 400 rather than silently serving the default tier.
• /v1/messages takes no service_tier. Claude Code traffic through the Anthropic provider is realtime at the default tier.
• Batch rejects priority. A request carries one tier; flex and batch combine, priority does not.

Verify the hop

• LiteLLM logs show the client request and the upstream call to api.sference.com.
• sference usage shows the traffic under your sk_... key, broken down by model, endpoint, and tier — the same attribution you would get without the proxy.
• GET /v1/models through the proxy lists the entries you configured; call one with max_tokens: 1 before rolling it out to a team.

Limits and caveats

• Catalog ids only. There are no gpt-* or claude-* aliases on sference; alias them in LiteLLM or set Claude Code's tier variables.
• Anthropic surface is realtime only. Flex, priority, background responses, and batches are OpenAI-shaped. If a workload needs them, route it through the openai/* provider.
• Base URLs differ by provider. OpenAI provider: https://api.sference.com/v1. Anthropic provider: https://api.sference.com (no /v1).
• Wildcards capture everything unmatched. Scope them with a prefix (sference/*) on a shared proxy.
• Header forwarding is all-or-nothing. Only x-prefixed and anthropic-beta headers cross the proxy; verify the first request.
• Batches, streams, and background responses are sference surfaces, not LiteLLM ones. They are reachable directly with the same sk_... key; see Batch inference and Streams.