# LiteLLM gateway

> Keep LiteLLM as your gateway and add sference as an upstream: one proxy for keys, budgets, and routing, with sference serving the models over its OpenAI- and Anthropic-compatible surfaces.

Source: https://sference.com/docs/sference-code/litellm

If your team already runs a [LiteLLM](https://docs.litellm.ai) proxy, you do not need a second gateway to use sference. Add sference as an upstream and keep LiteLLM as the control point: virtual keys, budgets, spend logs, fallbacks, and per-team routing stay where they are, and sference serves the models.

The two shapes line up:

| Client speaks | LiteLLM provider | sference surface |
| --- | --- | --- |
| OpenAI Chat Completions | `openai/*` | `POST /v1/chat/completions` |
| Anthropic Messages (Claude Code, `anthropic` SDKs) | `anthropic/*` | `POST /v1/messages` |

Requests flow **client → LiteLLM → sference**. LiteLLM authenticates to sference with a `sk_...` key; clients authenticate to LiteLLM with a LiteLLM virtual key and never see the sference key.

> **Model ids are catalog ids**
>
> sference has no `gpt-*` or `claude-*` aliases. Every request names a catalog id such as `zai-org/GLM-5.3`, `moonshotai/Kimi-K3`, or `deepseek-ai/DeepSeek-V4-Flash-0731`. `GET https://api.sference.com/v1/models` lists what your key can call. Map your clients' model names to catalog ids with LiteLLM aliases or Claude Code's tier variables, below.

## OpenAI-shaped upstream

Point LiteLLM's OpenAI provider at sference's OpenAI-compatible base URL. The base URL includes `/v1`; LiteLLM appends `/chat/completions`.

```yaml
model_list:
  - model_name: "glm-5.3"
    litellm_params:
      model: "openai/zai-org/GLM-5.3"
      api_base: https://api.sference.com/v1
      api_key: os.environ/SFERENCE_API_KEY

  - model_name: "kimi-k3"
    litellm_params:
      model: "openai/moonshotai/Kimi-K3"
      api_base: https://api.sference.com/v1
      api_key: os.environ/SFERENCE_API_KEY
```

`model_name` is what your clients ask for; `model` is the provider plus the sference catalog id, which LiteLLM sends as the request's `model`. Clients call `glm-5.3` through the proxy exactly as they would call any LiteLLM model.

To avoid enumerating the catalog, use a prefix wildcard. Only names under the prefix reach sference, so the entry is safe on a shared proxy:

```yaml
  - model_name: "sference/*"
    litellm_params:
      model: "openai/*"
      api_base: https://api.sference.com/v1
      api_key: os.environ/SFERENCE_API_KEY
```

Clients then ask for `sference/zai-org/GLM-5.3`. A bare `model_name: "*"` also works, but it captures every unmatched model name for every key — only do that on a proxy that exists for sference alone.

## Anthropic-shaped upstream (Claude Code)

sference also speaks the Anthropic Messages API, so Claude Code and the `anthropic` SDKs can run on open models through the same proxy. Use LiteLLM's Anthropic provider with sference's **host** as the base URL — no `/v1` suffix, because LiteLLM appends `/v1/messages`:

```yaml
model_list:
  - model_name: "sference-glm-5.3"
    litellm_params:
      model: "anthropic/zai-org/GLM-5.3"
      api_base: https://api.sference.com
      api_key: os.environ/SFERENCE_API_KEY
```

Claude Code sends its own tier names (`claude-opus-5-5`, `claude-sonnet-5-5`, `claude-haiku-5-5`, `claude-fable-5-1`). Map them to sference models in one of two ways.

**Aliases on the virtual key** — Claude Code's settings stay untouched and the key decides routing:

```json
{
  "aliases": {
    "claude-opus-5-5": "sference-glm-5.3",
    "claude-sonnet-5-5": "sference-glm-5.3",
    "claude-haiku-5-5": "sference-glm-5.3",
    "claude-fable-5-1": "sference-glm-5.3"
  }
}
```

An alias matches the exact model name Claude Code sends. If a newer Claude Code release renames a tier, the LiteLLM request log shows the name it now sends; add an alias for it.

**Tier variables** — set the model names in a managed Claude Code settings file instead:

```json
{
  "env": {
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "sference-glm-5.3",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "sference-glm-5.3",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "sference-glm-5.3",
    "ANTHROPIC_DEFAULT_FABLE_MODEL": "sference-glm-5.3"
  }
}
```

Then point Claude Code at the proxy. `ANTHROPIC_BASE_URL` is the LiteLLM host without `/v1` (Claude Code appends `/v1/messages`), and `ANTHROPIC_AUTH_TOKEN` is the LiteLLM virtual key:

```bash
export ANTHROPIC_BASE_URL=https://litellm.example.com
export ANTHROPIC_AUTH_TOKEN=sk-litellm-virtual-key
claude
```

Streaming, tool use, extended thinking, and vision all work on this path — sference's `/v1/messages` supports them — with one limit: the Messages surface is **realtime only**. Flex, priority, background responses, and batches live on the OpenAI-shaped endpoints.

## Forwarding client headers

LiteLLM makes its own upstream request, so headers your client sets are dropped at the proxy unless you forward them. Two matter on this path:

- `anthropic-beta` — gates beta features on the Messages surface, including extended thinking.
- Any `x-`-prefixed header your clients send.

Enable forwarding globally:

```yaml
general_settings:
  forward_client_headers_to_llm_api: true
```

Only `x`-prefixed headers and `anthropic-beta` are forwarded. Verify on the first request that the feature you rely on still works (for example, that extended thinking still streams); if it does not, the header is not arriving.

> **Use the global switch, not the per-model-group form**
>
> LiteLLM's per-group form (`litellm_settings.model_group_settings.forward_client_headers_to_llm_api: ["sference/*"]`) is evaluated before virtual-key aliases apply. If you route Claude Code through aliases, the per-group form forwards nothing; the global switch does.

## Virtual keys

Clients should authenticate to LiteLLM, not to sference:

- Create a LiteLLM virtual key allowed to call the sference model entries above (admin UI or API).
- Give that key to the client as its bearer token (`ANTHROPIC_AUTH_TOKEN` for Claude Code).
- Keep the sference `sk_...` key in LiteLLM's environment as `SFERENCE_API_KEY`; LiteLLM uses it upstream.

The sference key's usage, spend, and per-request logs still attribute every call, so the audit trail on the sference side is intact even though the client never holds the key.

## Service tiers through the proxy

sference's [priority](https://sference.com/docs/guides/priority-tier) and flex tiers are per-request fields on the OpenAI-shaped endpoints. LiteLLM forwards `service_tier` on OpenAI-shaped requests, so you can set it per request:

```python
resp = client.chat.completions.create(
    model="glm-5.3",
    service_tier="priority",
    messages=[{"role": "user", "content": "Summarize this thread."}],
)
```

or as a default for a model entry:

```yaml
    litellm_params:
      service_tier: "priority"
```

Three things to know before you route tiers through the proxy:

- **Priority is org-entitled.** Standard organizations are entitled by default; reseller organizations are not, and an unentitled priority request fails with a `400` rather than silently serving the default tier.
- **`/v1/messages` takes no `service_tier`.** Claude Code traffic through the Anthropic provider is realtime at the default tier.
- **Batch rejects priority.** A request carries one tier; flex and batch combine, priority does not.

## Verify the hop

- **LiteLLM logs** show the client request and the upstream call to `api.sference.com`.
- **sference usage** shows the traffic under your `sk_...` key, broken down by model, endpoint, and tier — the same attribution you would get without the proxy.
- **`GET /v1/models`** through the proxy lists the entries you configured; call one with `max_tokens: 1` before rolling it out to a team.

## Limits and caveats

- **Catalog ids only.** There are no `gpt-*` or `claude-*` aliases on sference; alias them in LiteLLM or set Claude Code's tier variables.
- **Anthropic surface is realtime only.** Flex, priority, background responses, and batches are OpenAI-shaped. If a workload needs them, route it through the `openai/*` provider.
- **Base URLs differ by provider.** OpenAI provider: `https://api.sference.com/v1`. Anthropic provider: `https://api.sference.com` (no `/v1`).
- **Wildcards capture everything unmatched.** Scope them with a prefix (`sference/*`) on a shared proxy.
- **Header forwarding is all-or-nothing.** Only `x`-prefixed and `anthropic-beta` headers cross the proxy; verify the first request.
- **Batches, streams, and background responses are sference surfaces**, not LiteLLM ones. They are reachable directly with the same `sk_...` key; see [Batch inference](https://sference.com/docs/guides/batches) and [Streams](https://sference.com/docs/guides/streams).