# OpenAI Chat API

> POST /v1/chat/completions on sference: the OpenAI Chat Completions shape, with flex processing, tools, structured output, vision, and streaming.

Source: https://sference.com/docs/api/chat-completions

`POST https://api.sference.com/v1/chat/completions` is the OpenAI Chat Completions shape, served natively. It is the surface the widest set of clients already speak, so most migrations are a base URL and a model id.

```bash
curl https://api.sference.com/v1/chat/completions \
  -H "Authorization: Bearer $SFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "messages": [{"role": "user", "content": "Say hello in one sentence."}]
  }'
```

## Base URL and authentication

| | |
| --- | --- |
| **Endpoint** | `POST https://api.sference.com/v1/chat/completions` |
| **Base URL for OpenAI SDKs** | `https://api.sference.com/v1` |
| **Auth** | `Authorization: Bearer sk_...` |
| **Key format** | sference keys (`sk_...`), minted in the [console](https://app.sference.com) |

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.sference.com/v1",
    api_key=os.environ["SFERENCE_API_KEY"],
)

response = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
)
print(response.choices[0].message.content)
```

> **Model ids are catalog ids**
>
> There are no `gpt-*` aliases. Pass a sference catalog id such as `zai-org/GLM-5.3` or `moonshotai/Kimi-K3`. `GET /v1/models` lists what your key can call, with capabilities and rates.

## Request fields

### Core

| Field | Notes |
| --- | --- |
| `model` | Catalog id |
| `messages` | `system` / `user` / `assistant` / `tool` roles; content may be a string or a list of parts |
| `stream` | SSE stream of `chat.completion.chunk` objects |
| `stream_options.include_usage` | Emits a final usage chunk — required to read token counts while streaming |
| `temperature`, `top_p` | Standard sampling |
| `top_k`, `min_p` | Additional sampling controls beyond OpenAI's set |
| `presence_penalty`, `frequency_penalty`, `repetition_penalty` | Penalties |
| `max_tokens`, `max_completion_tokens` | Output cap. On reasoning models this covers hidden reasoning too |
| `stop`, `seed` | Stop sequences and deterministic sampling |

### Tools and structured output

| Field | Notes |
| --- | --- |
| `tools`, `tool_choice` | OpenAI tool-calling shape, served natively |
| `response_format` | `{"type": "json_schema", ...}` or `{"type": "json_object"}` — see [Structured outputs](https://sference.com/docs/guides/structured-outputs) |

### Reasoning

| Field | Notes |
| --- | --- |
| `reasoning_effort` | `low` / `medium` / `high`, or an integer, on reasoning models |
| `enable_thinking` | Explicit on/off for thinking |

### sference extensions

| Field | Notes |
| --- | --- |
| `service_tier` | `"flex"` selects flex processing at 0.7 × list. Other accepted values serve as default |
| `prompt_cache_key` | Stable per-session id; pins the worker so the shared prefix bills at the cached rate — see [Prompt caching](https://sference.com/docs/guides/prompt-caching) |
| `max_queue_wait_s` | 5–300s admission bound; fails fast under congestion — see [Max queue wait](https://sference.com/docs/guides/max-queue-wait) |

## Streaming

```python
stream = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[{"role": "user", "content": "Write a haiku about GPUs."}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
```

Set `stream_options.include_usage` if you need the token counts — without it the final usage chunk is omitted.

## Tool calling

```python
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Return the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

response = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[{"role": "user", "content": "What is the weather in Ljubljana?"}],
    tools=tools,
)

tool_call = response.choices[0].message.tool_calls[0]
# Run the tool, append the assistant turn and a {"role": "tool", ...} message, call again
```

> **Tool reliability varies by model**
>
> Open models call tools less consistently than frontier closed models on long loops. `moonshotai/Kimi-K3` and `zai-org/GLM-5.3` are the strongest tool callers in the catalog; validate a tool-heavy workflow on your own prompts.

## Vision

Send images as content parts when the model supports it:

```python
messages=[{
    "role": "user",
    "content": [
        {"type": "text", "text": "What is the total on this invoice?"},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}},
    ],
}]
```

Check `capabilities.image_input.supported` on `GET /v1/models` first — the platform rejects an image sent to a text-only model. See [Images and vision](https://sference.com/docs/guides/vision).

## Response shape

Standard OpenAI Chat Completions: `id`, `object`, `created`, `model`, `choices[]` with `message` and `finish_reason`, and a `usage` block.

```json
{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "zai-org/GLM-5.3",
  "choices": [{
    "index": 0,
    "message": { "role": "assistant", "content": "Hello!" },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 18,
    "completion_tokens": 3,
    "total_tokens": 21,
    "prompt_tokens_details": { "cached_tokens": 0 }
  }
}
```

`usage.prompt_tokens_details.cached_tokens` is the number of prompt tokens served from the prefix cache. Reasoning models may also return `reasoning_content` on the message.

## Processing modes

`/v1/chat/completions` runs **realtime** by default and supports **flex**:

```python
response = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[{"role": "user", "content": doc}],
    service_tier="flex",   # 0.7x list, may queue up to 30 minutes
)
```

Background execution and batches are on the [OpenAI Responses API](https://sference.com/docs/api/responses) surface and the [Batch API](https://sference.com/docs/guides/batches).

## Errors

Standard OpenAI error envelope with `error.type`, `error.code`, and `error.message`. Two codes are worth handling explicitly:

| Code | Meaning |
| --- | --- |
| `max_queue_wait_exceeded` | The admission bound was hit — retry against a fallback. See [Max queue wait](https://sference.com/docs/guides/max-queue-wait) |
| `408` on a flex request | Flex could not complete in time; the OpenAI SDKs retry automatically |

## Not supported

| Field | Why |
| --- | --- |
| `n` (multiple completions) | One completion per request. Fan out with [concurrent requests](https://sference.com/docs/guides/requests-at-scale) |
| `logprobs`, `top_logprobs` | Not exposed by the serving engines |
| `logit_bias` | Not supported |

The full grid, including what is ignored rather than rejected, is on [API compatibility](https://sference.com/docs/compatibility).

- [OpenAI Responses API](https://sference.com/docs/api/responses): Background execution, streams, and the 24h window.
- [Messages](https://sference.com/docs/api/messages): The Anthropic-compatible surface.
- [Structured outputs](https://sference.com/docs/guides/structured-outputs): response_format with a JSON schema.
- [API compatibility](https://sference.com/docs/compatibility): Every field across all three surfaces.