# sference

> Inference stack for open models. One OpenAI-compatible API serving both realtime AI and long-running AI agents, with realtime, flex, and batch processing modes for open-weight models, fine-tunes, and distilled models. Independently measured as the fastest provider on the models we serve through the Requesty gateway. Full logging, auditing, and compliance built in.

## What is sference?

sference is a managed inference platform for open models. We run a scheduler and hardware-agnostic inference runtime built from the ground up, exposed as one OpenAI-compatible API. It serves both shapes of production AI from the same fleet: realtime AI, where a user is waiting on the response, and long-running AI agents, where work spans many model calls and nobody is waiting on any single one. Realtime, flex, and batch are scheduling priorities on one system, not separate deployments. The platform is high-performance and efficient at the same time: the design that keeps latency low is the same one that keeps GPU utilization high. Companies use it as one platform for all their inference needs, whether catalog open-weight models or their own custom weights, with request-level logging, auditing, and compliance built in, and without building and operating an inference stack themselves.

## Who is it for?

Series A+ companies and enterprises running AI in production who want off the closed-API treadmill for cost, control, or compliance reasons. Teams migrating workloads from OpenAI/Anthropic onto open models, teams with fine-tuned or distilled models that need production-grade serving, and companies in regulated industries whose customers ask where their data goes.

## Key capabilities

- **Any open model, including yours**: Open-weight frontier models: Kimi K3 (vision), GLM 5.3, DeepSeek V4.1 Flash (vision), GLM 5.3 Flash (vision), DeepSeek V4 Flash (0731), and Clef 27B (realtime decisions, vision) live today; DeepSeek V4 Pro, Gemma 4 31B, NVIDIA Nemotron 3 Super, OpenAI GPT-OSS 120B, and Qwen3.8 coming soon. Or upload your own weights: fine-tunes and distilled models run on the same API, infrastructure, and audit trail as catalog models. vLLM/SGLang models run out of the box; we stand up custom runtimes for models that don't. Custom model hosting is arranged per customer as part of an enterprise agreement.
- **Realtime, flex, and batch on the same API**: Sync endpoints (`/v1/chat/completions`, `/v1/messages`, blocking `/v1/responses`) for interactive inference; `service_tier: "flex"` for discounted latency-tolerant sync work; and a **24h** completion window for batches, background responses, and offline pipelines. All three share the same models, endpoints, and fleet. OpenAI's Flex processing is supported: requests with `service_tier: "flex"` get discounted tokens at lower scheduling priority (best-effort, 408 + retry on timeout). OpenAI-compatible, so most clients switch with a base-URL change.
- **Auto model routing (beta)**: `sference/auto` picks a model per session from the first user message — low `deepseek-ai/DeepSeek-V4.1-Flash`, medium `zai-org/GLM-5.3`, high `moonshotai/Kimi-K3` — and keeps it for the session so the prompt cache stays warm. The response `model` is the routed model, and `x-sference-routed-model` / `x-sference-route-tier` headers report the decision. Off by default, enabled per organization by an org admin; `/v1/chat/completions`, `/v1/messages`, and `/v1/responses` only (batches reject it). Docs: https://sference.com/docs/models#auto-model-routing
- **Anthropic-compatible too**: `POST /v1/messages` speaks the Anthropic Messages API, so the `anthropic` Python/TypeScript SDKs, Claude Code, and any Messages client run on open models by changing the base URL to `https://api.sference.com` and the model to a sference catalog id. Streaming SSE, tool use, extended thinking, and vision are supported; `x-api-key` and `Authorization: Bearer` both authenticate. Realtime only; flex, background, and batch live on the OpenAI-shaped endpoints. Docs: https://sference.com/docs/api/messages. Claude Code setup (one command: `sference launch claude`, with hybrid routing so Claude models still pass through to Anthropic): https://sference.com/docs/sference-code/claude-code
- **Serverless to dedicated**: Start on shared endpoints with per-token pricing and no minimums. Move hot paths to dedicated deployments (reserved GPU capacity, private endpoints, predictable latency), billed per GPU-hour on monthly commitments. Enterprise agreements add volume pricing, custom SLAs, and procurement-friendly terms.
- **Fastest measured route**: Independent live data (Requesty Model Discovery, gateway-wide provider comparison, mid-August 2026): sference serves `kimi-k3` at 426 ms average latency / 76 tok/s, `deepseek-v4-flash` at 861 ms / 157 tok/s, and `glm-5.2` at 3.1 s / 211 tok/s. That is the highest throughput of any provider on all three, and the lowest average latency on `kimi-k3` and `deepseek-v4-flash`.
- **High-performance and efficient, from one design**: A scheduler with priority queues and backpressure at the core. Realtime, flex, and batch windows are scheduling priorities on one system, not separate stacks, so interactive traffic never queues behind a backfill. Requests are recorded and dispatched transactionally and routed over per-model queues (the queue is the router, so no load balancer guesses at GPU state). Workers drive vLLM and SGLang as in-process libraries rather than servers behind another network hop, giving faster first tokens and higher throughput, with continuous batching, prefix caching, and bounded in-flight slots, so realtime traffic packs onto the same GPUs as batch backfill. Low latency and high utilization come from the same scheduler; utilization around the clock is where the per-token prices come from.
- **Forward-deployed onboarding**: Our engineers benchmark your real prompts across candidate open models side by side against your current provider, then work with your team through prompt migration and integration until the workload runs in production.
- **Pinned model versions**: Pin an exact version: every release has its own versioned id, so the version you validated is the version that serves. The serving version is recorded on every request, and a newer version can be validated by rerunning the same evals before switching.
- **Pipeline-friendly SDK**: Python SDK for batch and background inference from the orchestration you already run: submit batches, fan out background responses, collect results. Worked Prefect example in the docs.
- **Request-level logging and full audit trail**: Every request logged with model version, infrastructure region, timestamps, and data residency. The same logging and audit trail applies to catalog and custom models, realtime and batch alike. Configurable retention. Exportable compliance reports.

## Pricing

- **Serverless**: pay per token, no minimum spend. Serverless rates (USD per 1M tokens, realtime priority; confirmed per model at account provisioning; the site shows USD with a EUR toggle): Kimi K3 (vision) $3.00 / $15.00 / $0.45 cached; GLM 5.3 $1.20 / $4.20 / $0.26 cached; DeepSeek V4.1 Flash (vision) $0.50 / $1.50 / $0.05 cached; GLM 5.3 Flash (vision) $0.20 / $0.60 / $0.07 cached; DeepSeek V4 Flash (0731) $0.28 / $0.56 / $0.07 cached; Clef 27B (realtime decisions, vision) $0.24 / — / —. Coming soon: DeepSeek V4 Pro, Gemma 4 31B, NVIDIA Nemotron 3 Super, OpenAI GPT-OSS 120B, and Qwen3.8. Live rates: https://api.sference.com/v1/models. Batch and flex windows are discounted against the realtime rate. Custom model hosting and pricing are agreed per customer as part of an enterprise agreement.
- **Dedicated**: reserved GPU capacity with private endpoints, billed per GPU-hour on monthly commitments. Contact us.
- **Enterprise**: annual agreements, committed-spend discounts, volume pricing, custom SLAs, forward-deployed onboarding included. Contact us.
- **Why per-token beats self-managed**: our scheduler keeps GPU utilization high around the clock; on self-managed instances or on-prem clusters, idle capacity is paid for whether it serves tokens or not.

## Compliance and regulation

- GDPR compliant
- DORA ready (in enforcement since January 2025)
- EU AI Act deployer obligations ready (effective August 2026)
- DPA included; public copy at https://sference.com/legal/dpa; sub-processor list at https://sference.com/legal/subprocessors
- Configurable data retention policies
- Exportable compliance reports for security reviews and audits
- Zero training on customer data, contractually and technically

## Workloads

Production workloads teams run on sference. Overview at https://sference.com/#use-cases.

- **Coding and internal assistants**: Copilots, code review, and coding agents on open models, alongside the internal chat surfaces teams already run, such as Open WebUI. Streaming, tool calling, and structured output; Kimi K3 (vision), GLM 5.3, DeepSeek V4.1 Flash (vision), GLM 5.3 Flash (vision), DeepSeek V4 Flash (0731), and Clef 27B (realtime decisions, vision) in the catalog, your fine-tunes on the same API.
- **Features powered by agents**: Product features built on interactive and background agents. The turn a user waits on runs realtime; the steps behind it run on flex or batch windows, under one API and one audit trail, with queues and backpressure that keep long jobs from falling over and pinned versions so the checkpoint behind an agent stays explicit.

## Verticals

- **FinTech**: Batch KYC extraction, transaction classification, statement processing
- **LegalTech**: Contract corpus analysis, document review, embedding generation for legal RAG
- **HealthTech**: Medical record digitization, prescription extraction, clinical data processing
- **InsureTech**: Claims processing, policy document analysis, underwriting data extraction
- **AI/ML Teams**: Model serving for fine-tunes and distillations, evals, synthetic data generation

## Processing modes

- **realtime**: sync `/v1/chat/completions`, `/v1/messages`, or blocking `/v1/responses` when a user is waiting
- **flex**: `/v1/chat/completions` or `/v1/responses` with `service_tier: "flex"`: discounted tokens at lower scheduling priority; may queue longer (up to 30 minutes), fails with 408 on timeout (OpenAI SDKs retry automatically); enabled per account. Not available on `/v1/messages`, which is realtime only
- **batch (24h)**: background responses (`background: true`), streams, and batches on the 24h completion window, which is the only supported batch window value

## API

Base URL: https://api.sference.com/v1/, with one API key (`sk_...`) for every surface.

- **OpenAI-compatible**: `/v1/chat/completions`, `/v1/responses` (adds `background: true` and the 24h batch window), `/v1/embeddings`, `/v1/batches`, `/v1/streams`, `/v1/models`
- **Anthropic-compatible**: `/v1/messages` (Messages API shape; realtime only)
- **Decisions**: `/v1/decisions` (structured questions and calibrated probabilities; realtime only)
- **Control plane**: `/control/v1/...` for teams, API keys, catalog, deployments, activity, usage, billing
- Machine-readable OpenAPI specification: https://sference.com/openapi.json

## Model cards

One card per model in the live catalog: pricing, context and capabilities, how the model fits the portfolio, a quickstart, and curated benchmarks, serving details, and FAQ where available. Generated from the catalog, so they never drift. Index: https://sference.com/models (also as Markdown: https://sference.com/models.md — every card too, by appending `.md`).

- [Kimi K3](https://sference.com/models/moonshotai/Kimi-K3)
- [GLM 5.3](https://sference.com/models/zai-org/GLM-5.3)
- [DeepSeek V4.1 Flash](https://sference.com/models/deepseek-ai/DeepSeek-V4.1-Flash)
- [GLM 5.3 Flash](https://sference.com/models/zai-org/GLM-5.3-Flash)
- [DeepSeek V4 Flash (0731)](https://sference.com/models/deepseek-ai/DeepSeek-V4-Flash-0731)
- [Clef 27B](https://sference.com/models/Cloudflare/clef)

## Documentation

Full docs as a single Markdown file: https://sference.com/llms-full.txt
Any docs page as Markdown: append `.md` to its URL — guide pages and every API-reference page (e.g. https://sference.com/docs/api/chat-completions.md or https://sference.com/docs/api-reference/inference-api/V1ChatCompletionsCreateChatCompletion.md)
Automatic: send `Accept: text/markdown` on any /docs URL and receive Markdown without the suffix
Blog posts as Markdown: append `.md` to any /blog post URL (e.g. https://sference.com/blog/sference-is-live-on-opper.md)
Legal pages as Markdown: https://sference.com/legal/aup.md, https://sference.com/legal/dpa.md, https://sference.com/legal/subprocessors.md, https://sference.com/privacy.md, https://sference.com/terms.md

- [Overview](https://sference.com/docs): hub, three processing modes, where each client lives
- [Quickstart](https://sference.com/docs/quickstart): API key, CLI install, first request, curl fallback
- [Models](https://sference.com/docs/models): choosing a tier, pinning versions, aliases, auto model routing, catalog and BYOM entry points
- [Pricing](https://sference.com/docs/pricing): serverless rates per model, the three windows, cached input
- [OpenAI Chat API](https://sference.com/docs/api/chat-completions): `POST /v1/chat/completions`: the OpenAI-shaped path, streaming, tools, structured output
- [OpenAI Responses API](https://sference.com/docs/api/responses): `POST /v1/responses`: stateful responses, events, background mode, the realtime/flex/batch mode table
- [Anthropic Messages API](https://sference.com/docs/api/messages): `POST /v1/messages`: auth, supported fields and content blocks, SSE event sequence, tool use, extended thinking, images, errors, and the exact differences from api.anthropic.com
- [Custom Models](https://sference.com/docs/custom-models): bring your own Hugging Face weights, so fine-tunes and distilled models run on the same API
- [Deployments](https://sference.com/docs/deployments): dedicated reserved capacity for a base or custom model
- [Batch inference](https://sference.com/docs/guides/batches): one model per batch, catalog validation at create, chat or Responses row bodies, JSONL results, 24h window
- [Streams](https://sference.com/docs/guides/streams): group related requests into one window with a cost rollup and a resumable cursor
- [Prompt caching](https://sference.com/docs/guides/prompt-caching): keep the prefix stable and pin the worker to cut cached input cost
- [Images and vision](https://sference.com/docs/guides/vision): which models accept images, the request shapes, and how they bill
- [Max queue wait](https://sference.com/docs/guides/max-queue-wait): bound the wait and fail over instead of hanging
- [Webhooks](https://sference.com/docs/guides/webhooks): get a callback when a background response finishes instead of polling (beta)
- [Decisions](https://sference.com/docs/guides/decisions): `POST /v1/decisions`: choice, score, and yes/no questions with calibrated probabilities in one realtime call
- [Usage and analytics](https://sference.com/docs/usage): spend, tokens, and per-request activity grouped by model, key, member, and department
- [Build with agent frameworks](https://sference.com/docs/agents): LangChain, the OpenAI Agents SDK, and the Anthropic Agent SDK, plus structured output and scale patterns
- [Structured outputs](https://sference.com/docs/guides/structured-outputs): constrain the response to a JSON schema
- [Requests at scale](https://sference.com/docs/guides/requests-at-scale): fan out concurrent background responses, or submit one batch
- [Claude Code](https://sference.com/docs/sference-code/claude-code): `sference launch claude` proxy mode with hybrid routing, manual `ANTHROPIC_BASE_URL` config, model recommendations, known limits, troubleshooting
- [SferenceCode](https://sference.com/docs/sference-code): run Claude Code, Pi, opencode, Codex, or Cursor on open models, one command each
- [Python SDK](https://sference.com/docs/sdk): `pip install sference-sdk`, default `create_response(background=True)` + `wait_for_response`, `submit_batch` for JSONL jobs, Prefect example
- [CLI](https://sference.com/docs/cli): `curl -fsSL https://raw.githubusercontent.com/s-ference/sference/main/install.sh | sh`: browser sign-in (`sference auth login`, OAuth device flow; `--api-key` for CI), batch, responses, stream, and `sference launch claude|codex|opencode|pi` to run your coding agent on sference models for one session
- [Sference Switch](https://sference.com/docs/switch): macOS app (`curl -fsSL https://get.sference.com | sh`) with one on/off switch that routes every Claude Code and Codex session on the machine to sference models; Claude models still pass through to Anthropic
- [LLM and agent access](https://sference.com/docs/llm-access): Markdown views, llms.txt, and the OpenAPI spec for coding agents
- [Pricing for agents](https://sference.com/docs/pricing-for-agents): model by window rate table for agent workloads
- [API compatibility](https://sference.com/docs/compatibility): what each surface honors and what it ignores
- [Support matrix](https://sference.com/docs/support): one-page grid of every endpoint, field, and window
- [API Reference](https://sference.com/docs/api-reference): every path and schema generated from the OpenAPI specification

**Coding agents**: copy OSS **PROMPT.txt** (`https://raw.githubusercontent.com/s-ference/sference/main/PROMPT.txt`) for vibe coding; optional full **SKILL.md** for Cursor skills. Open-source clients: https://github.com/s-ference/sference

## Contact

- Website: https://sference.com
- Documentation: https://sference.com/docs
- Use cases: https://sference.com/#use-cases
- Pricing: https://sference.com/#pricing
- FAQ: https://sference.com/#faq
- Status: https://status.sference.com
- Changelog: https://sference.com/changelog
- Email: hello@sference.com
- LinkedIn: https://www.linkedin.com/company/sference/