Build with agent frameworks

An agent loop is many model calls in sequence: plan, call a tool, read the result, decide again. sference is built for that shape. The turn a user is waiting on runs realtime; the steps behind it can run on flex or the 24h batch window — same API, same fleet, same audit trail, one bill.

This section shows how to point three popular agent frameworks at sference, how to choose a window per step, and the two patterns every agent needs in production: structured output and sending requests at scale.

One base URL, three SDKs

Every framework below reaches sference the same way: an OpenAI-compatible or Anthropic-compatible base URL plus a sk_... key. Nothing else about the framework changes.

import os

BASE_URL = "https://api.sference.com/v1"
API_KEY = os.environ["SFERENCE_API_KEY"]

Choosing a window per step

The single biggest cost decision in an agent is which steps need to be synchronous. Most do not.

Flex and batch are 0.7× and 0.5× list price respectively, and they stack. See Pricing.

# The turn the user waits on — realtime
answer = client.chat.completions.create(model=MODEL, messages=messages)

# A background step nobody is waiting on — flex, 30% cheaper
summary = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": doc}],
    service_tier="flex",
)

# An overnight job — batch, 50% cheaper
job = client.responses.create(
    model=MODEL,
    input=task,
    background=True,
    metadata={"completion_window": "24h"},
)

Keeping an agent loop cheap and fast

Four practices apply to every framework on this page:

1. Pin the worker with prompt_cache_key. An agent resends the same system prompt and tool definitions on every step. Set prompt_cache_key to the run id so those tokens bill at the cached rate instead of full price. See Prompt caching.
2. Bound the wait with max_queue_wait_s. Fail over rather than hang when the fleet is busy. See Max queue wait.
3. Cap max_output_tokens on reasoning models. The cap covers hidden reasoning, so an uncapped thinking step can burn the budget before it acts.
4. Put the volatile part of the prompt last. Timestamps, retrieved documents, and the user's question go after the stable system prompt.

Where to go next