Streams
Group related requests into a named stream: one window, one cost rollup, and a resumable cursor for workers that consume results incrementally.
Streams
A stream is a named grouping of inference requests that share one completion window, one cost rollup, and one lifecycle. It is how you give a workload an identity on the platform instead of a pile of unrelated request ids.
Streams are the right primitive when:
- A job produces many requests and you want one cost number for the job.
- Several workers consume results from the same queue and need a resumable cursor so a restarted worker does not reprocess everything.
- You want to cancel or archive a whole workload in one call.
Create a stream
curl -X POST https://api.sference.com/v1/streams \
-H "Authorization: Bearer $SFERENCE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"name": "nightly-eval-2026-10-13", "window": "24h"}'from sference_sdk import SferenceClient
client = SferenceClient()
stream = client.create_stream(name="nightly-eval-2026-10-13", window="24h")24h is the only supported window value.
Attach requests to a stream
Include stream_id in the response's metadata:
response = client.create_response(
model="zai-org/GLM-5.3",
input=prompt,
background=True,
metadata={"stream_id": stream.id, "completion_window": "24h"},
)Every request carrying that stream_id belongs to the stream. Requests may be submitted from any number of processes or machines.
Read the rollup
GET /v1/streams/{stream_id} returns live counters:
{
"id": "0199c3e2-...",
"name": "nightly-eval-2026-10-13",
"window": "24h",
"status": "open",
"total_items": 12000,
"pending_items": 340,
"running_items": 64,
"completed_items": 11580,
"failed_items": 16,
"cancelled_items": 0,
"completion_ratio": 0.965,
"total_prompt_tokens": 48210000,
"total_completion_tokens": 1930000,
"total_tokens": 50140000,
"total_cost_micros": 61240000,
"last_completion_at": "2026-10-13T02:14:07Z"
}total_cost_micros is the stream's spend in millionths of a dollar — 61240000 is $61.24. Poll this instead of summing per-request costs yourself.
Consume results with a cursor
GET /v1/responses/events returns completions in order. Pass stream_id to read one stream's feed, or omit it for the account-wide feed of every background response. The cursor is starting_after (a completion id), so a consumer can resume exactly where it stopped:
from sference_sdk import SferenceClient
client = SferenceClient()
for event in client.iter_responses_events(stream_id=stream.id, consumer_name="worker-3"):
process(event)iter_responses_events persists the cursor per stream and consumer name via the SDK's checkpoint helpers, so a restarted worker picks up where it left off rather than replaying. Pass from_latest=True to skip history, or checkpoint=False to manage the cursor yourself with the lower-level list_responses_events(stream_id=..., starting_after=...).
This is what makes a stream safe to consume from a pool of workers: each consumer tracks its own position, so a duplicate is avoided by the cursor rather than by deduplicating after the fact.
| Parameter | Meaning |
|---|---|
stream_id | Read one stream's events. Omit for the account-wide feed. |
limit | Page size (default 20). |
starting_after | Cursor: return events after this completion id. |
ending_before | Cursor: return events before this completion id. |
wait_ms | Long-poll: hold the request open up to this many milliseconds for new events. |
Lifecycle
| Status | Meaning |
|---|---|
open | Accepting new requests |
cancelled | Cancelled — pending items will not run |
archived | Closed for new work; results remain readable |
# Rename
curl -X PATCH https://api.sference.com/v1/streams/$STREAM_ID \
-H "Authorization: Bearer $SFERENCE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"name": "nightly-eval-2026-10-13 (rerun)"}'
# Cancel everything still pending
curl -X PATCH https://api.sference.com/v1/streams/$STREAM_ID \
-H "Authorization: Bearer $SFERENCE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"status": "cancelled"}'
# Archive when done
curl -X PATCH https://api.sference.com/v1/streams/$STREAM_ID \
-H "Authorization: Bearer $SFERENCE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"status": "archived"}'CLI
sference stream create --name nightly-eval-2026-10-13
sference stream list
sference stream status --stream-id $STREAM_ID
sference stream submit --stream-id $STREAM_ID --model zai-org/GLM-5.3 --input prompt.txt
sference stream archive --stream-id $STREAM_ID
sference stream cancel --stream-id $STREAM_IDStreams vs. batches
Both group work on the 24h window, and they answer different questions.
| Stream | Batch | |
|---|---|---|
| Membership | by metadata.stream_id on any request | one uploaded JSONL file |
| Submission | incremental, from anywhere | one create call |
| Ordering | events endpoint with a resumable cursor | results file, one line per row |
| Cancellation | cancel the stream | cancel the batch |
| Best for | long-running pipelines, multi-worker consumers, jobs assembled over time | a fixed dataset submitted once |
Use a batch when you have the whole request set up front. Use a stream when requests accumulate over time, when several consumers need to read results independently, or when you want a running cost rollup for a workload.