# Streams

> Group related requests into a named stream: one window, one cost rollup, and a resumable cursor for workers that consume results incrementally.

Source: https://sference.com/docs/guides/streams

A **stream** is a named grouping of inference requests that share one completion window, one cost rollup, and one lifecycle. It is how you give a workload an identity on the platform instead of a pile of unrelated request ids.

Streams are the right primitive when:

- A job produces many requests and you want **one cost number** for the job.
- Several workers consume results from the same queue and need a **resumable cursor** so a restarted worker does not reprocess everything.
- You want to **cancel or archive** a whole workload in one call.

## Create a stream

```bash
curl -X POST https://api.sference.com/v1/streams \
  -H "Authorization: Bearer $SFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "nightly-eval-2026-10-13", "window": "24h"}'
```

```python
from sference_sdk import SferenceClient

client = SferenceClient()
stream = client.create_stream(name="nightly-eval-2026-10-13", window="24h")
```

`24h` is the only supported window value.

## Attach requests to a stream

Include `stream_id` in the response's `metadata`:

```python
response = client.create_response(
    model="zai-org/GLM-5.3",
    input=prompt,
    background=True,
    metadata={"stream_id": stream.id, "completion_window": "24h"},
)
```

Every request carrying that `stream_id` belongs to the stream. Requests may be submitted from any number of processes or machines.

## Read the rollup

`GET /v1/streams/{stream_id}` returns live counters:

```json
{
  "id": "0199c3e2-...",
  "name": "nightly-eval-2026-10-13",
  "window": "24h",
  "status": "open",
  "total_items": 12000,
  "pending_items": 340,
  "running_items": 64,
  "completed_items": 11580,
  "failed_items": 16,
  "cancelled_items": 0,
  "completion_ratio": 0.965,
  "total_prompt_tokens": 48210000,
  "total_completion_tokens": 1930000,
  "total_tokens": 50140000,
  "total_cost_micros": 61240000,
  "last_completion_at": "2026-10-13T02:14:07Z"
}
```

`total_cost_micros` is the stream's spend in millionths of a dollar — `61240000` is `$61.24`. Poll this instead of summing per-request costs yourself.

## Consume results with a cursor

`GET /v1/responses/events` returns completions in order. Pass `stream_id` to read one stream's feed, or omit it for the account-wide feed of every background response. The cursor is `starting_after` (a completion id), so a consumer can resume exactly where it stopped:

```python
from sference_sdk import SferenceClient

client = SferenceClient()

for event in client.iter_responses_events(stream_id=stream.id, consumer_name="worker-3"):
    process(event)
```

`iter_responses_events` persists the cursor per stream and consumer name via the SDK's checkpoint helpers, so a restarted worker picks up where it left off rather than replaying. Pass `from_latest=True` to skip history, or `checkpoint=False` to manage the cursor yourself with the lower-level `list_responses_events(stream_id=..., starting_after=...)`.

This is what makes a stream safe to consume from a pool of workers: each consumer tracks its own position, so a duplicate is avoided by the cursor rather than by deduplicating after the fact.

| Parameter | Meaning |
| --- | --- |
| `stream_id` | Read one stream's events. Omit for the account-wide feed. |
| `limit` | Page size (default 20). |
| `starting_after` | Cursor: return events after this completion id. |
| `ending_before` | Cursor: return events before this completion id. |
| `wait_ms` | Long-poll: hold the request open up to this many milliseconds for new events. |

## Lifecycle

| Status | Meaning |
| --- | --- |
| `open` | Accepting new requests |
| `cancelled` | Cancelled — pending items will not run |
| `archived` | Closed for new work; results remain readable |

```bash
# Rename
curl -X PATCH https://api.sference.com/v1/streams/$STREAM_ID \
  -H "Authorization: Bearer $SFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "nightly-eval-2026-10-13 (rerun)"}'

# Cancel everything still pending
curl -X PATCH https://api.sference.com/v1/streams/$STREAM_ID \
  -H "Authorization: Bearer $SFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"status": "cancelled"}'

# Archive when done
curl -X PATCH https://api.sference.com/v1/streams/$STREAM_ID \
  -H "Authorization: Bearer $SFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"status": "archived"}'
```

## CLI

```bash
sference stream create --name nightly-eval-2026-10-13
sference stream list
sference stream status --stream-id $STREAM_ID
sference stream submit --stream-id $STREAM_ID --model zai-org/GLM-5.3 --input prompt.txt
sference stream archive --stream-id $STREAM_ID
sference stream cancel --stream-id $STREAM_ID
```

## Streams vs. batches

Both group work on the 24h window, and they answer different questions.

| | Stream | Batch |
| --- | --- | --- |
| Membership | by `metadata.stream_id` on any request | one uploaded JSONL file |
| Submission | incremental, from anywhere | one create call |
| Ordering | events endpoint with a resumable cursor | results file, one line per row |
| Cancellation | cancel the stream | cancel the batch |
| Best for | long-running pipelines, multi-worker consumers, jobs assembled over time | a fixed dataset submitted once |

Use a batch when you have the whole request set up front. Use a stream when requests accumulate over time, when several consumers need to read results independently, or when you want a running cost rollup for a workload.

- [Batch inference](https://sference.com/docs/guides/batches): The fixed-dataset alternative.
- [Usage & analytics](https://sference.com/docs/usage): Attribute spend across streams, keys, and members.
- [Python SDK](https://sference.com/docs/sdk): Checkpoint helpers for stream consumers.