# Max queue wait

> Bound how long a request may wait behind queued work, and use the bound to fail over to another provider when sference is congested.

Source: https://sference.com/docs/guides/max-queue-wait

`max_queue_wait_s` is an **admission bound**. It states the maximum projected wait your request is willing to accept behind work that is already queued. If the scheduler cannot admit the request inside that budget, the request is rejected immediately with a `max_queue_wait_exceeded` error instead of sitting in a queue.

That rejection is the point: it turns "this will be slow" from something you discover after a timeout into something you can act on in milliseconds.

```python
response = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[{"role": "user", "content": prompt}],
    max_queue_wait_s=15,
)
```

## What the bound covers

| Counts against the bound | Does not count |
| --- | --- |
| Projected wait behind already-queued work | Your own prompt's prefill time |
| Pending time already spent waiting | Decode time after the first token |
| For flex requests, the deliberate dispatch hold | Network transfer to and from the API |

The bound is on **congestion**, not on prompt size. A 100k-token prompt on an idle fleet is admitted immediately; a short prompt behind a deep queue is not.

Accepted range: **5–300 seconds**. Omit the field to use the platform default ceiling (60s for streaming requests, 300s for synchronous ones).

## Why you would set it

### Fast fallback to another provider

This is the main use. Set a bound that reflects your real latency budget, and treat `max_queue_wait_exceeded` as a routing signal: retry the request against your fallback provider, or against a different sference model, rather than waiting.

```python
from openai import OpenAI
import openai

sference = OpenAI(base_url="https://api.sference.com/v1", api_key=SFERENCE_KEY)
fallback = OpenAI(base_url="https://api.other-provider.com/v1", api_key=OTHER_KEY)

def complete(prompt: str):
    try:
        return sference.chat.completions.create(
            model="zai-org/GLM-5.3",
            messages=[{"role": "user", "content": prompt}],
            max_queue_wait_s=15,
        )
    except openai.BadRequestError as e:
        if "max_queue_wait_exceeded" in str(e):
            return fallback.chat.completions.create(
                model="gpt-5.6",
                messages=[{"role": "user", "content": prompt}],
            )
        raise
```

### Adopting sference as a preferred provider

The bound is what makes sference safe to put **first** in a multi-provider chain. You get open-model economics on the overwhelming majority of requests, and a hard, fast signal to fall back when the fleet is busy — instead of a client timeout that wastes the whole budget before failing.

Pair it with `service_tier: "flex"` when the request is not user-facing: flex requests count their deliberate dispatch hold against the bound, so a bound on a flex request bounds end-to-end scheduling latency rather than just queue depth.

### Protecting a batch of concurrent calls

When you fan out hundreds of requests at once, a bound keeps a congested moment from turning into hundreds of connections held open for minutes. Requests that cannot be admitted fail fast and can be retried in a later wave.

## The error

A rejected request returns a `400` with a structured body:

```json
{
  "error": {
    "code": "max_queue_wait_exceeded",
    "message": "Projected wait exceeded max_queue_wait_s",
    "param": "max_queue_wait_s"
  }
}
```

| Field | Meaning |
| --- | --- |
| `code` | `max_queue_wait_exceeded` |
| `param` | `max_queue_wait_s` when **you** set the bound; `null` when the platform's default ceiling rejected it |
| `message` | Human-readable detail |

The distinction in `param` matters for monitoring: a rejection with `param: null` means the platform ceiling fired and you did not ask for a bound — lower your own bound to take control of that decision, or raise concurrency elsewhere.

> **Rejected requests are not billed**
>
> A `max_queue_wait_exceeded` rejection happens before any inference work, so it costs nothing. Retrying it against a fallback is free on our side.

## Interaction with flex

Flex requests are dispatched at lower priority by design, and that hold is counted against `max_queue_wait_s`. A flex request with a 30s bound will be rejected rather than held for the full flex window — which is usually what you want for a workload that has a deadline, and not what you want for a pure backfill job. For backfill, either omit the bound or use `background: true` with the 24h batch window, where the bound does not apply.

## Where it is accepted

| Surface | Field |
| --- | --- |
| `/v1/chat/completions` | `max_queue_wait_s` |
| `/v1/responses` | `max_queue_wait_s` |
| `/v1/messages` | `max_queue_wait_s` |
| Batch rows | Not supported — batch rows are already on the 24h window and cannot be rejected for congestion |

- [Pricing](https://sference.com/docs/pricing): Flex and batch windows, and what they cost.
- [Requests at scale](https://sference.com/docs/guides/requests-at-scale): Concurrency patterns that use the bound well.
- [Webhooks](https://sference.com/docs/guides/webhooks): For work you would rather not wait on at all.