Max queue wait

max_queue_wait_s is an admission bound. It states the maximum projected wait your request is willing to accept behind work that is already queued. If the scheduler cannot admit the request inside that budget, the request is rejected immediately with a max_queue_wait_exceeded error instead of sitting in a queue.

That rejection is the point: it turns "this will be slow" from something you discover after a timeout into something you can act on in milliseconds.

response = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[{"role": "user", "content": prompt}],
    max_queue_wait_s=15,
)

What the bound covers

The bound is on congestion, not on prompt size. A 100k-token prompt on an idle fleet is admitted immediately; a short prompt behind a deep queue is not.

Accepted range: 5–300 seconds. Omit the field to use the platform default ceiling (60s for streaming requests, 300s for synchronous ones).

Why you would set it

Fast fallback to another provider

This is the main use. Set a bound that reflects your real latency budget, and treat max_queue_wait_exceeded as a routing signal: retry the request against your fallback provider, or against a different sference model, rather than waiting.

from openai import OpenAI
import openai

sference = OpenAI(base_url="https://api.sference.com/v1", api_key=SFERENCE_KEY)
fallback = OpenAI(base_url="https://api.other-provider.com/v1", api_key=OTHER_KEY)

def complete(prompt: str):
    try:
        return sference.chat.completions.create(
            model="zai-org/GLM-5.3",
            messages=[{"role": "user", "content": prompt}],
            max_queue_wait_s=15,
        )
    except openai.BadRequestError as e:
        if "max_queue_wait_exceeded" in str(e):
            return fallback.chat.completions.create(
                model="gpt-5.6",
                messages=[{"role": "user", "content": prompt}],
            )
        raise

Adopting sference as a preferred provider

The bound is what makes sference safe to put first in a multi-provider chain. You get open-model economics on the overwhelming majority of requests, and a hard, fast signal to fall back when the fleet is busy — instead of a client timeout that wastes the whole budget before failing.

Pair it with service_tier: "flex" when the request is not user-facing: flex requests count their deliberate dispatch hold against the bound, so a bound on a flex request bounds end-to-end scheduling latency rather than just queue depth.

Protecting a batch of concurrent calls

When you fan out hundreds of requests at once, a bound keeps a congested moment from turning into hundreds of connections held open for minutes. Requests that cannot be admitted fail fast and can be retried in a later wave.

The error

A rejected request returns a 400 with a structured body:

{
  "error": {
    "code": "max_queue_wait_exceeded",
    "message": "Projected wait exceeded max_queue_wait_s",
    "param": "max_queue_wait_s"
  }
}

The distinction in param matters for monitoring: a rejection with param: null means the platform ceiling fired and you did not ask for a bound — lower your own bound to take control of that decision, or raise concurrency elsewhere.

Interaction with flex

Flex requests are dispatched at lower priority by design, and that hold is counted against max_queue_wait_s. A flex request with a 30s bound will be rejected rather than held for the full flex window — which is usually what you want for a workload that has a deadline, and not what you want for a pure backfill job. For backfill, either omit the bound or use background: true with the 24h batch window, where the bound does not apply.

Where it is accepted