·Blog

Engineering notes.

Async inference, batch scheduling, and building AI infrastructure outside the realtime assumption.

2026-09-22·Aleksander Pejcic·Co-founder & CTO

Prefill Concurrency in SGLang: Consistent TTFT Under Multi-Tenant Load

One oversized prompt can take the prefill stage hostage and stall every request behind it, including requests that have almost nothing left to do. We added a concurrent chunked prefill knob to SGLang's scheduler so the cheap requests get past, and sent it upstream.

inferencesglangschedulinglatencyopen-source

2026-08-03·Jernej Štrasner·Co-founder & CEO

Sference is live on Opper: European inference for frontier open models

Sference's managed inference is now available through the Opper gateway on a European route, with compute inside the EEA and zero data retention on by default. Kimi K3, GLM-5.2, DeepSeek V4 Flash, Qwen3.6 35B-A3B, Qwen3-VL 30B, and ThinkingCap are callable with an Opper API key today.

partnershipoppereuropeinference

2026-06-09·Aleksander Pejcic·Co-founder & CTO

Your Batch Pipeline Is Running Realtime Inference: The LET Trilemma

Prefect, Airflow, and Pydantic AI teach synchronous realtime inference patterns even inside batch DAGs, defeating the economic purpose of batch orchestration. We introduce the LET trilemma — latency, efficiency, throughput: pick two — and show how Sference reclaims batch economics.

inferencebatchefficiencyagentsarchitecture