LangChain
Use sference as the model provider for LangChain and LangGraph agents: ChatOpenAI, tool calling, streaming, and per-step windows.
LangChain
sference exposes an OpenAI-compatible /v1/chat/completions, so LangChain's ChatOpenAI works with a base-URL change. Chains, tools, and LangGraph graphs then run on open models with the same code.
Install
pip install -U langchain-openaiPoint ChatOpenAI at sference
import os
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
model="zai-org/GLM-5.3",
base_url="https://api.sference.com/v1",
api_key=os.environ["SFERENCE_API_KEY"],
temperature=0.2,
)Any LangChain component that takes a chat model now runs on sference — chains, output parsers, retrievers, and LangGraph nodes.
Tool calling
LangChain's tool decorator binds to the OpenAI tool-calling shape, which sference serves natively:
from langchain_core.tools import tool
@tool
def get_weather(city: str) -> str:
"""Return the current weather for a city."""
return weather_api(city)
llm_with_tools = llm.bind_tools([get_weather])
response = llm_with_tools.invoke("What is the weather in Ljubljana?")Tool reliability varies by model
Open models call tools less consistently than frontier closed models on long loops. moonshotai/Kimi-K3 and zai-org/GLM-5.3 are the strongest tool callers in the catalog; validate a tool-heavy workflow on your own prompts before committing to it.
Structured output
Use with_structured_output for typed results — it maps to response_format with a JSON schema:
from pydantic import BaseModel, Field
class Ticket(BaseModel):
severity: str = Field(description="One of: low, medium, high")
product: str
summary: str
extractor = llm.with_structured_output(Ticket)
ticket = extractor.invoke(f"Extract fields from this ticket:\n\n{ticket_text}")See Structured outputs for schema guidance that improves reliability.
Streaming
for chunk in llm.stream("Write a haiku about GPUs."):
print(chunk.content, end="", flush=True)Streaming is realtime by default. For long agent steps where nobody is waiting, prefer a background request — see below.
A LangGraph agent
from typing import Annotated, TypedDict
from langgraph.graph import StateGraph, END
from langgraph.prebuilt import ToolNode
class State(TypedDict):
messages: Annotated[list, "add_messages"]
def call_model(state: State):
return {"messages": [llm_with_tools.invoke(state["messages"])]}
graph = StateGraph(State)
graph.add_node("agent", call_model)
graph.add_node("tools", ToolNode([get_weather]))
graph.set_entry_point("agent")
graph.add_conditional_edges("agent", should_continue, {"tools": "tools", END: END})
graph.add_edge("tools", "agent")
app = graph.compile()
result = app.invoke({"messages": [("user", "What is the weather in Ljubljana?")]})Cost control in a graph
Two levers matter for a multi-step graph:
Pin the prefix cache. An agent resends the same system prompt and tool definitions on every node. Set prompt_cache_key to the graph run id so those tokens bill at the cached rate:
llm = ChatOpenAI(
model="zai-org/GLM-5.3",
base_url="https://api.sference.com/v1",
api_key=os.environ["SFERENCE_API_KEY"],
model_kwargs={"prompt_cache_key": run_id}, # stable per graph run
)Move background nodes to flex. Nodes whose output the user is not waiting on can run 30% cheaper:
background_llm = ChatOpenAI(
model="zai-org/GLM-5.3",
base_url="https://api.sference.com/v1",
api_key=os.environ["SFERENCE_API_KEY"],
model_kwargs={"service_tier": "flex"},
)Use background_llm for research, summarization, and retrieval-summarizing nodes, and llm for the nodes on the critical path. See Pricing and Prompt caching.
Bound the wait
Add max_queue_wait_s to fail fast under congestion instead of holding a graph node open:
llm = ChatOpenAI(
...,
model_kwargs={"max_queue_wait_s": 20},
)See Max queue wait for the failover pattern.
Build with agent frameworks
Build agent loops on sference with LangChain, the OpenAI Agents SDK, and the Anthropic Agent SDK — and pick the window each step runs on.
OpenAI Agents SDK
Run the OpenAI Agents SDK against sference: Responses API, tools, handoffs, guardrails, and background steps on the 24h window.