# LangChain

> Use sference as the model provider for LangChain and LangGraph agents: ChatOpenAI, tool calling, streaming, and per-step windows.

Source: https://sference.com/docs/agents/langchain

sference exposes an OpenAI-compatible `/v1/chat/completions`, so LangChain's `ChatOpenAI` works with a base-URL change. Chains, tools, and LangGraph graphs then run on open models with the same code.

## Install

```bash
pip install -U langchain-openai
```

## Point `ChatOpenAI` at sference

```python
import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="zai-org/GLM-5.3",
    base_url="https://api.sference.com/v1",
    api_key=os.environ["SFERENCE_API_KEY"],
    temperature=0.2,
)
```

Any LangChain component that takes a chat model now runs on sference — chains, output parsers, retrievers, and LangGraph nodes.

## Tool calling

LangChain's tool decorator binds to the OpenAI tool-calling shape, which sference serves natively:

```python
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Return the current weather for a city."""
    return weather_api(city)

llm_with_tools = llm.bind_tools([get_weather])
response = llm_with_tools.invoke("What is the weather in Ljubljana?")
```

> **Tool reliability varies by model**
>
> Open models call tools less consistently than frontier closed models on long loops. `moonshotai/Kimi-K3` and `zai-org/GLM-5.3` are the strongest tool callers in the catalog; validate a tool-heavy workflow on your own prompts before committing to it.

## Structured output

Use `with_structured_output` for typed results — it maps to `response_format` with a JSON schema:

```python
from pydantic import BaseModel, Field

class Ticket(BaseModel):
    severity: str = Field(description="One of: low, medium, high")
    product: str
    summary: str

extractor = llm.with_structured_output(Ticket)
ticket = extractor.invoke(f"Extract fields from this ticket:\n\n{ticket_text}")
```

See [Structured outputs](https://sference.com/docs/guides/structured-outputs) for schema guidance that improves reliability.

## Streaming

```python
for chunk in llm.stream("Write a haiku about GPUs."):
    print(chunk.content, end="", flush=True)
```

Streaming is realtime by default. For long agent steps where nobody is waiting, prefer a background request — see below.

## A LangGraph agent

```python
from typing import Annotated, TypedDict
from langgraph.graph import StateGraph, END
from langgraph.prebuilt import ToolNode

class State(TypedDict):
    messages: Annotated[list, "add_messages"]

def call_model(state: State):
    return {"messages": [llm_with_tools.invoke(state["messages"])]}

graph = StateGraph(State)
graph.add_node("agent", call_model)
graph.add_node("tools", ToolNode([get_weather]))
graph.set_entry_point("agent")
graph.add_conditional_edges("agent", should_continue, {"tools": "tools", END: END})
graph.add_edge("tools", "agent")

app = graph.compile()
result = app.invoke({"messages": [("user", "What is the weather in Ljubljana?")]})
```

## Cost control in a graph

Two levers matter for a multi-step graph:

**Pin the prefix cache.** An agent resends the same system prompt and tool definitions on every node. Set `prompt_cache_key` to the graph run id so those tokens bill at the cached rate:

```python
llm = ChatOpenAI(
    model="zai-org/GLM-5.3",
    base_url="https://api.sference.com/v1",
    api_key=os.environ["SFERENCE_API_KEY"],
    model_kwargs={"prompt_cache_key": run_id},   # stable per graph run
)
```

**Move background nodes to flex.** Nodes whose output the user is not waiting on can run 30% cheaper:

```python
background_llm = ChatOpenAI(
    model="zai-org/GLM-5.3",
    base_url="https://api.sference.com/v1",
    api_key=os.environ["SFERENCE_API_KEY"],
    model_kwargs={"service_tier": "flex"},
)
```

Use `background_llm` for research, summarization, and retrieval-summarizing nodes, and `llm` for the nodes on the critical path. See [Pricing](https://sference.com/docs/pricing) and [Prompt caching](https://sference.com/docs/guides/prompt-caching).

## Bound the wait

Add `max_queue_wait_s` to fail fast under congestion instead of holding a graph node open:

```python
llm = ChatOpenAI(
    ...,
    model_kwargs={"max_queue_wait_s": 20},
)
```

See [Max queue wait](https://sference.com/docs/guides/max-queue-wait) for the failover pattern.

- [Agent SDKs](https://sference.com/docs/agents): Compare frameworks and per-step window choices.
- [Prompt caching](https://sference.com/docs/guides/prompt-caching): The largest lever on an agent's bill.
- [Structured outputs](https://sference.com/docs/guides/structured-outputs): Typed results from with_structured_output.