Use Xantly with LangGraph

LangGraph is LangChain's graph-based agent runtime, build stateful, cyclic agent workflows as nodes + edges. Because it uses LangChain's ChatOpenAI / ChatAnthropic under the hood, pointing those models at Xantly routes every node's LLM call through smart routing, semantic cache, and memory.

Prerequisites

• Python 3.9+
• pip install langgraph langchain-openai
• A Xantly API key, create one

Setup

from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, END

llm = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-quality",
)

Minimal graph

from typing import TypedDict
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from langgraph.graph import StateGraph, END

class State(TypedDict):
    messages: list

llm = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-quality",
)

def call_model(state: State):
    response = llm.invoke(state["messages"])
    return {"messages": state["messages"] + [response]}

graph = StateGraph(State)
graph.add_node("model", call_model)
graph.set_entry_point("model")
graph.add_edge("model", END)
app = graph.compile()

result = app.invoke({"messages": [HumanMessage(content="Hi there")]})
print(result["messages"][-1].content)

ReAct agent (prebuilt)

from langgraph.prebuilt import create_react_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool

@tool
def multiply(a: int, b: int) -> int:
    """Multiply two integers."""
    return a * b

llm = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="anthropic/claude-sonnet-4.6",  # strong tool-calling
)

agent = create_react_agent(llm, tools=[multiply])
result = agent.invoke({"messages": [("user", "What is 17 * 23?")]})
print(result["messages"][-1].content)

Per-node models

LangGraph's killer move: different nodes can use different Xantly tiers.

from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, END

quality = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-quality",
)

value = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-value",
)

def plan(state):
    return {"plan": quality.invoke(f"Plan: {state['input']}").content}

def execute(state):
    return {"output": value.invoke(f"Execute {state['plan']}").content}

• Planning node → quality tier (reasoning matters).
• Execution nodes → value tier (cheaper).
• Summarization nodes → speed tier.

Checkpointing + memory

LangGraph's MemorySaver preserves graph state across runs:

from langgraph.checkpoint.memory import MemorySaver

checkpointer = MemorySaver()
app = graph.compile(checkpointer=checkpointer)

config = {"configurable": {"thread_id": "user-42"}}
app.invoke({"messages": [...]}, config=config)
# next call with same thread_id resumes state

Pair it with Xantly's L3 memory by setting X-Xantly-Memory-Mode: recall on the LLM's default_headers, LangGraph handles conversation state, Xantly handles cross-session knowledge.

Streaming

for event in app.stream({"messages": [HumanMessage(content="Stream a haiku")]}):
    for key, value in event.items():
        print(key, value)

Model choice

Verify

llm = ChatOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
    model="xantly/auto-speed",
)
print(llm.invoke("say pong").content)

What you get

• Per-node routing. Use Xantly Quality for expensive planning, Value for cheap executors.
• Cycle safety. Agent loops that re-ask similar questions hit semantic cache, cheaper + faster.
• Waterfall fallback per node. One provider outage doesn't derail a long-running graph.
• Cost per node. Each node's LLM call logs separately in your Xantly dashboard.
• Checkpoint interop. LangGraph's MemorySaver + Xantly's L3 memory, complementary, not competing.

Gotchas

LangGraph doesn't ship its own LLM client. It uses LangChain's models. All base-URL config happens on ChatOpenAI / ChatAnthropic, see LangChain (Python) for details.

Tool-calling failures in cycles. If a model can't reliably emit tool calls, LangGraph will loop forever. Use anthropic/claude-sonnet-4.6 or set a recursion_limit on graph.compile(...).

Async graphs. app.ainvoke(...) and app.astream(...) work; the underlying OpenAI SDK handles async.

State type hints. TypedDict is recommended, LangGraph introspects it for the graph schema.

Next steps

• LangChain (Python), the underlying library.
• CrewAI, alternative multi-agent approach.
• AutoGen, Microsoft's multi-agent framework.
• Multi-Agent Orchestration, Xantly-native graph orchestration.