Use Xantly with AutoGen

AutoGen is Microsoft's multi-agent conversation framework. Agents exchange messages, run code, call tools, and collaborate. Its OpenAIChatCompletionClient accepts a base_url, so pointing it at Xantly gives every agent conversation smart routing, semantic cache, and memory, without touching AutoGen internals.

Prerequisites

• Python 3.10+
• pip install "autogen-agentchat" "autogen-ext[openai]" (AutoGen v0.4+)
• A Xantly API key, create one

Setup

from autogen_ext.models.openai import OpenAIChatCompletionClient

model_client = OpenAIChatCompletionClient(
    model="xantly/auto-quality",
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)

For AutoGen v0.2 (legacy):

config_list = [{
    "model": "xantly/auto-quality",
    "api_key": "xantly_sk_...",
    "base_url": "https://api.xantly.com/v1",
    "api_type": "openai",
}]

Two-agent conversation (v0.4)

import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import TextMentionTermination
from autogen_ext.models.openai import OpenAIChatCompletionClient

async def main():
    client = OpenAIChatCompletionClient(
        model="xantly/auto-quality",
        base_url="https://api.xantly.com/v1",
        api_key="xantly_sk_...",
    )

    researcher = AssistantAgent(
        name="researcher",
        model_client=client,
        system_message="You research topics and present facts. Cite your sources.",
    )

    critic = AssistantAgent(
        name="critic",
        model_client=client,
        system_message="You critique the researcher's output. When satisfied, say APPROVED.",
    )

    team = RoundRobinGroupChat(
        [researcher, critic],
        termination_condition=TextMentionTermination("APPROVED"),
    )
    await team.run(task="Find 5 key facts about vector databases.")

asyncio.run(main())

Tools

from autogen_agentchat.agents import AssistantAgent

def multiply(a: int, b: int) -> int:
    """Multiply two integers."""
    return a * b

assistant = AssistantAgent(
    name="assistant",
    model_client=client,
    tools=[multiply],
)

Use anthropic/claude-sonnet-4.6 as the model for tool-heavy agents.

Per-agent models

Different agents on different Xantly tiers:

researcher_client = OpenAIChatCompletionClient(
    model="xantly/auto-quality",
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)

critic_client = OpenAIChatCompletionClient(
    model="xantly/auto-value",  # cheaper for critique
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)

Code execution

AutoGen's CodeExecutorAgent pairs with DockerCommandLineCodeExecutor. Model calls (reasoning about code) route through Xantly; actual execution happens locally, no Xantly involvement in the sandbox. Set the reasoning agent's model to anthropic/claude-sonnet-4.6 for best code-reasoning quality.

Model choice

Verify

import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.messages import TextMessage
from autogen_core import CancellationToken
from autogen_ext.models.openai import OpenAIChatCompletionClient

async def main():
    client = OpenAIChatCompletionClient(
        model="xantly/auto-speed",
        base_url="https://api.xantly.com/v1",
        api_key="xantly_sk_...",
    )
    agent = AssistantAgent("pinger", model_client=client)
    resp = await agent.on_messages(
        [TextMessage(content="say pong", source="user")],
        cancellation_token=CancellationToken(),
    )
    print(resp.chat_message.content)

asyncio.run(main())

What you get

• Multi-agent cache wins. AutoGen agents re-ask similar questions in round-robin conversations, L2 semantic cache turns repeats into near-zero cost.
• Waterfall fallback per agent. A single agent's model going down doesn't derail the whole team.
• Per-agent cost breakdown in your Xantly dashboard.
• Memory across runs. Each team's agents can share Xantly L3 memory via X-Xantly-Memory-Mode: recall in model-client headers.
• Code-execution safety. AutoGen sandboxes code in Docker; Xantly only sees the LLM calls, never executes anything.

Gotchas

v0.2 vs v0.4. The config shape changed in v0.4. This doc shows v0.4. For v0.2 you use config_list with base_url per entry.

/v1 required.

AutoGen v0.4 requires model_info for non-OpenAI models. If you see "unknown model" errors, pass model_info={"vision": False, "function_calling": True, "json_output": True, "family": "unknown", "structured_output": True} to OpenAIChatCompletionClient.

Streaming. v0.4's on_messages_stream works; Xantly delivers SSE deltas.

Next steps

• CrewAI, alternative multi-agent framework.
• LangGraph, graph-based agent runtime.
• Multi-Agent Orchestration, Xantly-native.
• Cost-Optimized Routing, keeping team bills down.