Use Xantly with CrewAI

CrewAI is a multi-agent orchestration framework, define agents with roles, assemble them into a crew, run a task. Under the hood it uses LiteLLM, which means any OpenAI-compatible endpoint plugs in. Xantly fits natively, and the semantic cache is disproportionately valuable for multi-agent workloads where agents re-ask similar questions.

Prerequisites

• Python 3.10+
• pip install crewai
• A Xantly API key, create one

Setup

CrewAI reads standard OpenAI env vars. Set two:

export OPENAI_API_BASE=https://api.xantly.com/v1
export OPENAI_API_KEY=xantly_sk_...
export OPENAI_MODEL_NAME=xantly/auto-quality

Every Agent, Task, Crew now routes through Xantly. No code changes.

In-code config

from crewai import Agent, Task, Crew, LLM

llm = LLM(
    model="openai/xantly/auto-quality",  # crewai/litellm prefix
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)

researcher = Agent(
    role="Senior Research Analyst",
    goal="Find key facts about {topic}",
    backstory="You cross-reference sources and cite them.",
    llm=llm,
)

writer = Agent(
    role="Technical Writer",
    goal="Write a concise summary",
    backstory="You turn dense research into 3-paragraph explainers.",
    llm=llm,
)

research = Task(
    description="Research {topic}.",
    expected_output="A bulleted list of 10 key facts with sources.",
    agent=researcher,
)

summarize = Task(
    description="Write a summary based on the research.",
    expected_output="A 3-paragraph explainer in plain English.",
    agent=writer,
)

crew = Crew(agents=[researcher, writer], tasks=[research, summarize])
result = crew.kickoff(inputs={"topic": "vector databases"})
print(result)

Per-agent model assignment

Different agents for different roles, use different Xantly tiers:

researcher = Agent(
    role="Researcher",
    llm=LLM(
        model="openai/xantly/auto-quality",
        base_url="https://api.xantly.com/v1",
        api_key="xantly_sk_...",
    ),
    # ...
)

summarizer = Agent(
    role="Summarizer",
    llm=LLM(
        model="openai/xantly/auto-value",
        base_url="https://api.xantly.com/v1",
        api_key="xantly_sk_...",
    ),
    # ...
)

Tools

CrewAI's tool system works unchanged:

from crewai.tools import tool

@tool("search_web")
def search_web(query: str) -> str:
    """Search the web for a query."""
    return "<search results for " + query + ">"

researcher = Agent(
    role="Researcher",
    tools=[search_web],
    llm=llm,
    # ...
)

Use anthropic/claude-sonnet-4.6 as the model for tool-heavy agents, reliable tool-calling is critical in multi-agent flows.

Model choice

Verify

from crewai import Agent, Task, Crew, LLM

llm = LLM(
    model="openai/xantly/auto-speed",
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)
agent = Agent(role="Pinger", goal="Respond", backstory="You say pong.", llm=llm)
task = Task(description="Say pong.", expected_output="pong", agent=agent)
crew = Crew(agents=[agent], tasks=[task])
print(crew.kickoff())

What you get

• Cache hits on multi-agent re-ask. Agents in a crew often query overlapping contexts, L2 semantic cache turns the repeats into near-zero cost.
• Waterfall fallback per agent. One agent's provider goes down mid-crew, Xantly transparently retries on another model. The crew doesn't derail.
• Per-agent cost tracking. Each agent's calls log individually in your Xantly dashboard.
• Memory across crew runs. Set X-Xantly-Memory-Mode: recall via llm.additional_drop_params to share learnings across runs.
• Tool-call translation. Mixed-provider crews (one agent on Claude, another on GPT) keep working regardless of which tool format the underlying provider uses.

Gotchas

The openai/ prefix is litellm-required. CrewAI routes through LiteLLM, which requires a provider prefix. Use openai/xantly/auto-quality (OpenAI is the protocol, not the model). Full slug: openai/<xantly-model-id>.

/v1 required in base_url.

Long crew runs exceed default timeouts. Set llm=LLM(..., timeout=300) for crews that run for minutes.

Parallel agent execution. CrewAI supports async crews (Crew(async_execution=True)). Xantly handles the concurrency, but watch your per-minute rate limits, upgrade your plan if you run big crews.

Next steps

• AutoGen, Microsoft's multi-agent framework.
• LangGraph, graph-based agent runtime.
• Multi-Agent Orchestration, Xantly-native agent chaining.
• Cost-Optimized Routing, how to keep crew bills down.