Xantly MCP Server, First LLM-Gateway MCP

Expose the Xantly LLM gateway routing, cache, memory, cost, and control surfaces as Model Context Protocol tools. 15 tools callable from Claude Desktop, Cursor, VS Code, Zed, ChatGPT Desktop. npx -y @xantly/mcp.

Status: Live. xantly-mcp ships with Xantly today, install via npx -y @xantly/mcp and wire it into Claude Desktop, Cursor, VS Code, or Zed. The first LLM-gateway MCP server.

TL;DR: LLM gateways today are black boxes. You send requests, they come back, and you squint at dashboards trying to understand why a request routed to one provider over another, or what the real cost breakdown looked like. We just flipped the script, Xantly now exposes its internals (routing decisions, cache, memory, cost, control) as Model Context Protocol tools. Meaning your AI agent inside Claude Desktop can just ask. Here's how.

What is MCP?

The Model Context Protocol is Anthropic's open standard for how AI assistants connect to tools, data, and services. Think of it as USB for LLMs, a single consistent way for Claude Desktop, Cursor, VS Code (via Cline or Continue), Zed, and ChatGPT Desktop to connect to any tool that speaks the protocol.

An MCP server exposes three things:

  1. Tools: functions the agent can call, with typed arguments and returns.
  2. Resources: URI-addressable data the agent can read.
  3. Prompts: templated workflows the user can invoke by name.

MCP was released in November 2024. One year later, a rapidly-growing ecosystem of servers exposes everything from Slack to Postgres to Stripe, but not a single LLM gateway has shipped one until now.

Why no other gateway has this

We checked every major LLM gateway today:

GatewayMCP server?
LiteLLMNo
OpenRouterNo
HeliconeNo
Portkey"MCP Gateway", but it's a gateway for MCP servers, not an MCP server for the gateway itself
Kong AI GatewayNo
TrueFoundryNo
EdenAINo
XantlyYes (shipping today)

Why hasn't anyone else done this? Two reasons:

  1. Their admin surfaces are REST APIs designed for dashboards, not agents. MCP demands typed schemas, semantic tool names, and prompt templates. Wrapping a generic HTTP client around an admin API isn't enough, you need to design tools specifically for an LLM caller.
  2. Most gateways don't have a single unified internals layer. Xantly does, x-core + xantly-api's BaRP router, pricing engine, cache service, memory router, and waterfall fallback are all in one Rust workspace. That made wrapping them in an MCP surface a natural fit.

The short version: we're positioned to be first, and being first in a new category matters.

What Xantly MCP exposes

Fifteen tools across six domains. Here's the quick tour.

Routing & Observability

Ask your AI agent things like:

Why was request abc-123 routed to Claude Sonnet instead of DeepSeek?

And get a detailed answer backed by explain_route, which returns the 396-dimensional BaRP feature vector, UCB scores per arm, chosen arm, exploration ratio, and the full waterfall fallback chain. It cites specific numbers.

Other tools in this group:

Cache

Memory

Cost & Budget

Control (write, Admin-only)

Completion

Plus four URI-addressable resources (xantly://tenants, xantly://request/{id}, xantly://cache/{fingerprint}, xantly://model/{provider}/{name}) and three prompt templates.

Install in 30 seconds

Three env vars in your MCP client config, and you're done.

Claude Desktop

Edit ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "xantly": {
      "command": "npx",
      "args": ["-y", "@xantly/mcp"],
      "env": {
        "XANTLY_API_KEY": "xantly_sk_..."
      }
    }
  }
}

Restart Claude Desktop. You'll see 15 new Xantly tools under the hammer icon.

Cursor, VS Code, Zed

The same shape works. Drop the config into .cursor/mcp.json, .vscode/cline_mcp_settings.json, or your Zed settings.json. Full per-client setup: see the integration guide.

Docker / Standalone binary

If you prefer a container or standalone binary, both work:

# Docker
docker run --rm -i -e XANTLY_API_KEY=xantly_sk_... ghcr.io/xantly/mcp:latest

# Standalone binary
curl -L -o /usr/local/bin/xantly-mcp \
  https://github.com/xantly/X-Engine/releases/latest/download/xantly-mcp-darwin-arm64
chmod +x /usr/local/bin/xantly-mcp

Hosted (Streamable HTTP, OAuth 2.1)

Point ChatGPT Desktop or any HTTP-speaking MCP client at https://mcp.xantly.com/v1. The server implements OAuth 2.1 per the November 2025 MCP revision, click through the grant flow to approve your read/write scopes.

The killer demo: cost-optimization-check

Here's the prompt that sells the whole thing.

Inside Claude Desktop, type:

/mcp prompts cost-optimization-check

The prompt template kicks off a multi-step analysis that:

  1. Calls cost_breakdown with window=24h, dimension=model to get top spenders.
  2. Calls routing_status to see provider health + BaRP rankings.
  3. Calls cache_stats to check the current L0/L1/L2 hit rate.
  4. Returns exactly 3 concrete routing tweaks with estimated $/month savings.

Example output against a real tenant's 24h data:

# 24h Cost Review, 
47 spend

**Summary:** Switch 40% of GPT-4o calls to Claude Sonnet + enable
semantic cache, $52/mo savings.

## Tweak 1: Route generic Q&A to DeepSeek-V2 (8/mo saved)
- Current: claude-3-5-sonnet @ 
.00/1M in, 
5.00/1M out
- Proposed: deepseek-chat @ $0.27/1M in, 
.10/1M out
- Affected traffic: 34% of requests have prompt_tokens <1000
  and no reasoning-required tags

## Tweak 2: Enable L2 semantic cache (
8/mo saved)
- Current hit rate: 31% (L0 + L1 only)
- Projected with L2 on: 48%
- 17-point lift × 
07/mo vendor cost = 
8/mo

## Tweak 3: Lower default preference_dial from 0.2 → 0.4 ($6/mo saved)
- Current: quality-biased (dial=0.2), avg $0.0043/req
- Proposed: balanced (dial=0.4), avg $0.0036/req (−16%)
- Quality delta on your test set: −2% win rate, well within tolerance

No dashboard. No CSV download. No SQL query against ClickHouse. The agent ran four tool calls, synthesized the data, and wrote the report in under 10 seconds.

That's the whole pitch.

Another demo: debug a failed request

/mcp prompts waterfall-post-mortem

With request_id=abc-123. The agent calls waterfall_trace, explain_route, and budget_status, then produces a concise incident report:

# Waterfall post-mortem for abc-123

**Summary:** request failed 5 steps deep in the fallback chain due to a
context-window overflow; all NVIDIA keys were billing-exhausted.

## Chain
1. claude-3-5-sonnet, 413 `ContextWindowExceeded` (input 210k tokens,
   model cap 200k)
2. gpt-4o, same, cap 128k
3. claude-3-5-haiku, 413 `ContextWindowExceeded`, cap 200k
4. deepseek-chat, 500 `InternalError`
5. nvidia llama-70b (key fp a831), 402 `BillingExhausted`
6. nvidia llama-70b (key fp b22c), 402 `BillingExhausted`
7. Fatal, no more providers

## Root cause
Request body shipped a 210k-token prompt. The intelligence-mode resolver
selected `full` (tool planning + reasoning), which routed to T1
providers first. All T1 models share a ≤200k context cap; none could
serve. The waterfall correctly escalated to T2 (DeepSeek) and T3 (NVIDIA)
but both were exhausted in different ways.

## Remediation
1. Client-side: implement context pre-flight, if prompt >180k tokens,
   auto-switch to Gemini 2.5 Pro (1M context) before sending.
2. Top up or rotate the NVIDIA key pool. Use `rotate_key(provider=nvidia)`
   from this MCP server.
3. Consider enabling the context-compaction healer (see
   `docs/architecture/output-verification`).

What ships today

What's next

If there's a tool you want us to add, particularly something domain-specific to your workload, open an issue in the X-Engine repo with [mcp] in the title.

Try it now

# 1. Create an API key
open https://xantly.com/dashboard/api-keys

# 2. Drop this in your Claude Desktop config
cat >> ~/Library/Application\ Support/Claude/claude_desktop_config.json <<'EOF'
{
  "mcpServers": {
    "xantly": {
      "command": "npx",
      "args": ["-y", "@xantly/mcp"],
      "env": { "XANTLY_API_KEY": "xantly_sk_YOUR_KEY" }
    }
  }
}
EOF

# 3. Restart Claude Desktop, hit the hammer icon, ask:
#    "Which of my models cost the most yesterday?"

Questions? We read every reply. [email protected]