# Xantly MCP Server, Your LLM Gateway is Now a First-Class MCP Tool

> **Status:** Live. `xantly-mcp` ships with Xantly today, install via `npx -y @xantly/mcp` and wire it into Claude Desktop, Cursor, VS Code, or Zed. The first LLM-gateway MCP server.

**TL;DR:** LLM gateways today are black boxes. You send requests, they come back, and you squint at dashboards trying to understand why a request routed to one provider over another, or what the real cost breakdown looked like. We just flipped the script, Xantly now exposes its internals (routing decisions, cache, memory, cost, control) as [Model Context Protocol](https://modelcontextprotocol.io/) tools. Meaning your AI agent inside Claude Desktop can just *ask*. Here's how.

---

## What is MCP?

The [Model Context Protocol](https://modelcontextprotocol.io/specification/2025-11-25) is Anthropic's open standard for how AI assistants connect to tools, data, and services. Think of it as USB for LLMs, a single consistent way for Claude Desktop, Cursor, VS Code (via Cline or Continue), Zed, and ChatGPT Desktop to connect to any tool that speaks the protocol.

An MCP server exposes three things:

1. **Tools**: functions the agent can call, with typed arguments and returns.
2. **Resources**: URI-addressable data the agent can read.
3. **Prompts**: templated workflows the user can invoke by name.

MCP was released in November 2024. One year later, a rapidly-growing ecosystem of servers exposes everything from Slack to Postgres to Stripe, but not a single LLM gateway has shipped one until now.

## Why no other gateway has this

We checked every major LLM gateway today:

| Gateway | MCP server? |
|---|---|
| LiteLLM | No |
| OpenRouter | No |
| Helicone | No |
| Portkey | "MCP Gateway", but it's a *gateway for MCP servers*, not an MCP server for the gateway itself |
| Kong AI Gateway | No |
| TrueFoundry | No |
| EdenAI | No |
| **Xantly** | **Yes (shipping today)** |

Why hasn't anyone else done this? Two reasons:

1. **Their admin surfaces are REST APIs designed for dashboards, not agents.** MCP demands typed schemas, semantic tool names, and prompt templates. Wrapping a generic HTTP client around an admin API isn't enough, you need to design tools specifically for an LLM caller.
2. **Most gateways don't have a single unified internals layer.** Xantly does, `x-core` + `xantly-api`'s BaRP router, pricing engine, cache service, memory router, and waterfall fallback are all in one Rust workspace. That made wrapping them in an MCP surface a natural fit.

The short version: we're positioned to be first, and being first in a new category matters.

## What Xantly MCP exposes

Fifteen tools across six domains. Here's the quick tour.

### Routing & Observability

Ask your AI agent things like:

> Why was request `abc-123` routed to Claude Sonnet instead of DeepSeek?

And get a detailed answer backed by `explain_route`, which returns the 396-dimensional BaRP feature vector, UCB scores per arm, chosen arm, exploration ratio, and the full waterfall fallback chain. It cites specific numbers.

Other tools in this group:

- `waterfall_trace`, full fallback chain for a request, step by step.
- `routing_status`, live provider health matrix (p95 latency, error rate, 402 rate per 10-min window) plus BaRP rankings.
- `barp_weights`, LinUCB `θ̂` vector, A-matrix trace, update count for a tenant.

### Cache

- `inspect_cache`, glob search over L0/L1/L2 with TTL and hit count.
- `cache_stats`, hit rates per layer, bytes stored, USD savings from cache hits.

### Memory

- `memory_recall`, semantic search over Qdrant (L2) + graph neighbors from Neo4j (L3).
- `memory_store`, persist a fact into L2 + L3.

### Cost & Budget

- `cost_breakdown`, aggregate cost rows by model/provider/tier/key/project/env.
- `cost_estimate`, pre-flight quote before you send a request.
- `budget_status`, remaining budget, burn rate, 80%-threshold forecast.

### Control (write, Admin-only)

- `set_intelligence_mode`, flip the tenant's default mode (proxy / cache / full).
- `preference_dial`, 0.0 (quality-biased) to 1.0 (cost-biased).
- `rotate_key`, force the next waterfall pass to pick a different provider credential.

### Completion

- `completion_run`, run a chat completion through the gateway; `mode=agentic` runs the full tool-planning pipeline.

Plus four URI-addressable resources (`xantly://tenants`, `xantly://request/{id}`, `xantly://cache/{fingerprint}`, `xantly://model/{provider}/{name}`) and three prompt templates.

## Install in 30 seconds

Three env vars in your MCP client config, and you're done.

### Claude Desktop

Edit `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or `%APPDATA%\Claude\claude_desktop_config.json` (Windows):

```json
{
  "mcpServers": {
    "xantly": {
      "command": "npx",
      "args": ["-y", "@xantly/mcp"],
      "env": {
        "XANTLY_API_KEY": "xantly_sk_..."
      }
    }
  }
}
```

Restart Claude Desktop. You'll see 15 new Xantly tools under the hammer icon.

### Cursor, VS Code, Zed

The same shape works. Drop the config into `.cursor/mcp.json`, `.vscode/cline_mcp_settings.json`, or your Zed `settings.json`. Full per-client setup: see the [integration guide](https://github.com/xantly/X-Engine/blob/main/crates/xantly-mcp/INTEGRATION.md).

### Docker / Standalone binary

If you prefer a container or standalone binary, both work:

```bash
# Docker
docker run --rm -i -e XANTLY_API_KEY=xantly_sk_... ghcr.io/xantly/mcp:latest

# Standalone binary
curl -L -o /usr/local/bin/xantly-mcp \
  https://github.com/xantly/X-Engine/releases/latest/download/xantly-mcp-darwin-arm64
chmod +x /usr/local/bin/xantly-mcp
```

### Hosted (Streamable HTTP, OAuth 2.1)

Point ChatGPT Desktop or any HTTP-speaking MCP client at `https://mcp.xantly.com/v1`. The server implements OAuth 2.1 per the November 2025 MCP revision, click through the grant flow to approve your read/write scopes.

## The killer demo: `cost-optimization-check`

Here's the prompt that sells the whole thing.

Inside Claude Desktop, type:

> /mcp prompts cost-optimization-check

The prompt template kicks off a multi-step analysis that:

1. Calls `cost_breakdown` with `window=24h, dimension=model` to get top spenders.
2. Calls `routing_status` to see provider health + BaRP rankings.
3. Calls `cache_stats` to check the current L0/L1/L2 hit rate.
4. Returns exactly 3 concrete routing tweaks with estimated $/month savings.

Example output against a real tenant's 24h data:

```markdown
# 24h Cost Review, $147 spend

**Summary:** Switch 40% of GPT-4o calls to Claude Sonnet + enable
semantic cache, $52/mo savings.

## Tweak 1: Route generic Q&A to DeepSeek-V2 ($28/mo saved)
- Current: claude-3-5-sonnet @ $3.00/1M in, $15.00/1M out
- Proposed: deepseek-chat @ $0.27/1M in, $1.10/1M out
- Affected traffic: 34% of requests have prompt_tokens <1000
  and no reasoning-required tags

## Tweak 2: Enable L2 semantic cache ($18/mo saved)
- Current hit rate: 31% (L0 + L1 only)
- Projected with L2 on: 48%
- 17-point lift × $107/mo vendor cost = $18/mo

## Tweak 3: Lower default preference_dial from 0.2 → 0.4 ($6/mo saved)
- Current: quality-biased (dial=0.2), avg $0.0043/req
- Proposed: balanced (dial=0.4), avg $0.0036/req (−16%)
- Quality delta on your test set: −2% win rate, well within tolerance
```

No dashboard. No CSV download. No SQL query against ClickHouse. The agent ran four tool calls, synthesized the data, and wrote the report in under 10 seconds.

That's the whole pitch.

## Another demo: debug a failed request

> /mcp prompts waterfall-post-mortem

With `request_id=abc-123`. The agent calls `waterfall_trace`, `explain_route`, and `budget_status`, then produces a concise incident report:

```markdown
# Waterfall post-mortem for abc-123

**Summary:** request failed 5 steps deep in the fallback chain due to a
context-window overflow; all NVIDIA keys were billing-exhausted.

## Chain
1. claude-3-5-sonnet, 413 `ContextWindowExceeded` (input 210k tokens,
   model cap 200k)
2. gpt-4o, same, cap 128k
3. claude-3-5-haiku, 413 `ContextWindowExceeded`, cap 200k
4. deepseek-chat, 500 `InternalError`
5. nvidia llama-70b (key fp a831), 402 `BillingExhausted`
6. nvidia llama-70b (key fp b22c), 402 `BillingExhausted`
7. Fatal, no more providers

## Root cause
Request body shipped a 210k-token prompt. The intelligence-mode resolver
selected `full` (tool planning + reasoning), which routed to T1
providers first. All T1 models share a ≤200k context cap; none could
serve. The waterfall correctly escalated to T2 (DeepSeek) and T3 (NVIDIA)
but both were exhausted in different ways.

## Remediation
1. Client-side: implement context pre-flight, if prompt >180k tokens,
   auto-switch to Gemini 2.5 Pro (1M context) before sending.
2. Top up or rotate the NVIDIA key pool. Use `rotate_key(provider=nvidia)`
   from this MCP server.
3. Consider enabling the context-compaction healer (see
   `docs/architecture/output-verification`).
```

## What ships today

- **15 tools, 4 resources, 3 prompts**: covering routing, cache, memory, cost, control, completion.
- **stdio and Streamable HTTP transports**: stdio for desktop clients, HTTP for ChatGPT Desktop and anyone running remote.
- **OAuth 2.1 for HTTP**: RS256 JWT verification, scope-based authorization, `/.well-known/oauth-protected-resource` metadata endpoint.
- **Native Rust**: no Python, no Node runtime in the hot path. Direct reuse of `x-core`, `xantly-api` services. No FFI, no duplicate implementation.
- **Published to six registries**: Official MCP Registry, smithery.ai, mcp.so, Glama, PulseMCP, Awesome-MCP-Servers (see submission status).
- **npm, Docker, `.mcpb`, standalone binary**: install whichever way you like.

## What's next

- **Week 2 of rollout:** all 15 tools live with real admin backends (today, 3 backends are TBD, see `BACKEND_REQUIREMENTS.md`).
- **Week 3:** Hosted `mcp.xantly.com` endpoint live with OAuth 2.1.
- **Week 4+:** More prompts (`model-selection-advisor`, `quota-forecast`), plus `subscribe` support for real-time tool-call streams.

If there's a tool you want us to add, particularly something domain-specific to your workload, open an issue in the [X-Engine repo](https://github.com/xantly/X-Engine/issues) with `[mcp]` in the title.

## Try it now

```bash
# 1. Create an API key
open https://xantly.com/dashboard/api-keys

# 2. Drop this in your Claude Desktop config
cat >> ~/Library/Application\ Support/Claude/claude_desktop_config.json <<'EOF'
{
  "mcpServers": {
    "xantly": {
      "command": "npx",
      "args": ["-y", "@xantly/mcp"],
      "env": { "XANTLY_API_KEY": "xantly_sk_YOUR_KEY" }
    }
  }
}
EOF

# 3. Restart Claude Desktop, hit the hammer icon, ask:
#    "Which of my models cost the most yesterday?"
```

## Links

- Source: [`crates/xantly-mcp/`](https://github.com/xantly/X-Engine/tree/main/crates/xantly-mcp)
- Architecture spec: `docs-internal-src/mcp-server-architecture.md`
- Integration guide: [`INTEGRATION.md`](https://github.com/xantly/X-Engine/blob/main/crates/xantly-mcp/INTEGRATION.md)
- MCP specification: <https://modelcontextprotocol.io/specification/2025-11-25>
- Discord: join #mcp-launch in [our community](https://xantly.com/community)

---

*Questions? We read every reply.* team@xantly.com