Xantly MCP Server, an MCP Surface Over the Gateway

Status: live. The server runs at https://api.xantly.com/mcp over Streamable HTTP. Authenticate with your Xantly API key as a bearer token and point Claude Desktop, Cursor, VS Code or Zed at that URL as a remote MCP server. The RFC 9728 metadata a client probes for is at https://api.xantly.com/.well-known/oauth-protected-resource/mcp. There is no package to install: the hosted endpoint is how you use it.

TL;DR: LLM gateways today are black boxes. You send requests, they come back, and you squint at dashboards trying to understand why a request routed to one provider over another, or what the real cost breakdown looked like. We built the other thing: a Model Context Protocol surface over the gateway's own internals (routing decisions, cache, memory, cost, control), so that an AI agent inside Claude Desktop could just ask rather than squint. This page is the design tour. It is not yet something you can install, for the reason in the status note above.

What is MCP?

The Model Context Protocol is Anthropic's open standard for how AI assistants connect to tools, data, and services. Think of it as USB for LLMs, a single consistent way for Claude Desktop, Cursor, VS Code (via Cline or Continue), Zed, and ChatGPT Desktop to connect to any tool that speaks the protocol.

An MCP server exposes three things:

1. Tools: functions the agent can call, with typed arguments and returns.
2. Resources: URI-addressable data the agent can read.
3. Prompts: templated workflows the user can invoke by name.

MCP was released in November 2024. One year later, a rapidly-growing ecosystem of servers exposes everything from Slack to Postgres to Stripe, and as far as we can tell no LLM gateway has shipped one for its own internals.

Why no other gateway has this

We checked every major LLM gateway today:

Why hasn't anyone else done this? Two reasons:

1. Their admin surfaces are REST APIs designed for dashboards, not agents. MCP demands typed schemas, semantic tool names, and prompt templates. Wrapping a generic HTTP client around an admin API isn't enough, you need to design tools specifically for an LLM caller.
2. Most gateways don't have a single unified internals layer. Xantly does, x-core + xantly-api's BaRP router, pricing engine, cache service, memory router, and waterfall fallback are all in one Rust workspace. That made wrapping them in an MCP surface a natural fit.

The short version: the hard part for us was already done, so the MCP surface was cheap to build. Shipping it to a public registry is a separate piece of work that has not happened yet.

What Xantly MCP exposes

Fifteen tools across six domains. Here's the quick tour.

Routing & Observability

Ask your AI agent things like:

Why was request abc-123 routed to Claude Sonnet instead of DeepSeek?

And get a detailed answer backed by explain_route, which returns the 396-dimensional BaRP feature vector, UCB scores per arm, chosen arm, exploration ratio, and the full waterfall fallback chain. It cites specific numbers.

Other tools in this group:

• waterfall_trace, full fallback chain for a request, step by step.
• routing_status, live provider health matrix (p95 latency, error rate, 402 rate per 10-min window) plus BaRP rankings.
• barp_weights, LinUCB θ̂ vector, A-matrix trace, update count for a tenant.

Cache

• inspect_cache, glob search over L0/L1/L2 with TTL and hit count.
• cache_stats, hit rates per layer, bytes stored, USD savings from cache hits.

Memory

• memory_recall, semantic search over Qdrant (L2) + graph neighbors from Neo4j (L3).
• memory_store, persist a fact into L2 + L3.

Cost & Budget

• cost_breakdown, aggregate cost rows by model/provider/tier/key/project/env.
• cost_estimate, pre-flight quote before you send a request.
• budget_status, remaining budget, burn rate, 80%-threshold forecast.

Control (write, Admin-only)

• set_intelligence_mode, flip the tenant's default mode (proxy / cache / full).
• preference_dial, 0.0 (quality-biased) to 1.0 (cost-biased).
• rotate_key, force the next waterfall pass to pick a different provider credential.

Completion

• completion_run, run a chat completion through the gateway; mode=agentic runs the full tool-planning pipeline.

Plus four URI-addressable resources (xantly://tenants, xantly://request/{id}, xantly://cache/{fingerprint}, xantly://model/{provider}/{name}) and three prompt templates.

Installing it: there is no path yet

This is the part that does not work, so it gets its own heading rather than a footnote.

The transports the crate implements are stdio and Streamable HTTP with OAuth 2.1, so when a distribution channel does exist the wiring will be the ordinary MCP client configuration and nothing exotic. Until then, use the REST API and the dashboard, which cover the same ground. Details in What you can do today.

Questions about connecting a client to the hosted endpoint go to support@xantly.com.

The demo it was built for: cost-optimization-check

This is the prompt the whole design is aimed at. It is a walkthrough of the intended behaviour, not something you can run yet, for the reason above.

The prompt template, invoked from an MCP client:

/mcp prompts cost-optimization-check

The prompt template kicks off a multi-step analysis that:

1. Calls cost_breakdown with window=24h, dimension=model to get top spenders.
2. Calls routing_status to see provider health + BaRP rankings.
3. Calls cache_stats to check the current L0/L1/L2 hit rate.
4. Returns exactly 3 concrete routing tweaks with estimated $/month savings.

Illustrative output, with figures made up to show the shape:

# 24h Cost Review, $147 spend

**Summary:** Switch 40% of GPT-4o calls to Claude Sonnet + enable
semantic cache, $52/mo savings.

## Tweak 1: Route generic Q&A to DeepSeek-V2 ($28/mo saved)
- Current: claude-3-5-sonnet @ $3.00/1M in, $15.00/1M out
- Proposed: deepseek-chat @ $0.27/1M in, $1.10/1M out
- Affected traffic: 34% of requests have prompt_tokens <1000
  and no reasoning-required tags

## Tweak 2: Enable L2 semantic cache ($18/mo saved)
- Current hit rate: 31% (L0 + L1 only)
- Projected with L2 on: 48%
- 17-point lift × $107/mo vendor cost = $18/mo

## Tweak 3: Lower default preference_dial from 0.2 → 0.4 ($6/mo saved)
- Current: quality-biased (dial=0.2), avg $0.0043/req
- Proposed: balanced (dial=0.4), avg $0.0036/req (−16%)
- Quality delta on your test set: −2% win rate, well within tolerance

No dashboard, no CSV download, no SQL query: four tool calls, synthesized into a report. That is the argument for the surface. Today the same figures are available from the cost and routing views in the dashboard and from the usage endpoints on the REST API.

Another intended workflow: debug a failed request

/mcp prompts waterfall-post-mortem

With request_id=abc-123. The agent would call waterfall_trace, explain_route, and budget_status, then produce a concise incident report. Again, illustrative:

# Waterfall post-mortem for abc-123

**Summary:** request failed 5 steps deep in the fallback chain due to a
context-window overflow; all NVIDIA keys were billing-exhausted.

## Chain
1. claude-3-5-sonnet, 413 `ContextWindowExceeded` (input 210k tokens,
   model cap 200k)
2. gpt-4o, same, cap 128k
3. claude-3-5-haiku, 413 `ContextWindowExceeded`, cap 200k
4. deepseek-chat, 500 `InternalError`
5. nvidia llama-70b (key fp a831), 402 `BillingExhausted`
6. nvidia llama-70b (key fp b22c), 402 `BillingExhausted`
7. Fatal, no more providers

## Root cause
Request body shipped a 210k-token prompt. The intelligence-mode resolver
selected `full` (tool planning + reasoning), which routed to T1
providers first. All T1 models share a ≤200k context cap; none could
serve. The waterfall correctly escalated to T2 (DeepSeek) and T3 (NVIDIA)
but both were exhausted in different ways.

## Remediation
1. Client-side: implement context pre-flight, if prompt >180k tokens,
   auto-switch to Gemini 2.5 Pro (1M context) before sending.
2. Top up or rotate the NVIDIA key pool. Use `rotate_key(provider=nvidia)`
   from this MCP server.
3. Consider enabling the context-compaction healer (see
   `docs/architecture/output-verification`).

What the crate contains

These are properties of the code that exists, not of a package you can pull:

• 15 tools, 4 resources, 3 prompts: covering routing, cache, memory, cost, control, completion.
• stdio and Streamable HTTP transports: stdio for desktop clients, HTTP for remote clients.
• OAuth 2.1 for HTTP: RS256 JWT verification, scope-based authorization, /.well-known/oauth-protected-resource metadata endpoint.
• Native Rust: no Python, no Node runtime in the hot path. Direct reuse of the existing gateway services rather than a duplicate implementation.

The server is distributed as the hosted endpoint only. There is no package, container image or standalone binary to install, and none is planned: a remote MCP server needs no local runtime, and the endpoint is the whole integration.

What is still open

• Three of the 15 tools still read from backends that are not finished, so a released build would answer those with an error rather than data.
• The hosted endpoint is deployed: https://api.xantly.com/mcp over Streamable HTTP, bearer token = your Xantly API key.
• More prompt templates (model-selection-advisor, quota-forecast) and subscribe support for real-time tool-call streams are designed, not built.

If there is a tool you want, particularly something specific to your workload, mail support@xantly.com with [mcp] in the subject.

What you can do today

Everything the MCP tools would read is already served over the REST API and rendered in the dashboard. The MCP server is a different way to reach that data, not a different set of data. Concretely:

1. Create an account. https://xantly.com/register. Self-serve, no sales call, no waitlist.

2. Mint an API key. https://xantly.com/dashboard/developer/keys, which opens the API keys tab of the Models and keys screen. Also self-serve.

3. Check the key without spending anything. GET https://api.xantly.com/v1/models needs your key and returns the live catalog. It runs no model, so it costs nothing. There is no free tier and no trial credit at Xantly, so this is the zero-cost way to confirm a key works before you top up.

   curl -s https://api.xantly.com/v1/models \
     -H "Authorization: Bearer $XANTLY_API_KEY" | head -c 400
   

   GET https://api.xantly.com/v1/health needs no key at all and confirms the gateway is up.

4. Then top up and call a model. Credits are prepaid, from $5 to $10,000, with a single 3% platform fee at top-up. Routed usage is billed at the upstream provider's exact per-token price; cache and memory hits are billed at a flat $0.25 per million tokens.

5. Read the same signals the MCP tools would return. Every response carries cost, chosen model, cache status and latency on its headers, and the dashboard has the cost breakdown, routing detail and cache statistics.

Links

• MCP specification: https://modelcontextprotocol.io/specification/2025-11-25
• Xantly API reference: https://docs.xantly.com/docs/
• Create an account: https://xantly.com/register
• Mint an API key: https://xantly.com/dashboard/developer/keys
• Pricing: https://xantly.com/pricing

The source lives in the Xantly engine repository, which is private, so the crate, its integration guide and its architecture note are not linkable from here. Earlier versions of this page linked to them anyway, along with a Discord invite at a URL that does not exist. Both are removed.

Questions? We read every reply. team@xantly.com