Chat Completions

Create completions with OpenAI-compatible request/response shapes plus optional Xantly orchestration controls.

• POST /v1/chat/completions
• HEAD /v1/chat/completions (returns 204 No Content; useful as a lightweight probe)
• Auth: Authorization: Bearer <token>

Recommended rollout: start with standard fields only (model, messages, temperature, max_tokens), then add orchestration controls incrementally.

Quick start example

curl -sS https://api.xantly.com/v1/chat/completions \
  -H "Authorization: Bearer $XANTLY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [
      {"role": "user", "content": "Explain vector databases in 2 bullets."}
    ]
  }'

Request body

Standard parameters (OpenAI-compatible)

ChatMessage object

Proprietary parameters (Xantly)

These are optional. If omitted, the gateway uses defaults and automatic policy.

routing_hints (soft preferences)

routing_override (harder overrides)

xantly orchestration block

Legacy request headers (still supported)

The gateway maps selected request headers into metadata for backward compatibility.

Choosing how the router picks

model accepts a routing alias instead of a model id. This is the normal way to use Xantly: you state what matters and the router picks per request, rather than you maintaining a model list that goes stale.

An objective reorders the models that can serve your request; it never lowers the bar for what the request needs. Ask for speed on a hard question and you get the fastest model that can still answer it, not a cheap one that cannot. That is why xantly/auto-quality may return a mid-priced model on an easy question: it was the best answer available for the work, and paying more would have bought nothing.

Every response tells you what happened: x-xantly-objective echoes the objective in force, and x-xantly-model-used names the model that answered.

Any xantly/* id that is not in this table is rejected with a 400, so a typo fails loudly instead of silently routing somewhere you did not intend.

Response body

Non-stream (200 OK)

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "created": 1741400000,
  "model": "bedrock/v3.2/us-east-1",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 31,
    "completion_tokens": 18,
    "total_tokens": 49
  },
  "xantly_metadata": {
    "request_id": "req_01abc...",
    "routing_decision": "Lane: Turbo (Source: BarpRouter)",
    "provider": "deepseek",
    "tier": "T2",
    "provider_tier": "T2",
    "latency_ms": 142,
    "requested_model": "auto",
    "decision_source": "BarpRouter",
    "task_family": "analysis",
    "cost_usd": 0.00032,
    "baseline_cost_usd": 0.00120,
    "savings_usd": 0.00088,
    "savings_pct": 73.3,
    "cost_attribution": "xantly",
    "estimated_cost_usd": 0.00032,
    "healer_report": null
  }
}

xantly_metadata fields

Notes

• choices[].logprobs is part of schema, but may be omitted in normalized API responses.
• For n > 1, if upstream returns fewer choices, the gateway may replicate the first choice to satisfy requested cardinality.

Stream (text/event-stream)

• All stream responses emit chat.completion.chunk events and terminate with data: [DONE].
• Non-voice streaming currently emits a compact sequence (role chunk + content chunk + [DONE]) instead of token-by-token chunking.
• stream_options.include_usage is forwarded to providers where supported; do not assume a terminal usage chunk is always present.

Response headers

Common observability headers

Every response also carries the standard OpenAI fields, so a client that reads response.model and response.usage needs no Xantly-specific code at all.

Cache-path headers

Usage / cost headers

These are included when usage is available:

• x-xantly-input-tokens
• x-xantly-output-tokens
• x-xantly-cost-usd

Timeouts

The gateway gives a request a budget for one attempt and a budget for the whole attempt chain. Both are derived from the shape of your request, because a tool call and a reasoning turn are not the same kind of wait.

Two ceilings apply to the scaled row. A buffered answer is capped at 90s per attempt and 95s for the chain, because the edge in front of the API closes a connection that has sent no bytes for 100 seconds. A streamed answer is not subject to that, since bytes are already flowing, so it runs up to the 15 minute hard cap. If you ask for a long answer, stream it.

Deciding for yourself

routing_override.timeout_ms sets the chain budget and raises the per-attempt ceiling with it, from 1ms to 900000ms (15 minutes). It raises as well as lowers: the gateway's own default is a starting point, not a limit you must live under.

curl -sS https://api.xantly.com/v1/chat/completions \
  -H "Authorization: Bearer $XANTLY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Prove it, showing every step."}],
    "max_tokens": 8192,
    "stream": true,
    "routing_override": {"timeout_ms": 600000}
  }'

An organization-level timeout, if an administrator has set one, still applies as a ceiling over whatever a request asks for.

What happens when an attempt runs out

A timeout is a failed attempt, not a failed request: the gateway substitutes another model and keeps going, until max_fallbacks or the chain budget is spent. Set allow_fallbacks: false to fail on the first attempt instead. x-xantly-escalation-count on the response tells you how many substitutions it took, and x-xantly-wasted-attempts how many produced nothing.

Errors

Error payloads use OpenAI-style shape:

{
  "error": {
    "message": "temperature (2.5) must be between 0 and 2",
    "type": "invalid_request_error",
    "code": "validation_error",
    "param": null
  }
}

Error headers

• x-error-id is returned on error responses for support correlation.
• retry-after is returned on eligible rate-limit responses.

Edge cases and implementation notes

1. Model validation is catalog-aware

   • model validation is strict when the model catalog is loaded.
   • During catalog cold-start/empty states, strict rejection may be relaxed.

2. top_logprobs validation

   • Server enforces top_logprobs <= 20.
   • Provider-specific logprobs behavior may vary.

3. Chain limit overrides are conditional

   • xantly.max_chain_steps and xantly.chain_timeout_secs apply only when workflow resolves to long_horizon_autonomous.

4. Streaming semantics

   • Non-voice stream mode is SSE-compatible but not guaranteed token-by-token.

5. Reserved fields

   • Some accepted proprietary fields are currently advisory/reserved for compatibility (force_provider, cache_ttl_secs, compress_context, enable_tool_reranking, and certain routing_hints fields).

Practical rollout checklist

1. Start with model: "auto" + standard fields.
2. Enable response_format for structured outputs.
3. Add one routing/orchestration control at a time.
4. Track x-xantly-* response headers in observability.
5. Add override fields only for test/debug traffic.

See also

• Benchmark Results, Every parameter on this page is individually validated. See the full 252-test scorecard including boundary and invalid-input cases.
• Streaming Responses, Complete guide to SSE format, chunk handling, and SDK examples.
• Rate Limits, RPM and TPM limits that apply to this endpoint.
• Billing & Credits, Prepaid credits, budget caps, and cost visibility via xantly_metadata.