# Use Xantly with the OpenAI Python SDK

The official `openai` Python library ships a `base_url` argument on every client. Point it at Xantly once and every downstream call, chat completions, streaming, tools, structured outputs, async, routes through smart routing, semantic cache, and memory. Zero code changes beyond the two kwargs.

## Prerequisites

- Python 3.8+
- `pip install openai` (version 1.0.0+)
- A Xantly API key, [create one](/dashboard/api-keys)

## Setup

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)
```

That's the entire migration. Every existing OpenAI call keeps working, `client.chat.completions.create(...)`, `client.embeddings.create(...)`, all of it.

Persist in env vars if you don't want to edit your code:

```bash
export OPENAI_BASE_URL=https://api.xantly.com/v1
export OPENAI_API_KEY=xantly_sk_...
```

Then `OpenAI()` with no arguments picks them up.

## Chat completions

```python
resp = client.chat.completions.create(
    model="xantly/auto-quality",
    messages=[{"role": "user", "content": "Write a bubble sort in Rust."}],
)
print(resp.choices[0].message.content)
```

## Streaming

Works exactly like OpenAI's SSE shape, Xantly normalizes every provider back into OpenAI events:

```python
stream = client.chat.completions.create(
    model="xantly/auto-quality",
    messages=[{"role": "user", "content": "Stream a haiku."}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    print(delta, end="", flush=True)
```

## Function / tool calling

```python
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

resp = client.chat.completions.create(
    model="anthropic/claude-sonnet-4.6",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=tools,
)
tool_call = resp.choices[0].message.tool_calls[0]
print(tool_call.function.name, tool_call.function.arguments)
```

Xantly translates tool-call format across providers, so you can swap `model=` from OpenAI to Claude to Groq and the same tool schema keeps working.

## Structured outputs (`response_format`)

```python
from pydantic import BaseModel

class ExtractedUser(BaseModel):
    name: str
    email: str
    role: str

resp = client.beta.chat.completions.parse(
    model="openai/gpt-5.4",
    messages=[{"role": "user", "content": "Extract: Jane Doe, jane@acme.com, CTO"}],
    response_format=ExtractedUser,
)
print(resp.choices[0].message.parsed)
```

## Async client

```python
import asyncio
from openai import AsyncOpenAI

aclient = AsyncOpenAI(
    base_url="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)

async def main():
    resp = await aclient.chat.completions.create(
        model="xantly/auto-value",
        messages=[{"role": "user", "content": "Ping"}],
    )
    print(resp.choices[0].message.content)

asyncio.run(main())
```

## Model choice

| Model ID | When |
|---|---|
| `xantly/auto-quality` | BaRP routes the T1 pool, best default for production. |
| `xantly/auto-value` | Balanced cost/quality, T2 default. |
| `xantly/auto-speed` | Fastest path, short completions, commit messages. |
| `openai/gpt-5.4` | Pin to OpenAI, Xantly passes through. |
| `anthropic/claude-sonnet-4.6` | Pin to Claude, response is translated back to OpenAI shape. |
| `groq/llama-3.3-70b` | Ultra-fast inference. |

## Verify

```python
resp = client.chat.completions.create(
    model="xantly/auto-speed",
    messages=[{"role": "user", "content": "say pong"}],
)
print(resp.choices[0].message.content)
# Check response headers for routing info:
print(resp._response_headers.get("x-xantly-model-id"))
print(resp._response_headers.get("x-xantly-cache-status"))
```

## What you get

- **Drop-in compatibility.** No new SDK, no new mental model.
- **Waterfall fallback.** Provider outages don't propagate up, Xantly retries on the next-best model in the same tier.
- **Semantic cache.** Repeat prompts hit L2 cache at near-zero cost.
- **Memory.** Opt in per request via `extra_headers={"X-Xantly-Memory-Mode": "recall"}`.
- **Cost transparency.** Every call logs model + cache status + USD cost to your dashboard.

## Gotchas

**`base_url` must include `/v1`.** The SDK appends paths like `/chat/completions` to it. Bare `https://api.xantly.com` won't work.

**The `openai` library v0.x is EOL.** You need v1.0.0+ (`pip install -U openai`). The old `openai.api_base` global was removed.

**Azure-style clients.** If you use `AzureOpenAI(...)`, switch back to `OpenAI(base_url=...)`. Xantly isn't Azure-compatible (yet).

**Retries.** The SDK retries on 5xx. Xantly's own waterfall already does this smarter, set `max_retries=0` on the client to avoid double retries.

## Next steps

- [OpenAI SDK (TypeScript)](/docs/use-with-openai-sdk-typescript), the Node.js equivalent.
- [Streaming Responses](/docs/streaming-responses), SSE format details.
- [Cost-Optimized Routing](/docs/cost-optimized-routing), how to keep bills down.
- [Bring Your Own Key](/docs/bring-your-own-key), route through your own OpenAI credits.