# Use Xantly with DSPy

[DSPy](https://github.com/stanfordnlp/dspy) is Stanford's "programming-not-prompting" framework, you declare signatures and modules, DSPy optimizes the prompts. Its `dspy.LM` class accepts any OpenAI-compatible endpoint. Point it at Xantly and DSPy's optimizer runs + program execution both route through smart routing + cache + memory.

## Prerequisites

- Python 3.9+
- `pip install dspy`
- A Xantly API key, [create one](/dashboard/api-keys)

## Setup

```python
import dspy

lm = dspy.LM(
    "openai/xantly/auto-quality",  # protocol/model
    api_base="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)
dspy.configure(lm=lm)
```

That's the entire setup, now every DSPy module uses Xantly.

## Signature + module

```python
import dspy

class QA(dspy.Signature):
    """Answer the question concisely."""
    question = dspy.InputField()
    answer = dspy.OutputField()

qa = dspy.Predict(QA)
result = qa(question="What is DSPy?")
print(result.answer)
```

## Chain-of-thought

```python
cot = dspy.ChainOfThought(QA)
result = cot(question="What's 17 * 23?")
print(result.reasoning)
print(result.answer)
```

## ReAct

```python
def search(query: str) -> str:
    """Search the web."""
    return f"results for {query}"

agent = dspy.ReAct(QA, tools=[search])
result = agent(question="What is the capital of France?")
print(result.answer)
```

## Optimizer (MIPROv2, BootstrapFewShot)

DSPy's differentiator, it optimizes your prompts against a train set:

```python
from dspy.teleprompt import BootstrapFewShot

trainset = [
    dspy.Example(question="What is 2+2?", answer="4").with_inputs("question"),
    dspy.Example(question="What is 3*3?", answer="9").with_inputs("question"),
]

def accuracy(example, pred, trace=None):
    return pred.answer.strip() == example.answer

compiler = BootstrapFewShot(metric=accuracy, max_bootstrapped_demos=4)
optimized = compiler.compile(qa, trainset=trainset)
```

The compiler issues many LM calls during optimization. Xantly's semantic cache absorbs the repeated ones → 40-70% cost savings on big training runs.

## Per-module models

Use different tiers for different modules:

```python
fast = dspy.LM(
    "openai/xantly/auto-speed",
    api_base="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)
quality = dspy.LM(
    "openai/xantly/auto-quality",
    api_base="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)

classifier = dspy.Predict(MyClassifierSig, lm=fast)
reasoner = dspy.ChainOfThought(MyReasoningSig, lm=quality)
```

## Streaming

```python
for chunk in dspy.streamify(qa)(question="Stream an explanation of DSPy."):
    print(chunk, end="", flush=True)
```

## Model choice

| Model ID | When |
|---|---|
| `xantly/auto-quality` | Optimizer runs + production reasoning. |
| `xantly/auto-value` | High-volume predict modules. |
| `xantly/auto-speed` | Classifiers, short completions. |
| `anthropic/claude-sonnet-4.6` | ReAct + tool-heavy modules. |
| `openai/gpt-5.4` | Structured-signature adherence. |

## Verify

```python
import dspy

lm = dspy.LM(
    "openai/xantly/auto-speed",
    api_base="https://api.xantly.com/v1",
    api_key="xantly_sk_...",
)
dspy.configure(lm=lm)
print(dspy.Predict("input -> output")(input="say pong").output)
```

## What you get

- **Big savings on optimizer runs.** `BootstrapFewShot` / `MIPROv2` fire hundreds of LM calls during compilation. Xantly's semantic cache turns repeats into free hits. Common 40-70% cost reductions.
- **Waterfall fallback during long compilation.** A mid-optimization provider outage doesn't wreck your run.
- **Per-signature cost tracking.** See which module burned what in the dashboard.
- **Memory for ReAct.** Set `X-Xantly-Memory-Mode: recall` via `default_headers` on the `dspy.LM` to persist tool-use patterns across runs.
- **Tool-call translation.** Switch `dspy.LM` between providers and tool-using modules keep working.

## Gotchas

**The `openai/` prefix is required.** DSPy uses LiteLLM under the hood. Full model ID: `openai/xantly/auto-quality` (`openai` is the protocol, `xantly/auto-quality` is the model).

**`api_base` not `base_url`.** DSPy matches the LiteLLM kwarg name.

**`/v1` required in `api_base`.**

**Optimizer cost explosion.** If you set `max_bootstrapped_demos=16` with a quality model, expect thousands of calls. Use a `value` tier LM for compilation and a `quality` tier for the final compiled program.

**`dspy.settings` is thread-local.** If you multithread, each thread needs `dspy.configure(lm=...)`.

## Next steps

- [Guidance](/docs/use-with-guidance), Microsoft's alternative to DSPy.
- [LangChain (Python)](/docs/use-with-langchain-python), classic chaining framework.
- [Instructor](/docs/use-with-instructor), structured-output library.
- [Cost-Optimized Routing](/docs/cost-optimized-routing), critical for expensive optimizer runs.