|
| 1 | +--- |
| 2 | +title: OpenAI-Compatible Adapter |
| 3 | +id: openai-compatible-adapter |
| 4 | +description: "Use any OpenAI-compatible provider (DeepSeek, Moonshot/Kimi, Together, Fireworks, Cerebras, Qwen, Perplexity, local servers, and more) in TanStack AI with one generic adapter." |
| 5 | +keywords: |
| 6 | + - tanstack ai |
| 7 | + - openai compatible |
| 8 | + - deepseek |
| 9 | + - moonshot |
| 10 | + - kimi |
| 11 | + - together |
| 12 | + - fireworks |
| 13 | + - cerebras |
| 14 | + - qwen |
| 15 | + - perplexity |
| 16 | + - lm studio |
| 17 | + - vllm |
| 18 | + - adapter |
| 19 | +--- |
| 20 | + |
| 21 | +Many providers expose the OpenAI **Chat Completions** API (`/chat/completions`) — DeepSeek, Moonshot/Kimi, Together, Fireworks, Cerebras, Alibaba Qwen, Perplexity, NVIDIA NIM, and local servers like LM Studio, Ollama, and vLLM. Instead of a dedicated package per provider, TanStack AI ships one generic adapter: point it at any compatible `baseURL`, give it your models, and you get the same type-safe `chat()` experience as the first-class adapters. |
| 22 | + |
| 23 | +Use this when your provider speaks the OpenAI Chat Completions wire format but doesn't have its own `@tanstack/ai-*` package. If a dedicated adapter exists (OpenAI, Grok, Groq, OpenRouter), prefer it — those carry curated per-model metadata. |
| 24 | + |
| 25 | +## Installation |
| 26 | + |
| 27 | +The adapter ships inside `@tanstack/ai-openai` under the `/compatible` subpath — no extra install: |
| 28 | + |
| 29 | +```bash |
| 30 | +npm install @tanstack/ai-openai |
| 31 | +``` |
| 32 | + |
| 33 | +## Basic Usage |
| 34 | + |
| 35 | +Configure the provider once with `openaiCompatible({ baseURL, apiKey, models })`, then select a model per call. The returned model name is a type-safe union of the models you declared: |
| 36 | + |
| 37 | +```typescript |
| 38 | +import { chat } from "@tanstack/ai"; |
| 39 | +import { openaiCompatible } from "@tanstack/ai-openai/compatible"; |
| 40 | + |
| 41 | +const deepseek = openaiCompatible({ |
| 42 | + name: "deepseek", // optional label shown in devtools/errors (default: "openai-compatible") |
| 43 | + baseURL: "https://api.deepseek.com/v1", |
| 44 | + apiKey: process.env.DEEPSEEK_API_KEY!, |
| 45 | + models: ["deepseek-chat", "deepseek-reasoner"], |
| 46 | +}); |
| 47 | + |
| 48 | +const stream = chat({ |
| 49 | + adapter: deepseek("deepseek-chat"), |
| 50 | + messages: [{ role: "user", content: "Hello!" }], |
| 51 | +}); |
| 52 | +``` |
| 53 | + |
| 54 | +`deepseek("deepseek-reasoner")` is valid; `deepseek("gpt-4o")` is a type error — only declared models are accepted. |
| 55 | + |
| 56 | +## One-Shot Usage |
| 57 | + |
| 58 | +For a single model, skip the provider-factory and build the adapter inline with `openaiCompatibleText`: |
| 59 | + |
| 60 | +```typescript |
| 61 | +import { chat } from "@tanstack/ai"; |
| 62 | +import { openaiCompatibleText } from "@tanstack/ai-openai/compatible"; |
| 63 | + |
| 64 | +const stream = chat({ |
| 65 | + adapter: openaiCompatibleText("deepseek-chat", { |
| 66 | + baseURL: "https://api.deepseek.com/v1", |
| 67 | + apiKey: process.env.DEEPSEEK_API_KEY!, |
| 68 | + }), |
| 69 | + messages: [{ role: "user", content: "Hello!" }], |
| 70 | +}); |
| 71 | +``` |
| 72 | + |
| 73 | +## Declaring Models |
| 74 | + |
| 75 | +The `models` array accepts two forms, which you can mix: |
| 76 | + |
| 77 | +- **A bare string** — gets optimistic defaults: `text` + `image` input, with `streaming`, `function_calling`, and `structured_outputs` support. Good for mainstream chat models. |
| 78 | +- **A `createModel(name, capabilities)` definition** — declares precise per-model capabilities so the types match reality (e.g. a reasoning model with no image input). |
| 79 | + |
| 80 | +```typescript |
| 81 | +import { openaiCompatible } from "@tanstack/ai-openai/compatible"; |
| 82 | +import { createModel } from "@tanstack/ai"; |
| 83 | + |
| 84 | +const provider = openaiCompatible({ |
| 85 | + baseURL: "https://api.deepseek.com/v1", |
| 86 | + apiKey: process.env.DEEPSEEK_API_KEY!, |
| 87 | + models: [ |
| 88 | + "deepseek-chat", // string → optimistic defaults |
| 89 | + createModel("deepseek-reasoner", { |
| 90 | + input: ["text"], // text only |
| 91 | + features: ["reasoning", "structured_outputs"], |
| 92 | + }), |
| 93 | + ], |
| 94 | +}); |
| 95 | +``` |
| 96 | + |
| 97 | +> Capabilities are enforced at the type level. If a provider rejects a feature at runtime (e.g. tools on a model that doesn't support them), declare that model with `createModel` and omit the unsupported feature so the types stop you from calling it. |
| 98 | +
|
| 99 | +## Configuration |
| 100 | + |
| 101 | +`openaiCompatible` accepts every OpenAI SDK `ClientOptions` field besides `apiKey`/`baseURL` (which are required and promoted to the top level). The most useful are `defaultHeaders` and `defaultQuery`, for providers that need extra auth or routing parameters: |
| 102 | + |
| 103 | +```typescript |
| 104 | +const provider = openaiCompatible({ |
| 105 | + baseURL: "https://api.example.com/v1", |
| 106 | + apiKey: process.env.EXAMPLE_API_KEY!, |
| 107 | + models: ["some-model"], |
| 108 | + defaultHeaders: { "X-Custom-Header": "value" }, |
| 109 | + defaultQuery: { "api-version": "2026-01-01" }, |
| 110 | +}); |
| 111 | +``` |
| 112 | + |
| 113 | +## Chat Completions vs Responses |
| 114 | + |
| 115 | +By default the adapter targets the **Chat Completions** API (`/chat/completions`) — the surface virtually every compatible provider implements. For the rare provider that also implements OpenAI's **Responses** API (e.g. Azure OpenAI), opt in with `api: "responses"`: |
| 116 | + |
| 117 | +```typescript |
| 118 | +const provider = openaiCompatible({ |
| 119 | + baseURL: "https://my-resource.openai.azure.com/openai/v1", |
| 120 | + apiKey: process.env.AZURE_OPENAI_API_KEY!, |
| 121 | + models: ["gpt-4o"], |
| 122 | + api: "responses", // default is "chat-completions" |
| 123 | +}); |
| 124 | +``` |
| 125 | + |
| 126 | +## Supported Providers |
| 127 | + |
| 128 | +Any provider implementing the OpenAI Chat Completions API works. Common ones are below — **verify the `baseURL` and model ids against each provider's current docs**, since they change over time. Set the API key via the provider's own environment variable and pass it as `apiKey`. |
| 129 | + |
| 130 | +| Provider | `baseURL` | Example model | |
| 131 | +| --- | --- | --- | |
| 132 | +| DeepSeek | `https://api.deepseek.com/v1` | `deepseek-chat`, `deepseek-reasoner` | |
| 133 | +| Moonshot / Kimi | `https://api.moonshot.ai/v1` | `kimi-k2-0711-preview` | |
| 134 | +| Alibaba Qwen (DashScope, intl) | `https://dashscope-intl.aliyuncs.com/compatible-mode/v1` | `qwen-max`, `qwen-plus` | |
| 135 | +| Alibaba Qwen (DashScope, China) | `https://dashscope.aliyuncs.com/compatible-mode/v1` | `qwen-max` | |
| 136 | +| Together AI | `https://api.together.xyz/v1` | `meta-llama/Llama-3.3-70B-Instruct-Turbo` | |
| 137 | +| Fireworks AI | `https://api.fireworks.ai/inference/v1` | `accounts/fireworks/models/llama-v3p3-70b-instruct` | |
| 138 | +| Cerebras | `https://api.cerebras.ai/v1` | `llama-3.3-70b` | |
| 139 | +| DeepInfra | `https://api.deepinfra.com/v1/openai` | `meta-llama/Llama-3.3-70B-Instruct` | |
| 140 | +| Perplexity | `https://api.perplexity.ai` | `sonar`, `sonar-pro` | |
| 141 | +| Mistral | `https://api.mistral.ai/v1` | `mistral-large-latest` | |
| 142 | +| Nebius | `https://api.studio.nebius.ai/v1` | `meta-llama/Llama-3.3-70B-Instruct` | |
| 143 | +| Z.AI (GLM) | `https://api.z.ai/api/paas/v4` | `glm-4.6` | |
| 144 | +| Baseten | `https://inference.baseten.co/v1` | model-dependent | |
| 145 | +| Hugging Face (router) | `https://router.huggingface.co/v1` | `meta-llama/Llama-3.3-70B-Instruct` | |
| 146 | +| NVIDIA NIM | `https://integrate.api.nvidia.com/v1` | `meta/llama-3.3-70b-instruct` | |
| 147 | + |
| 148 | +## Local & Self-Hosted Servers |
| 149 | + |
| 150 | +Point the adapter at any local OpenAI-compatible server. The API key is usually a placeholder: |
| 151 | + |
| 152 | +```typescript |
| 153 | +import { openaiCompatible } from "@tanstack/ai-openai/compatible"; |
| 154 | + |
| 155 | +// LM Studio |
| 156 | +const lmstudio = openaiCompatible({ |
| 157 | + name: "lmstudio", |
| 158 | + baseURL: "http://localhost:1234/v1", |
| 159 | + apiKey: "lm-studio", |
| 160 | + models: ["local-model"], |
| 161 | +}); |
| 162 | + |
| 163 | +// vLLM |
| 164 | +const vllm = openaiCompatible({ |
| 165 | + name: "vllm", |
| 166 | + baseURL: "http://localhost:8000/v1", |
| 167 | + apiKey: "not-needed", |
| 168 | + models: ["meta-llama/Llama-3.3-70B-Instruct"], |
| 169 | +}); |
| 170 | + |
| 171 | +// Ollama's OpenAI-compatible endpoint |
| 172 | +const ollama = openaiCompatible({ |
| 173 | + name: "ollama", |
| 174 | + baseURL: "http://localhost:11434/v1", |
| 175 | + apiKey: "ollama", |
| 176 | + models: ["llama3.3"], |
| 177 | +}); |
| 178 | +``` |
| 179 | + |
| 180 | +> Ollama also has a dedicated adapter, [`@tanstack/ai-ollama`](./ollama), which understands its native API. Use `openaiCompatible` only if you specifically want Ollama's OpenAI-compatible surface. |
| 181 | +
|
| 182 | +## Azure OpenAI |
| 183 | + |
| 184 | +Azure uses a resource-scoped URL and a separate API-version. Use the `/openai/v1` endpoint with `defaultQuery` for the version and `defaultHeaders` for the `api-key` header: |
| 185 | + |
| 186 | +```typescript |
| 187 | +const azure = openaiCompatible({ |
| 188 | + name: "azure", |
| 189 | + baseURL: "https://YOUR_RESOURCE.openai.azure.com/openai/v1", |
| 190 | + apiKey: process.env.AZURE_OPENAI_API_KEY!, // also sent as Bearer; Azure accepts the api-key header below |
| 191 | + models: ["gpt-4o"], // your Azure deployment name |
| 192 | + defaultQuery: { "api-version": "2026-01-01-preview" }, |
| 193 | + defaultHeaders: { "api-key": process.env.AZURE_OPENAI_API_KEY! }, |
| 194 | +}); |
| 195 | +``` |
| 196 | + |
| 197 | +> Confirm the current `api-version` and endpoint shape in Azure's documentation — Azure's API surface evolves independently of OpenAI's. |
| 198 | +
|
| 199 | +## Example: With Tools |
| 200 | + |
| 201 | +Tools work exactly as they do with any other adapter, for models that support function calling: |
| 202 | + |
| 203 | +```typescript |
| 204 | +import { chat, toolDefinition } from "@tanstack/ai"; |
| 205 | +import { openaiCompatible } from "@tanstack/ai-openai/compatible"; |
| 206 | +import { z } from "zod"; |
| 207 | + |
| 208 | +const getWeatherDef = toolDefinition({ |
| 209 | + name: "get_weather", |
| 210 | + description: "Get the current weather", |
| 211 | + inputSchema: z.object({ location: z.string() }), |
| 212 | +}); |
| 213 | + |
| 214 | +const getWeather = getWeatherDef.server(async ({ location }) => { |
| 215 | + return { temperature: 72, conditions: "sunny" }; |
| 216 | +}); |
| 217 | + |
| 218 | +const deepseek = openaiCompatible({ |
| 219 | + baseURL: "https://api.deepseek.com/v1", |
| 220 | + apiKey: process.env.DEEPSEEK_API_KEY!, |
| 221 | + models: ["deepseek-chat"], |
| 222 | +}); |
| 223 | + |
| 224 | +const stream = chat({ |
| 225 | + adapter: deepseek("deepseek-chat"), |
| 226 | + messages: [{ role: "user", content: "What's the weather in Tokyo?" }], |
| 227 | + tools: [getWeather], |
| 228 | +}); |
| 229 | +``` |
| 230 | + |
| 231 | +## Next Steps |
| 232 | + |
| 233 | +- [OpenAI Adapter](./openai) - The first-class OpenAI adapter |
| 234 | +- [OpenRouter Adapter](./openrouter) - Access 300+ models through one gateway |
| 235 | +- [Tools Guide](../tools/tools) - Learn about tools |
| 236 | +- [Extending Adapters](../advanced/extend-adapter) - Add custom models to any adapter |
0 commit comments