Skip to content

Commit 98979f7

Browse files
tombeckenhamautofix-ci[bot]claudeAlemTuzlak
authored
feat: streaming structured output across openai/openrouter/grok/groq + summarize fix (TanStack#527)
* feat(ai): streaming structured output (chat outputSchema + stream:true) Adds an optional `structuredOutputStream` method to the `TextAdapter` interface plus the activity-layer wiring so `chat({ outputSchema, stream: true })` returns a typed `StructuredOutputStream<T>`. The stream yields raw JSON deltas via the existing TEXT_MESSAGE_* lifecycle and terminates with a CUSTOM `structured-output.complete` event whose `value` is `{ object, raw, reasoning? }`. Adapters that don't implement `structuredOutputStream` natively fall back to `fallbackStructuredOutputStream`, which wraps the non-streaming `structuredOutput()` call so consumers see a consistent lifecycle on every adapter. With tools, the activity layer runs the agent loop, drops its RUN_STARTED/RUN_FINISHED, and lets the structured stream bracket the run. `TextActivityResult` uses `[TStream] extends [true]` (not bare `TStream extends true`) so the default `boolean` value of `TStream` does *not* match the streaming branch. This fixes TanStack#526 where `chat({ outputSchema })` typed as a stream while the runtime returned a Promise. Native streaming structured output for each provider lands in follow-up commits via a centralised lift into @tanstack/openai-base. * feat(openai-base): centralised structuredOutputStream + isAbortError hook Adds `structuredOutputStream` to both `OpenAICompatibleChatCompletionsTextAdapter` and `OpenAICompatibleResponsesTextAdapter`. Chat Completions issues a single request with `response_format: json_schema` + `stream: true`; Responses uses `text.format: json_schema` + `stream: true`. Subclasses inherit the method β€” reasoning lifecycle flows through the existing `extractReasoning` hook (Chat Completions) or Responses-API event-type discrimination (Responses), and the final parsed JSON runs through the existing `transformStructuredOutput` hook. Subclass changes: - ai-groq: new `extractReasoning` override reading `delta.reasoning` / `delta.reasoning_content` so Groq reasoning models stream reasoning under the centralised path. (ai-groq's existing `processStreamChunks` override only fires on the chatStream path; the new structuredOutputStream independently captures usage from `chunk.x_groq?.usage` outside the `choices[0]` guard.) - ai-grok: new `extractReasoning` override for xAI's `reasoning_content` / `reasoning` convention. - ai-openrouter: new `isAbortError` override mapping `RequestAbortedError` from `@openrouter/sdk` to `RUN_ERROR { code: 'aborted' }`. Existing `extractReasoning` (`_reasoningText` on adapted chunks) and `transformStructuredOutput` (identity, preserves nulls) overrides apply to the new path unchanged. Net deletion: ~1k LOC of per-adapter structuredOutputStream implementations (landed in prior commit but never reached production) collapse into ~330 LOC in the chat-completions base + ~340 LOC in the responses base. * fix(openai-base): tighten structuredOutputStream conditionals for eslint Drop dead `hasEmittedTextMessageEnd` flag (only set, never read), unwrap unneeded `?.` on `chunk.choices[0]` (type already nullable), and remove `?? 0` fallbacks on SDK-typed numeric usage fields. * refactor: drop \`as unknown as\` from streaming structured-output paths Replace \`as unknown as StreamChunk\` casts in fallbackStructuredOutputStream and runStreamingStructuredOutputImpl with \`satisfies StreamChunk\` on EventType-enum-tagged event literals (the AG-UI types tag \`.type\` with \`EventType.*\` enum values, not string literals β€” so import \`EventType\` and use it). The custom-event narrow now uses the existing \`isStructuredOutputCompleteEvent\` type guard instead of an inline shape check + cast, which lets the inner \`value\` reference drop its \`as { object; raw; reasoning? }\` cast. In openai-base, the request-cleanup destructures now operate on the SDK's typed params directly (the OpenAI SDK types are well-formed enough to spread without coercing to \`Record<string, unknown>\` first). * fix(openai-base): align structuredOutputStream with TanStack#545 asChunk cleanup chatStream path. The structuredOutputStream lift on this branch was emitting those events without \`threadId\`; the new \`satisfies StreamChunk\` checks now catch it. Plumb \`threadId\` through structuredOutputStream's aguiState in both bases. Also drop the residual \`asChunk()\` wrappers in my structuredOutputStream yields and use \`type: EventType.X, ... } satisfies StreamChunk\` directly, matching TanStack#545's new convention. While we're here: the chat-completions \`processStreamChunks\` finalisation forwards the SDK's \`finish_reason\` directly into \`RUN_FINISHED.finishReason\`, but the SDK type still includes the legacy \`function_call\` value that AG-UI doesn't accept. TanStack#545's \`satisfies\` cleanup exposed the mismatch β€” collapse \`function_call\` to \`stop\` alongside the existing orphan \`tool_calls\` collapse. * ci: apply automated fixes * fix: align structured streaming with 543 openai-base + port to ai-openrouter After rebasing onto TanStack#543 (openai-base adopts the openai SDK directly and decouples ai-openrouter), wire the structured-output stream to the SDK client and re-implement it inside ai-openrouter: - openai-base: call `this.client.chat.completions.create` / `this.client.responses.create` directly instead of the removed `callChatCompletion*` / `callResponse*` abstract hooks; drop the defensive cast on `response.completed` now that `chunk.response` narrows via the SDK's `ResponseStreamEvent` union. - ai-openrouter: add `structuredOutputStream` mirroring the openai-base implementation, adapted to OpenRouter's camelCase wire shape (`responseFormat` / `streamOptions: { includeUsage: true }`) and SDK call surface (`orClient.chat.send({ chatRequest })`). Maps both DOM `AbortError` and SDK `RequestAbortedError` to `RUN_ERROR { code: 'aborted' }`. - ai-grok / ai-groq: switch to the canonical `OpenAI.Chat.Completions.ChatCompletionChunk` namespace form (ai-grok was importing a non-existent re-export from `@tanstack/openai-base`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(ai-openrouter): structuredOutputStream for Responses (beta) adapter Adds streaming structured output to `OpenRouterResponsesTextAdapter` for parity with the chat-completions variant and the openai-base Responses adapter. Single call to `beta.responses.send` with `text.format: { type: 'json_schema', strict: true }` + `stream: true`; events flow through the existing `normalizeStreamEvent` so the canonical shape matches `processStreamChunks` (including the Speakeasy UNKNOWN-with-`raw` fallback for events that fail strict per-variant validation upstream). Adaptations vs the openai-base port: camelCase usage shape (`inputTokens`/`outputTokens`/`totalTokens`) on `response.completed`, both `response.failed` and `response.incomplete` treated as terminal RUN_ERROR (matching `processStreamChunks`), SSE-level `error` event also surfaced as RUN_ERROR, and inline abort detection for `RequestAbortedError` / `AbortError` β†’ `code: 'aborted'`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: openai Chat Completions adapter + summarize streaming fix + example wiring Packages - ai-openai: add openaiChatCompletions / OpenAIChatCompletionsTextAdapter sibling to the existing Responses adapter. Thin subclass of OpenAIBaseChatCompletionsTextAdapter so callers can pick the older /v1/chat/completions wire format against the OpenAI SDK. - ai: ChatStreamSummarizeAdapter.summarizeStream now accumulates summary text and emits a terminal CUSTOM { name: 'generation:result' } event before passing RUN_FINISHED through. Fixes useSummarize never populating result in connection/server-fn streaming modes β€” GenerationClient only sets result on that specific CUSTOM event. ts-react-chat example - Structured Output menu: drop the misleading '(OpenRouter)' suffix from the sidebar entry; relabel the OpenAI option as 'OpenAI (Responses)'; add 'OpenAI (Chat Completions)' and 'OpenRouter (Responses beta)' so the page exposes all four wire-format combinations end-to-end. - Summarize page: add a model picker (gpt-4o-mini through gpt-5.2) wired through to the API route and both server-fns. Drop the hard-coded maxLength: 200 which on Responses-API reasoning models gets the whole max_output_tokens budget consumed by hidden reasoning; the style instruction in the prompt already drives length. Live-render TEXT_MESSAGE_CONTENT deltas via onChunk so streaming mode is visibly streaming rather than appearing identical to direct. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: apply automated fixes * refactor(ai-openrouter): drop casts and `satisfies StreamChunk` from structured-output streams Address PR TanStack#527 review feedback from @AlemTuzlak (3 comments on responses-text.ts, 1 on text.ts, 2 on activities/chat/index.ts): - ai-openrouter/responses-text.ts (`structuredOutputStream`): - Drop `(await this.orClient.beta.responses.send(...)) as AsyncIterable<StreamEvents>` β€” `EventStream<T>` already extends `AsyncIterable<T>`. - Drop `as ResponsesRequest['text']` on the inner `text` object β€” the SDK's request type accepts the literal shape directly. - Drop inline `(chunk as { ... }).delta` / `(chunk.response ?? {}) as {...}` casts. `NormalizedStreamEvent` already types `delta` and `response`; the existing `processStreamChunks` reads the same fields without casts. - Drop redundant `satisfies StreamChunk` (20Γ—). The `AsyncIterable<StreamChunk>` / `Generator<StreamChunk>` return types already validate every yield site via contextual typing. - ai-openrouter/text.ts (`structuredOutputStream`): - Drop `(await this.orClient.chat.send(...)) as AsyncIterable<ChatStreamChunk>`. - Drop redundant `satisfies StreamChunk` (17Γ—). - ai/activities/chat/index.ts: - Replace `{ chatOptions: TextOptions<any, any>; outputSchema: any }` parameter on `fallbackStructuredOutputStream` with `StructuredOutputOptions<Record<string, unknown>>` β€” the adapter-side type already exists. - Drop `(adapter as { provider?: string }).provider ?? adapter.name` in the structured-stream logger. `provider` is not a `TextAdapter` field; `adapter.name` is the canonical provider identifier. - Drop redundant `satisfies StreamChunk` / `satisfies StructuredOutputCompleteEvent` (8Γ—) in `fallbackStructuredOutputStream` and `runStreamingStructuredOutputImpl`. - ai/tests/chat-result-types.test.ts (new): - Add type-only regression test for `TextActivityResult`. Pins each `(outputSchema?, stream?)` combination so TanStack#526's streaming-structured-output branch can't silently regress to a Promise (or vice versa). * Removed satisfies StreamChunk * refactor(ai): drop `as unknown as` casts in chat() dispatch Use narrowed locals (`outputSchema`, `stream`) and explicit `outputSchema: undefined` overrides instead of double-casting `options` through `unknown`. The trailing `as TextActivityResult<TSchema, TStream>` stays β€” TS narrows value types from runtime guards but not generic type parameters, so the conditional return type can't be reduced from inside a branch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: apply automated fixes * docs(ai): document streaming structured output in skill + chat docs Cover chat({ outputSchema, stream: true }) in docs/chat/structured-outputs.md and the ai-core/structured-outputs skill: StructuredOutputStream<T> return type, isStructuredOutputCompleteEvent example, structured-output.complete event shape, per-adapter coverage (native vs. fallback), and a HIGH common mistake against parsing partial JSON deltas. Adds a cross-ref from the skill to ai-core/chat-experience. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: apply automated fixes * feat(ai): tag custom events in StructuredOutputStream + debug-log chunks in structuredOutputStream Public `StructuredOutputStream<T>` is now a discriminated union over three tagged CUSTOM variants: `structured-output.complete<T>`, `approval-requested`, and `tool-input-available`. Each has a literal `name` and typed `value`, so `chunk.type === 'CUSTOM' && chunk.name === '<literal>'` narrows directly to the exact shape β€” no `isStructuredOutputCompleteEvent` helper or cast needed. The bare CustomEvent is excluded from the union (its `value: any` would collapse the narrow to `any`); user-emitted events via the `emitCustomEvent` API still flow at runtime as a documented residual gap. New exports from @tanstack/ai: `ApprovalRequestedEvent`, `ToolInputAvailableEvent`. The `isStructuredOutputCompleteEvent` helper is removed (this overload is new in this PR β€” no shipped consumers). Per-chunk `logger.provider(...)` debug logging added inside `structuredOutputStream` for the four affected adapters (openai-base chat-completions + responses, ai-openrouter text + responses-text), matching the existing pattern in `chatStream` for end-to-end introspection in debug mode. ai-openrouter uses `finishReason` (camelCase) consistent with the SDK and the sibling chatStream logger; openai-base uses `finish_reason` per the openai SDK shape. Docs (`docs/chat/structured-outputs.md`) and the AI-core `structured-outputs` SKILL.md updated to use the direct discriminated narrow. * chore: consolidate streaming-structured-output changesets into one Merge the 10 changesets covering this PR (streaming structured output across chat/grok/groq/openai/openai-base/openrouter, the openrouter decoupling + narrowing, and the summarize subsystem unification) into a single `.changeset/streaming-structured-output.md` with the union of version bumps. The body retains every meaningful section from the originals (core, openai-base, provider adapters, openrouter decoupling, summarize) and adds the tagged-CustomEvent type design from the previous commit. * chore: scaffold .agent/self-learning pile with build-before-examples lesson Initial scaffold of `.agent/self-learning/` for the self-improve plugin (INDEX.md, config.yml, curation-state.yml, coupling.json, .gitignore, `lessons/promoted/`). Captures the first repo-scoped lesson: `2026-05-14-build-before-running-examples.md` β€” run `pnpm -w run build:all` before starting any example dev server so the workspace packages have `dist/` outputs vite can resolve. * ci: apply automated fixes * docs: streaming structured output with tools + OpenAI Chat Completions adapter docs/chat/structured-outputs.md Add "Streaming with tools that may pause" subsection covering the approval-requested / tool-input-available tagged variants the agent loop can emit before structured-output.complete. Code example shows the narrowing pattern for all three CUSTOM variants. Cross-links the Tool Approval Flow and Client Tools pages. docs/adapters/openai.md Add "Chat Completions API" section after Basic Usage covering the new openaiChatCompletions / createOpenaiChatCompletions factories β€” when to pick Chat Completions vs. Responses (reasoning-summary streaming, wire-format compatibility), code example, and a link to the Structured Outputs page for the streaming case. API Reference at the bottom now includes both factories. * docs(chat/structured-outputs): lead with client+server flow, demote manual iteration to advanced The previous streaming section opened with \`for await (const chunk of stream)\` β€” that's the advanced/server-side-only path. The typical use case is a UI streaming JSON deltas through SSE from a server endpoint, and the docs should lead with it. - New "Server endpoint" subsection: \`chat({outputSchema, stream: true})\` + \`toServerSentEventsResponse(stream)\`. One short example, no ceremony. - New "Client with useChat" subsection: \`useChat\` + \`fetchServerSentEvents\` + \`onChunk\`, with \`parsePartialJSON\` driving progressive UI. Shows where the validated object lives (the terminal \`structured-output.complete\` event, typed as \`T\` via the schema). Notes Vue/Solid/Svelte share the shape. - "What the stream contains" + "Adapter coverage" tables retained verbatim. - Old standalone \`for await\` example moved to a new "Advanced: iterating the stream directly" subsection at the end, framed as the path for Node scripts, CLIs, server-only flows, and tests. - "Streaming with tools that may pause" reframed to use the \`onChunk\` signature (matching the new primary path); a note points back to the advanced section for callers iterating the stream directly. * feat(ai-react): useChat managed partial/final for structured-output streaming Pass the same schema you give chat() on the server to useChat() on the client, and the hook tracks the progressive object and the validated terminal payload for you β€” no external useState, no onChunk ceremony, no parsePartialJSON calls in user code. API: const { sendMessage, isLoading, partial, final } = useChat({ connection: fetchServerSentEvents("/api/extract"), outputSchema: PersonSchema, }) // partial: DeepPartial<Person> β€” updates per TEXT_MESSAGE_CONTENT delta // final: Person | null β€” snaps on structured-output.complete Implementation: - New generic param TSchema extends SchemaInput | undefined = undefined on UseChatOptions / UseChatReturn / useChat. - UseChatReturn is conditional on TSchema: when supplied, adds typed partial/final; when undefined (default), return is unchanged. Inferred automatically from outputSchema option. - Internal onChunk handler tracks raw JSON buffer via ref, runs parsePartialJSON on each TEXT_MESSAGE_CONTENT delta, snaps final on the terminal CUSTOM structured-output.complete event, resets all three on RUN_STARTED. User's own onChunk callback still fires after internal processing β€” both compose. - DeepPartial<T> exported for handlers that need to annotate. The schema is used purely for client-side type inference; server-side validation still runs against the schema passed to chat({ outputSchema }) on the server route. Works identically for non-streaming endpoints β€” for those, partial stays {} and final populates when the single terminal event arrives. Type-level tests (tests/use-chat-types.test.ts) pin both branches of the discriminated return type β€” useChat() without outputSchema rejects access to partial/final via @ts-expect-error, useChat() with outputSchema asserts typed DeepPartial<Person> / Person | null. * ci: apply automated fixes * feat(ai-vue, ai-solid, ai-svelte): mirror useChat outputSchema/partial/final Apply the same schema-driven structured-output API that landed in @tanstack/ai-react to the other three framework hooks. Same options shape (`outputSchema?: TSchema`), same discriminated return type, identical runtime behavior β€” only the reactivity primitive differs per framework. Reactivity primitives: Vue β€” `Readonly<ShallowRef<DeepPartial<T>>>` / `Readonly<ShallowRef<T | null>>` Solid β€” `Accessor<DeepPartial<T>>` / `Accessor<T | null>` Svelte β€” `readonly partial: DeepPartial<T>` / `readonly final: T | null` (rune-backed getters) Each hook is now generic on `TSchema extends SchemaInput | undefined`, inferred from the `outputSchema` option. When omitted (default), the return type is byte-identical to before; when supplied, `partial`/`final` are added via a conditional `UseChatReturn<TTools, TSchema>` / `CreateChatReturn<TTools, TSchema>`. The internal onChunk handler is the same in all four β€” RUN_STARTED resets, TEXT_MESSAGE_CONTENT accumulates + parses, CUSTOM structured-output.complete snaps final. User onChunk is still invoked after the internal pass. DeepPartial<T> is exported from each framework package. Type-level tests in each package pin both branches of the discriminated return type, mirroring the React variant β€” pure types, no renderer required. Existing test suites pass on all three packages: ai-vue: 93 tests pass ai-solid: 103 tests pass ai-svelte: 56 tests pass * docs: structured-outputs cross-framework + rendering reasoning/tool-calls - structured-outputs.md "Client with useChat" section: add a "Rendering reasoning and tool calls" subsection explaining that those land on messages[…].parts (ThinkingPart, ToolCallPart, ToolResultPart) just like normal chat β€” no separate hook fields. Includes a render snippet showing how to hide the raw-JSON TextPart and let the structured view (partial/final) replace it. - Note that useChat (React/Vue/Solid) and createChat (Svelte) all accept the same outputSchema option with the same semantics β€” only the reactivity primitive differs. - Changeset: bump @tanstack/ai-vue, @tanstack/ai-solid, @tanstack/ai-svelte to minor alongside @tanstack/ai-react. Replaced the "React" section with a unified "Framework hooks" section covering all four packages and documenting the per-framework reactivity types. * ci: apply automated fixes * docs(structured-outputs): fix 'with tools that may pause' to use real APIs The previous draft of the streaming-with-tools-that-may-pause subsection invented showApprovalPrompt / runClientTool / resumeWithToolResult helpers. The actual flow uses the standard chat APIs, identical to a non-structured chat: - Server tools with needsApproval:true land on messages[...].parts as ToolCallPart with state === 'approval-requested'. Render approval UI from messages, respond via addToolApprovalResponse({ id, approved }) from the hook return (see docs/tools/tool-approval). - Client tools with execute() set run automatically via the ChatClient's onToolCall handler (chat-client.ts:198-233). For manual handling, use addToolResult({ toolCallId, tool, output, state }) β€” see docs/tools/client-tools. Replaced the made-up code with a real example showing an approval- gated tool inside a structured-output run, using addToolApprovalResponse and rendering the prompt from messages.parts. The structured stream layers on top of standard chat β€” no special pause-handling logic. * test: cover useChat({outputSchema}) runtime + runStreamingStructuredOutput orchestrator Two runtime test files closing the highest-value gaps in the PR's test coverage: packages/typescript/ai-react/tests/use-chat-structured-output.test.ts (4 tests) - partial updates progressively from TEXT_MESSAGE_CONTENT deltas, final snaps on the terminal CUSTOM structured-output.complete event - state resets between runs via the stateful mock adapter (RUN_STARTED clears partial/final before the second run's deltas land) - user-supplied onChunk callback fires after internal tracking, with full visibility of the same chunks - useChat() without outputSchema doesn't track structured state β€” the internal handler's outputSchema-gate is a no-op packages/typescript/ai/tests/chat-structured-output-stream.test.ts (6 tests) - native adapter.structuredOutputStream path: validated structured- output.complete event forwarded with parsed object, schema validation failure β†’ RUN_ERROR { code: 'schema-validation' } and NO complete event is emitted, reasoning carries through validation onto the terminal event, TEXT_MESSAGE_CONTENT deltas pass through - fallbackStructuredOutputStream path (adapter lacks native streaming): synthesizes RUN_STARTED β†’ TEXT_MESSAGE_* β†’ structured-output.complete β†’ RUN_FINISHED around the non-streaming structuredOutput call; schema validation failure on the fallback path also emits RUN_ERROR Together: ai package 769 tests, ai-react 110, ai-vue 93, ai-solid 103, ai-svelte 56 β€” all green. * ci: apply automated fixes * test(openai-base): cover structuredOutputStream on both base adapters The server-side adapter implementations of structuredOutputStream (shared by ai-openai, ai-grok, ai-groq via inheritance) had zero unit coverage β€” only the e2e suite exercised them. Two new focused test files close that gap by stubbing the openai SDK client and verifying the AG-UI lifecycle, request shape, error paths, and per-chunk debug logging. tests/chat-completions-structured-output-stream.test.ts (6 tests) - happy path: RUN_STARTED β†’ TEXT_MESSAGE_* β†’ CUSTOM structured-output.complete (typed object + raw JSON) β†’ RUN_FINISHED - request shape: stream: true + response_format: { type: 'json_schema', json_schema: { strict: true } }; tools are stripped - delta accumulation across multiple chunks produces exactly one structured-output.complete with the fully-parsed object - empty content β†’ RUN_ERROR { code: 'empty-response' }, no structured-output.complete is emitted - malformed JSON β†’ RUN_ERROR { code: 'parse-error' } - per-chunk logger.provider is called once per SDK chunk (verified via a spy logger threaded through resolveDebugOption) tests/responses-structured-output-stream.test.ts (7 tests) - same matrix against the Responses API event shape (response.created / response.output_text.delta / response.completed) - request shape: stream: true + text.format: { type: 'json_schema', strict: true }; tools stripped - usage promoted from response.completed onto RUN_FINISHED - empty content / parse-error β†’ RUN_ERROR with the correct code - response.refusal.delta β†’ RUN_ERROR { code: 'refusal' } (Responses- only failure surface) - per-chunk logger.provider invocation Stub adapters extend the base directly and pass a fake OpenAI client whose chat.completions.create / responses.create routes into a per-test mock β€” same pattern as the existing chat-completions-text.test.ts and responses-text.test.ts suites. openai-base test count: 70 β†’ 83 (all passing). Types + lint clean. * ci: apply automated fixes * fix(ci): list @standard-schema/spec as devDep on framework packages Two failures from the previous push, both stemming from the type-test files I added: knip flagged @standard-schema/spec as an unlisted import across ai-react, ai-vue, ai-solid, and ai-svelte test files, and @tanstack/ai-vue:test:types failed because β€” unlike the other three β€” its tsconfig included tests/, so tsc strictly resolved the import (which isn't a direct dep, only transitively via @tanstack/ai). Fixes: - Add \`@standard-schema/spec: ^1.1.0\` to devDependencies on all four framework packages. The import is purely for type-level construction in the type tests (StandardJSONSchemaV1<Person, Person> β€” a phantom branded type that simulates what a Zod schema's inferred type would look like). devDep is the right scope. - Align ai-vue's tsconfig with ai-react/ai-solid/ai-svelte by dropping tests/ from the tsc include block. Tests are still type-checked by vitest at runtime; tsc now only checks src/. Verified locally: pnpm test:knip, pnpm test:sherif, and test:types on all four framework packages pass. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Alem Tuzlak <t.zlak@hotmail.com>
1 parent 02527c2 commit 98979f7

69 files changed

Lines changed: 5647 additions & 426 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
fallback-counts.json
2+
*.bak
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# Lessons Index
2+
3+
Read this index every turn. Each entry below is a routing condition.
4+
If a `Use when ...` condition matches the current task, read the full lesson file.
5+
6+
<!-- LESSONS:START -->
7+
<!-- Auto-managed by self-improve plugin. Manual edits preserved between markers. -->
8+
9+
- [build-before-running-examples](lessons/2026-05-14-build-before-running-examples.md) β€” Use when starting any tanstack/ai example dev server β€” build workspace packages first
10+
<!-- LESSONS:END -->
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Self-improve plugin behavior knobs. Edit and commit per repo.
2+
correction_detection:
3+
enabled: true
4+
regex_strictness: loose # loose | strict
5+
coupling_detection:
6+
enabled: true
7+
regex_strictness: loose
8+
enforcement:
9+
pre_push_block: true # false = warn only, do not block push
10+
curation:
11+
default_interval_days: 30
12+
promotion:
13+
auto_suggest_global: true
14+
skill_improve_threshold: 3
15+
skills_repo: ~/.claude/skills
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"$schema": "./coupling.schema.json",
3+
"couplings": []
4+
}
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
last_curated: 2026-05-14
2+
next_nag: 2026-06-13
3+
default_interval_days: 30
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
name: build-before-running-examples
3+
description: Use when starting any tanstack/ai example dev server β€” build workspace packages first
4+
tags: [monorepo, examples, dev-workflow, build]
5+
scope: repo
6+
source:
7+
type: auto-captured
8+
created: 2026-05-14T13:05:00Z
9+
related_skill: null
10+
related: []
11+
---
12+
13+
# Build Workspace Packages Before Running Examples
14+
15+
**Rule:** Run `pnpm -w run build:all` from the repo root before starting any example dev server (`examples/ts-react-chat`, `ts-solid-chat`, `ts-vue-chat`, `ts-svelte-chat`, `vanilla-chat`, `php-slim`, `python-fastapi`, `ts-group-chat`).
16+
17+
**Why:** "this was a mistake by you, you should always build packages inside of this repo before you run the examples" β€” examples import workspace packages (`@tanstack/ai`, `@tanstack/react-ai-devtools`, `@tanstack/ai-devtools-core`, etc.) via `workspace:*` and resolve through each package's `exports` field pointing at `dist/`. If `dist/` is missing for any package β€” including transitive ones β€” vite's dep-scan fails and SSR returns a 500. Fixing the first missing package one at a time wastes round-trips: I tried `pnpm --filter @tanstack/react-ai-devtools build`, hit a missing `@tanstack/ai-devtools-core`, etc. The cure is one command up front.
18+
19+
**How to apply:** Before any `pnpm --filter "<example-name>" dev` (or running an example via its own directory), run `pnpm -w run build:all` from the worktree root. Nx caches the build so re-runs are cheap. Skip only if the user has just explicitly said the workspace is freshly built.

β€Ž.agent/self-learning/lessons/promoted/.gitkeepβ€Ž

Whitespace-only changes.

β€Ž.changeset/decouple-openrouter-collapse-openai-base.mdβ€Ž

Lines changed: 0 additions & 35 deletions
This file was deleted.

β€Ž.changeset/openrouter-narrow-stream-chunk-types.mdβ€Ž

Lines changed: 0 additions & 5 deletions
This file was deleted.
Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
---
2+
'@tanstack/ai': minor
3+
'@tanstack/openai-base': minor
4+
'@tanstack/ai-openai': minor
5+
'@tanstack/ai-grok': minor
6+
'@tanstack/ai-groq': minor
7+
'@tanstack/ai-openrouter': minor
8+
'@tanstack/ai-react': minor
9+
'@tanstack/ai-vue': minor
10+
'@tanstack/ai-solid': minor
11+
'@tanstack/ai-svelte': minor
12+
'@tanstack/ai-anthropic': patch
13+
'@tanstack/ai-gemini': patch
14+
'@tanstack/ai-ollama': patch
15+
---
16+
17+
Streaming structured output across the OpenAI-compatible providers, an OpenAI Chat Completions sibling adapter, a summarize-subsystem unification, and the decoupling of `@tanstack/ai-openrouter` from the shared OpenAI base.
18+
19+
## Core β€” `@tanstack/ai`
20+
21+
- New `chat({ outputSchema, stream: true })` overload returning `StructuredOutputStream<InferSchemaType<TSchema>>`. The stream yields raw JSON deltas via `TEXT_MESSAGE_CONTENT` plus a terminal `CUSTOM` `structured-output.complete` event whose `value.object` is typed against the caller's schema with no helper or cast required.
22+
- `StructuredOutputStream<T>` is a discriminated union over three tagged `CUSTOM` variants β€” `structured-output.complete<T>`, `approval-requested`, and `tool-input-available` (new `ApprovalRequestedEvent` / `ToolInputAvailableEvent` interfaces exported from `@tanstack/ai`). Narrowing on `chunk.type === 'CUSTOM' && chunk.name === '<literal>'` resolves `chunk.value` to the exact shape per variant. The bare `CustomEvent` (with `value: any`) is deliberately excluded to keep the narrow from collapsing to `any`; user-emitted events via the `emitCustomEvent` context API still flow at runtime and are documented as a small residual gap.
23+
- Activity-layer hardening: always-finalise after the stream loop (no silent hangs on missing `finishReason`), typed `RUN_ERROR` on empty content, mid-stream provider errors terminate cleanly, schema-validation failures carry `runId / model / timestamp`.
24+
- `fallbackStructuredOutputStream` in the activity layer is the single source of truth for adapters that don't implement `structuredOutputStream` natively; `BaseTextAdapter` no longer ships a default.
25+
- `ChatStreamSummarizeAdapter.summarizeStream` accumulates summary text and emits a terminal `CUSTOM` `generation:result` event before the final `RUN_FINISHED`. Fixes `useSummarize` never populating `result` over streaming connections (the client only sets `result` on that specific CUSTOM event).
26+
- `SummarizationOptions` is now generic in `TProviderOptions` and `modelOptions` is plumbed through end-to-end (previously silently dropped by `runSummarize` / `runStreamingSummarize`).
27+
28+
## Framework hooks β€” `@tanstack/ai-react`, `@tanstack/ai-vue`, `@tanstack/ai-solid`, `@tanstack/ai-svelte`
29+
30+
`useChat` (React/Vue/Solid) and `createChat` (Svelte) now accept an `outputSchema` option mirroring `chat({ outputSchema })` on the server. When supplied, the hook's return adds two managed reactive fields:
31+
32+
- `partial` β€” the live progressive object, typed `DeepPartial<InferSchemaType<typeof outputSchema>>`. Updated from `TEXT_MESSAGE_CONTENT` deltas via `parsePartialJSON`. Resets on every new run.
33+
- `final` β€” the validated terminal payload from the `structured-output.complete` event, typed `InferSchemaType<typeof outputSchema> | null`. `null` until the run completes.
34+
35+
Both fields are typed against the schema with no helper or cast β€” each hook is generic on `TSchema` and conditionally adds the fields to the return type. Without `outputSchema`, the return type is unchanged. Works the same for streaming and non-streaming endpoints β€” for non-streaming, `partial` stays `{}` and `final` snaps when the single terminal event arrives. Reasoning text and tool calls aren't surfaced as separate hook fields β€” they're already on `messages[…].parts` (as `ThinkingPart`, `ToolCallPart`, `ToolResultPart`), same as a normal chat. When `outputSchema` is set, the assistant's `TextPart` contains the raw JSON the model produced; filter `text` parts out of your message renderer and let the structured view (driven by `partial` / `final`) replace it.
36+
37+
Reactivity primitive per framework:
38+
39+
| Framework | `partial` type | `final` type |
40+
| ------------------------------ | ------------------------------------------------------- | ------------------------------------------------ |
41+
| React (`@tanstack/ai-react`) | `DeepPartial<T>` (plain state) | `T \| null` (plain state) |
42+
| Vue (`@tanstack/ai-vue`) | `Readonly<ShallowRef<DeepPartial<T>>>` | `Readonly<ShallowRef<T \| null>>` |
43+
| Solid (`@tanstack/ai-solid`) | `Accessor<DeepPartial<T>>` | `Accessor<T \| null>` |
44+
| Svelte (`@tanstack/ai-svelte`) | `readonly partial: DeepPartial<T>` (rune-backed getter) | `readonly final: T \| null` (rune-backed getter) |
45+
46+
`DeepPartial<T>` is exported from each framework package for callers who want to annotate handlers explicitly.
47+
48+
## Base β€” `@tanstack/openai-base`
49+
50+
- Package renamed from `@tanstack/ai-openai-compatible` (which remains published for pinned lockfiles but receives no further updates). Imports change:
51+
52+
```diff
53+
- import { OpenAICompatibleChatCompletionsTextAdapter } from '@tanstack/ai-openai-compatible'
54+
+ import { OpenAIBaseChatCompletionsTextAdapter } from '@tanstack/openai-base'
55+
- import { OpenAICompatibleResponsesTextAdapter } from '@tanstack/ai-openai-compatible'
56+
+ import { OpenAIBaseResponsesTextAdapter } from '@tanstack/openai-base'
57+
```
58+
59+
- Centralised `structuredOutputStream` on both bases. Chat Completions uses `response_format: { type: 'json_schema', strict: true }` + `stream: true`; Responses uses `text.format: { type: 'json_schema', strict: true }` + `stream: true`. Subclasses (`ai-openai`, `ai-grok`, `ai-groq`) inherit it; OpenRouter implements its own (see below).
60+
- Base now adopts the `openai` SDK directly and imports types from `openai/resources/...`. The previously-vendored ~720 LOC of wire-format types (`ChatCompletion`, `ResponseStreamEvent`, etc.) is removed; consumers that imported wire types from the package should import them from the openai SDK instead. The abstract `callChatCompletion*` / `callResponse*` hooks are gone β€” the base constructor now takes a pre-built `OpenAI` client (`new OpenAIBaseChatCompletionsTextAdapter(model, name, openaiClient)`) and calls `client.chat.completions.create` / `client.responses.create` itself.
61+
- New protected `isAbortError(error)` hook duck-types abort detection so `RUN_ERROR { code: 'aborted' }` is emitted consistently across SDK error types β€” subclasses with proprietary error classes (e.g. `@openrouter/sdk`'s `RequestAbortedError`) override.
62+
- Per-chunk `logger.provider(...)` debug logging now fires inside `structuredOutputStream` loops, matching the existing pattern in `chatStream` for end-to-end introspection in debug mode.
63+
64+
The other extension hooks (`extractReasoning`, `extractTextFromResponse`, `processStreamChunks`, `makeStructuredOutputCompatible`, `transformStructuredOutput`, `mapOptionsToRequest`, `convertMessage`) remain. Groq's `processStreamChunks` and `makeStructuredOutputCompatible` overrides (for `x_groq.usage` promotion and Groq's structured-output schema quirks) are unchanged.
65+
66+
## Provider adapters
67+
68+
| Adapter | API | Reasoning surface |
69+
| ---------------------------------------------------------- | ---------------- | ---------------------------------------------------------------------------------------------------------------- |
70+
| `@tanstack/ai-openai` `openaiText` | Responses | `response.reasoning_text.delta` + `response.reasoning_summary_text.delta` (requires `reasoning.summary: 'auto'`) |
71+
| `@tanstack/ai-openai` `openaiChatCompletions` (new) | Chat Completions | reasoning emitted silently β€” Chat Completions has no `reasoning.summary` opt-in |
72+
| `@tanstack/ai-grok` `grokText` | Chat Completions | `delta.reasoning_content` (DeepSeek convention; not typed by OpenAI SDK) |
73+
| `@tanstack/ai-groq` `groqText` | Chat Completions | `delta.reasoning` (requires `reasoning_format: 'parsed'`; not typed by groq-sdk) |
74+
| `@tanstack/ai-openrouter` `openRouterText` | Chat Completions | `delta.reasoningDetails` (camelCase) |
75+
| `@tanstack/ai-openrouter` `openRouterResponsesText` (beta) | Responses (beta) | `response.reasoning_text.delta` + `response.reasoning_summary_text.delta` via `normalizeStreamEvent` |
76+
77+
All six emit the contractual `REASONING_*` lifecycle (`REASONING_START` β†’ `REASONING_MESSAGE_START` β†’ `REASONING_MESSAGE_CONTENT` deltas β†’ `REASONING_MESSAGE_END` β†’ `REASONING_END`) and close it before `TEXT_MESSAGE_START`. Accumulated reasoning is also surfaced on `structured-output.complete.value.reasoning` for consumers that only subscribe to the terminal event. OpenRouter SDK's proprietary `RequestAbortedError` is mapped (alongside DOM `AbortError`) to `code: 'aborted'` in the two openrouter adapters.
78+
79+
`@tanstack/ai-openai` also exports a new `OpenAIChatCompletionsTextAdapter` / `openaiChatCompletions` / `createOpenaiChatCompletions` factory β€” a sibling to the existing Responses adapter for callers who want the older `/v1/chat/completions` wire format against the OpenAI SDK.
80+
81+
## Decouple `@tanstack/ai-openrouter` from the OpenAI base
82+
83+
OpenRouter ships its own SDK (`@openrouter/sdk`) with a camelCase shape, so inheriting from the OpenAI-shaped base forced a snake_case ↔ camelCase round-trip on every request and stream event. ai-openrouter now extends `BaseTextAdapter` directly and inlines its own stream processors (`OpenRouterTextAdapter` for chat-completions, `OpenRouterResponsesTextAdapter` for the Responses beta), reading OpenRouter's camelCase types natively. The `@tanstack/openai-base` and `openai` dependencies are removed from ai-openrouter; only `@openrouter/sdk`, `@tanstack/ai`, and `@tanstack/ai-utils` remain. The ~300 LOC of inbound/outbound shape converters (`toOpenRouterRequest`, `toChatCompletion`, `adaptOpenRouterStreamChunks`, `toSnakeResponseResult`, …) are gone. Internal: duck-typed `as { ... }` casts on stream chunks in `OpenRouterResponsesTextAdapter` are replaced with direct narrowing via the SDK's discriminated unions.
84+
85+
Public OpenRouter API is unchanged: `openRouterText`, `openRouterResponsesText`, `createOpenRouterText`, `createOpenRouterResponsesText`, the OpenRouter tool factories, provider routing surface (`provider`, `models`, `plugins`, `variant`, `transforms`), app attribution headers (`httpReferer`, `appTitle`), `:variant` model suffixing, `RequestAbortedError` propagation, and the OpenRouter-specific structured-output null-preservation all behave the same.
86+
87+
`ai-ollama` remains on `BaseTextAdapter` directly β€” its native API uses a different wire format from Chat Completions and was never on the shared base.
88+
89+
## Summarize subsystem
90+
91+
Anthropic, Gemini, Ollama, and OpenRouter previously each shipped a bespoke 200–300 LOC summarize adapter. They now construct a `ChatStreamSummarizeAdapter` (formerly `ChatStreamWrapperAdapter`, renamed and exported from `@tanstack/ai/activities`) wrapping their own text adapter, matching the existing OpenAI/Grok pattern. Removes ~600 LOC of duplicated logic across the six providers and ensures behavioural parity.
92+
93+
Bespoke `*SummarizeProviderOptions` interfaces (e.g. `OpenAISummarizeProviderOptions`, `AnthropicSummarizeProviderOptions`, `GeminiSummarizeProviderOptions`, `OllamaSummarizeProviderOptions`, `OpenRouterSummarizeProviderOptions`) are removed from the provider packages' public exports. Consumers who imported them should switch to inferring the type from the adapter (`InferTextProviderOptions<typeof adapter>`) or remove the explicit annotation (it'll be inferred from the adapter argument).
94+
95+
`SummarizeAdapter` interface methods are now generic in `TProviderOptions`. `summarize` and `summarizeStream` previously took `SummarizationOptions` (defaulted, so `modelOptions` was effectively `Record<string, any>` regardless of the adapter's typed shape). They now take `SummarizationOptions<TProviderOptions>`. Source-compatible for callers that didn't specify the generic; type-tighter for implementers and downstream consumers. `SummarizationOptions`, `SummarizeAdapter`, `BaseSummarizeAdapter`, and `ChatStreamSummarizeAdapter` previously had a mixed `Record<string, any>` / `Record<string, unknown>` / `object` set of defaults for `TProviderOptions`; they now uniformly default to `Record<string, unknown>`.

0 commit comments

Comments
Β (0)