fix(otel-exporter): export RAG_EMBEDDING spans using GenAI embedding semantics#17917
fix(otel-exporter): export RAG_EMBEDDING spans using GenAI embedding semantics#17917Akash504-ai wants to merge 2 commits into
Conversation
🦋 Changeset detectedLatest commit: ae7ac9a The changes in this PR will be included in the next version bump. This PR includes changesets to release 6 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
@Akash504-ai is attempting to deploy a commit to the Mastra Team on Vercel. A member of the Team first needs to authorize it. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
WalkthroughExtends ChangesRAG Embedding OTel Span Mapping
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint install timed out. The project may have too many dependencies for the sandbox. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
PR triageLinked issue check passed (#17914). Mastra uses CodeRabbit for automated code reviews. Please address all feedback from CodeRabbit by either making changes to your PR or leaving a comment explaining why you disagree with the feedback. Since CodeRabbit is an AI, it may occasionally provide incorrect feedback. PR complexity score
Applied label: Changed test gateChanged Test Gate is pending. The |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@observability/otel-exporter/src/gen-ai-semantics.test.ts`:
- Around line 93-112: The test `should extract model, provider, usage, and RAG
metadata for embedding spans` uses a hardcoded literal model ID
`text-embedding-3-small` in the createRagEmbeddingSpan function call. Replace
this hardcoded model ID with a registered placeholder token from
docs/src/plugins/remark-model-tokens/models.ts to comply with the repo's coding
guidelines for model names and IDs in tests.
In `@observability/otel-exporter/src/span-converter.test.ts`:
- Around line 75-95: Replace the literal model IDs with placeholder tokens from
docs/src/plugins/remark-model-tokens/models.ts according to coding guidelines.
In the test case spanning lines 75-95 (the convertSpan test for RAG_EMBEDDING
spans), replace the hardcoded model value 'text-embedding-3-small' in the
attributes object at line 86 and the corresponding assertion at line 94 with an
appropriate placeholder token from the models.ts file. Apply the same approach
to all other test cases in the file that use literal model IDs, specifically in
the sections at lines 242-260 and 364-399, replacing all hardcoded model names
with registered placeholder tokens and updating their corresponding assertions
consistently.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: e75d497c-56ee-4943-8e23-376de6abbbb0
📒 Files selected for processing (5)
.changeset/quiet-clouds-embed.mdobservability/otel-exporter/src/gen-ai-semantics.test.tsobservability/otel-exporter/src/gen-ai-semantics.tsobservability/otel-exporter/src/span-converter.test.tsobservability/otel-exporter/src/span-converter.ts
1 participant
Description
Fixes missing OpenTelemetry GenAI semantic mappings for
RAG_EMBEDDINGspans.RAG_EMBEDDINGspans already include model, provider, usage, dimensions, mode, and inputCount metadata, but the OTel exporter only mapped GenAI semantic attributes forMODEL_GENERATIONspans. As a result, embedding spans were exported without model/provider/usage information and were treated as generic internal spans by downstream observability tools.This PR:
RAG_EMBEDDINGtogen_ai.operation.name = "embeddings"gen_ai.request.modelgen_ai.provider.namegen_ai.usage.*metricsmode,dimensions,inputCount)RAG_EMBEDDINGspans asSpanKind.CLIENTThese changes allow downstream consumers such as Langfuse to receive the embedding metadata required for model attribution and cost tracking without custom span rewriting.
Related issue(s)
Fixes #17914
Type of change
Checklist
ELI5
This PR teaches the OpenTelemetry exporter how to recognize “embedding” spans (RAG embedding jobs) the same way it already does for text generation. As a result, monitoring tools now see which embedding model was used and how many tokens it used, instead of showing it as a vague “internal” event.
Summary
Mastra already attaches rich metadata to
RAG_EMBEDDINGspans (model/provider, token usage, and embedding-specific details), but the OTel exporter wasn’t translating that metadata into OpenTelemetry GenAI semantic attributes. That made downstream observability tools (e.g., Langfuse) treat embedding runs as generic internal spans, losing model attribution and usage/cost information.Solution
The OTel exporter was updated to add explicit
RAG_EMBEDDINGhandling:gen_ai.operation.name = "embeddings"forRAG_EMBEDDINGspansgen_ai.request.model(from the embedding model attribute)gen_ai.provider.name(normalized from the provider attribute)gen_ai.usage.*token usage fields (including token/cost-related metrics)mastra.rag_embedding.*(mode, dimensions, input count)RAG_EMBEDDINGspan kind fromINTERNALtoSpanKind.CLIENTso it’s categorized as an outbound/GenAI operationChanges
RAG_EMBEDDING→gen_ai.operation.name = "embeddings"mappingMODEL_GENERATIONandRAG_EMBEDDINGRAG_EMBEDDINGattributes branch to emit embedding-specificmastra.rag_embedding.*metadatagetSpanKind()soRAG_EMBEDDINGconverts toSpanKind.CLIENTmastra.rag_embedding.*attributesSpanKind.CLIENT@mastra/otel-exporterwith a documented patch fix for RAG embedding span OTel exports