Skip to content

feat: expose OTel semantic convention version in trace metadata#166

Merged
krisztianfekete merged 17 commits into
agentevals-dev:mainfrom
AaronProbha18:feat/otel-genai-attr-aliases
Jul 8, 2026
Merged

feat: expose OTel semantic convention version in trace metadata#166
krisztianfekete merged 17 commits into
agentevals-dev:mainfrom
AaronProbha18:feat/otel-genai-attr-aliases

Conversation

@AaronProbha18

Copy link
Copy Markdown
Contributor

Description

  • Capture and propagate OTel schema_url to expose semconv_version in conversion metadata.
  • Resolve GenAI attribute aliases at read time via the extractor (extract_*_from_attrs) without rewriting attributes during ingestion.
  • Introduce a flat attribute resolver for canonical-first alias lookup (currently for gen_ai.provider.namegen_ai.system).
  • Keep semantic convention version informational only; no schema-version-based branching or GenAI detection changes.
  • Add test coverage for alias resolution, semconv_version extraction, metadata propagation, and API serialization.

Related Issues

Closes #162

Notes

  • Scope intentionally limited to the extractor and conversion metadata; no run/storage model changes or migrations.
  • otel.schema_url is treated as generic OTel metadata and not as a GenAI detection signal.
  • Fix: add explicit UTF-8 encoding when opening files in loaders and runner.

@krisztianfekete krisztianfekete left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, it's going in the correct direction as per our discussion on the issue. Added some comments!

Comment thread src/agentevals/loader/auto.py
Comment thread src/agentevals/extraction.py Outdated
Comment thread src/agentevals/extraction.py Outdated
if match:
return match.group(1)

return "unknown"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since TraceConversionMetadata.semconv_version is typed, this can never be null, but "unknown". Let's return None and serialise to null.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. resolve_schema_version now degrades to None instead of "unknown" for missing or invalid schema URLs

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated. GENAI_ATTRIBUTE_ALIASES now deafults to None

Comment thread src/agentevals/loader/otlp.py Outdated
Comment thread src/agentevals/api/otlp_processing.py Outdated
Comment thread src/agentevals/api/models.py Outdated
Comment thread src/agentevals/trace_metrics.py Outdated
Comment thread tests/test_runner.py Outdated
Comment thread tests/test_api.py

@krisztianfekete krisztianfekete left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added one more comment, otherwise we are good to go!

Comment thread src/agentevals/trace_attrs.py Outdated
ADK_INVOCATION_ID = "gcp.vertex.agent.invocation_id"

# GenAI attribute alias table
GENAI_ATTRIBUTE_ALIASES: Optional[dict[str, list[str]]] = None

@krisztianfekete krisztianfekete Jul 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a regression. resolve_attr requires a dict, None raises AttributeError. A "no aliases" state is {} or a missing key, never None. Restore the dict please.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad, I misunderstood the comment there. I've restored the dict and applied the changes to schema_version in trace_metrics

Comment thread src/agentevals/trace_metrics.py Outdated
"model": None,
"response_model": None,
"provider": None,
"schema_version": "unknown",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all we need I guess.

Suggested change
"schema_version": "unknown",
"schema_version": None,

@krisztianfekete krisztianfekete left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@krisztianfekete krisztianfekete merged commit 77bd0bd into agentevals-dev:main Jul 8, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Version-aware OTel GenAI semconv normalization

2 participants