Skip to content

fix(metrics): Prometheus metrics output#3640

Open
m3diumrare wants to merge 1 commit into
sei-protocol:mainfrom
m3diumrare:prom-metrics-fix
Open

fix(metrics): Prometheus metrics output#3640
m3diumrare wants to merge 1 commit into
sei-protocol:mainfrom
m3diumrare:prom-metrics-fix

Conversation

@m3diumrare

@m3diumrare m3diumrare commented Jun 24, 2026

Copy link
Copy Markdown

After the sei-chain to sei_chain prefix change in #3554, it surfaced that otel labels also had invalid Prometheus label characters ('.')

Tested on local Sei node, with this patch the Prometheus remote write endpoint no longer drops any metrics from the node.

Example before:

curl -si -H "Accept: application/openmetrics-text;version=1.0.0;escaping=allow-utf-8" http://127.0.0.1:26660/metrics | grep 'telemetry.sdk.language'
target_info{"service.name"="unknown_service:seid","telemetry.sdk.language"="go","telemetry.sdk.name"="opentelemetry","telemetry.sdk.version"="1.39.0"} 1

after:

 curl -si -H "Accept: application/openmetrics-text;version=1.0.0;escaping=allow-utf-8" http://127.0.0.1:26660/metrics | grep 'telemetry.sdk.language'
target_info{service_name="unknown_service:seid",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.39.0"} 1

@cursor

cursor Bot commented Jun 24, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Observability-only change to metric name/label translation; no consensus, auth, or transaction logic affected.

Overview
Fixes Prometheus remote write dropping node metrics by configuring both OTel→Prometheus exporters to use UnderscoreEscapingWithSuffixes when translating OTel metric and label names.

SetupOtelPrometheus (sei-db/common/metrics/setup.go) and SetupOtelMetricsProvider (utils/metrics/metrics_util.go) now pass prometheus.WithTranslationStrategy(otlptranslator.UnderscoreEscapingWithSuffixes) (or the otel exporter equivalent). That rewrites invalid label characters—such as dots in OTel attributes—into Prometheus-safe names, which addresses fallout from the earlier sei-chainsei_chain namespace change.

Reviewed by Cursor Bugbot for commit b68220b. Bugbot is set up for automated code reviews on this repo. Configure here.

@m3diumrare

Copy link
Copy Markdown
Author

@philipsu522 this would be great to get into 6.6 release if possible

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant