Add native Micrometer metrics backend (Prometheus + optional JMX legs)#18786
Open
gortiz wants to merge 1 commit into
Open
Add native Micrometer metrics backend (Prometheus + optional JMX legs)#18786gortiz wants to merge 1 commit into
gortiz wants to merge 1 commit into
Conversation
gortiz
force-pushed
the
feature/micrometer-metrics
branch
from
13d677f to
2b8838b
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #18786 +/- ##
============================================
- Coverage 64.78% 64.74% -0.04%
- Complexity 1309 1319 +10
============================================
Files 3380 3392 +12
Lines 209544 210949 +1405
Branches 32797 33119 +322
============================================
+ Hits 135746 136589 +843
- Misses 62870 63338 +468
- Partials 10928 11022 +94
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
gortiz
force-pushed
the
feature/micrometer-metrics
branch
from
2b8838b to
1ba1255
Compare
gortiz
changed the title
gortiz
requested review from
Jackie-Jiang and
yashmayya
and removed request for
yashmayya
Introduces a new `pinot-micrometer` metrics plugin implementing the metrics SPI as a genuinely native Micrometer backend. A single CompositeMeterRegistry fans each meter to its enabled, config-toggled export legs: - Prometheus leg (opt-in): a Micrometer PrometheusMeterRegistry exporting directly in Prometheus exposition format, optionally over an HTTP scrape endpoint. - JMX leg (opt-in): a Micrometer JmxMeterRegistry exposing meters as MBeans under Micrometer-native names. The standard Micrometer JVM/system binders (memory, GC, threads, class loader, CPU, uptime, file descriptors) are bound to the composite so basic JVM observability is exported whenever a leg is enabled. Exported metric names/resolutions follow Micrometer conventions and INTENTIONALLY differ from the legacy Yammer/Dropwizard JMX names (e.g. a meter exports a single cumulative `_total` with no EWMA rate series; a timer exports `_count`/`_sum`/`_max`). Callers needing the exact legacy names compose Yammer + Micrometer via the compound registry. The default factory (Yammer) is unchanged; this backend is opt-in. Naming is pluggable via a `MicrometerMetricNamer` SPI (`pinot.metrics.micrometer.namer.className`); the default drops the metrics-class prefix for Yammer-closer names (`pinot_server_queries_total`). Timers and distribution summaries publish configurable client-side percentiles, defaulting to the legacy Yammer set (`pinot.metrics.micrometer.percentiles`, validated to (0,1]). Gauges are registered with a strong reference to their value holder so they do not silently decay to NaN after GC (Micrometer holds gauge state weakly by default). Config keys (pinot.metrics.micrometer.*): prometheus.enabled, prometheus.port, prometheus.path, jmx.enabled, namer.className, percentiles. Tests: native-behavior tests (per-type Prometheus series + values, JVM metrics, HTTP endpoint, NaN-safety, gauge-survives-GC, JMX-leg MBeans, default/pluggable namer, default/custom/empty/malformed percentiles) and a MicrometerVsLegacyDifferenceTest characterizing how the native series differ from the legacy Yammer/Dropwizard backends.
gortiz
force-pushed
the
feature/micrometer-metrics
branch
from
1ba1255 to
46d26c8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
pinot-micrometermetrics plugin (pinot-plugins/pinot-metrics/pinot-micrometer) implementing the existing metrics SPI as a genuinely native Micrometer backend — a step toward adopting the de-facto JVM metrics standard.A single Micrometer
CompositeMeterRegistrynatively fans each meter to its enabled, config-toggled export legs (no Dropwizard store, no mirroring):PrometheusMeterRegistryexporting directly in Prometheus exposition format (no JMX hop), optionally over an HTTP scrape endpoint.JmxMeterRegistryexposing meters as MBeans under Micrometer-native names.The standard Micrometer JVM/system binders (memory, GC, threads, class loader, CPU, uptime, file descriptors) are bound to the composite, so basic JVM observability is exported whenever a leg is enabled.
Unlike
pinot-yammer/pinot-dropwizard, this backend does not reproduce the legacy JMX-derived Prometheus names. It uses native Micrometer conventions, which are a different shape:_Count,_MeanRate,_OneMinuteRate,_FiveMinuteRate,_FifteenMinuteRate(5 series, server-side EWMA)_total; no EWMA rates (rate computed downstream via Prometheusrate())_Min/_Max/_Mean/_StdDev+_50th…_999thPercentile_count/_sum/_max(+ optional client-side percentiles); no EWMA rates_ValueThis divergence is deliberate. Callers that need the exact legacy names will compose Yammer (→ JMX) + Micrometer (→ Prometheus/everything else) via the compound registry (a separate follow-up). The default metrics factory (Yammer) is unchanged; this backend is strictly opt-in via the factory class name.
Configuration
All keys under
pinot.metrics.micrometer.:prometheus.enabledfalseprometheus.port00= no server)prometheus.path/metricsjmx.enabledfalseIf no leg is enabled the registry logs a
WARN(meters are accepted but nothing is exported).Implementation notes
WeakReferenceby default and the SPI caller (AbstractMetrics) discards the returned wrapper, so without this gauges would silently decay toNaNafter GC. Covered by a dedicated GC regression test.PinotMeteredrate methods returnNaN(no EWMA equivalent in Micrometer).makePinotJmxReporteris a no-op (theJmxMeterRegistryexports continuously once in the composite);addListeneris a documented no-op (no native equivalent).Testing
Native-behavior tests assert the per-type Prometheus series + values, JVM metrics presence, the HTTP scrape endpoint, NaN-safety, gauge-survives-GC, and the JMX leg's MBeans.
MicrometerVsLegacyDifferenceTestinstantiates both a Yammer and a Micrometer registry and characterizes the divergence (legacymeanRate()finite vs nativeNaN; native scrape carries the native suffixes and none of the legacy CamelCase ones). All green.Release Notes
Adds an opt-in, native Micrometer metrics backend that exports directly to Prometheus (and/or Micrometer-native JMX) with standard JVM/system metrics. Its exported metric names/resolutions follow Micrometer conventions and differ from the legacy Yammer/Dropwizard names — it is opt-in and does not affect the default backend. Operators needing the exact legacy names can combine backends via the compound registry.