-
Notifications
You must be signed in to change notification settings - Fork 936
Sync instrument stress test #7986
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sync instrument stress test #7986
Conversation
| AggregationTemporality aggregationTemporality, | ||
| InstrumentType instrumentType, | ||
| MemoryMode memoryMode, | ||
| InstrumentValueType instrumentValueType) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Parameterized testing really shines here.
| Uninterruptibles.joinUninterruptibly(collectThread); | ||
|
|
||
| // Assert collected data is consistent with recorded measurements by independently computing the | ||
| // expected aggregated value and comparing to the actual results. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the key bit. We independently compute the expected aggregated state outside the concurrent record environment, and compare against the SDK, treating it as a black box.
sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/SynchronousInstrumentStressTest.java
Outdated
Show resolved
Hide resolved
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7986 +/- ##
=========================================
Coverage 90.16% 90.16%
Complexity 7478 7478
=========================================
Files 836 834 -2
Lines 22550 22540 -10
Branches 2224 2236 +12
=========================================
- Hits 20333 20324 -9
+ Misses 1515 1512 -3
- Partials 702 704 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
...telemetry/sdk/metrics/internal/aggregator/DoubleBase2ExponentialHistogramAggregatorTest.java
Outdated
Show resolved
Hide resolved
zeitlinger
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! I've asked Copilot for an additional review.
sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/SynchronousInstrumentStressTest.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Adds a comprehensive stress test for synchronous metric instruments to exercise record/collect concurrency across all key configuration combinations, while removing older ad-hoc stress tests.
Changes:
- Introduces a new parameterized synchronous-instrument stress test spanning temporality, instrument type/aggregation, memory mode, and value type combinations.
- Removes legacy stress/stochastic concurrency tests spread across multiple unit test classes.
- Deletes the now-unused
StressTestRunnerutility.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/internal/state/SynchronousMetricStorageTest.java | Removes the old concurrent record/collect stress test and related imports. |
| sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/internal/aggregator/DoubleExplicitBucketHistogramAggregatorTest.java | Removes the multithreaded histogram update test and related imports. |
| sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/SynchronousInstrumentStressTest.java | Adds a new parameterized stress test plus helpers to copy/reduce collected MetricData. |
| sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/StressTestRunner.java | Deletes the generic stress-test runner utility. |
| sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/SdkLongUpDownCounterTest.java | Removes instrument-specific stress tests that used StressTestRunner. |
| sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/SdkLongHistogramTest.java | Removes instrument-specific stress tests that used StressTestRunner. |
| sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/SdkLongGaugeTest.java | Removes instrument-specific stress tests that used StressTestRunner. |
| sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/SdkLongCounterTest.java | Removes instrument-specific stress tests that used StressTestRunner. |
| sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/SdkDoubleUpDownCounterTest.java | Removes instrument-specific stress tests that used StressTestRunner. |
| sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/SdkDoubleHistogramTest.java | Removes instrument-specific stress tests that used StressTestRunner. |
| sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/SdkDoubleGaugeTest.java | Removes instrument-specific stress tests that used StressTestRunner. |
| sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/SdkDoubleCounterTest.java | Removes instrument-specific stress tests that used StressTestRunner. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
I'm doing some work to improve the performance of metric recording under high contention, and touching sensitive code in
DefaultSynchronousMetricStorageand other concurrent code.I have promising results, but the first step is buffing up our stress tests for synchronous instruments.
A stress test for synchronous instruments concurrently runs record and collect operations at a high rate, and tries to find instances of concurrency bugs: lost writes, duplicate writes, partial writes, etc.
We need to test every combination of of the following dimensions, since each plays a key role in the recording and collecting code flows.
That's 40 combinations. Our current test suite is hand rolled for some of these cases, but misses many.