Skip to content

fix(integration): Fix flaky TestGRPCCompression by waiting for missin…#7483

Open
yeya24 wants to merge 1 commit intocortexproject:masterfrom
yeya24:fix-grpc-compression-test
Open

fix(integration): Fix flaky TestGRPCCompression by waiting for missin…#7483
yeya24 wants to merge 1 commit intocortexproject:masterfrom
yeya24:fix-grpc-compression-test

Conversation

@yeya24
Copy link
Copy Markdown
Contributor

@yeya24 yeya24 commented May 6, 2026

…g metrics

Add e2e.WaitMissingMetrics option to the WaitSumMetricsWithOptions call so the test retries when cortex_ring_members is not yet exposed by the distributor, rather than failing immediately with 'metric not found'.

What this PR does:

Ports for container: e2e-cortex-test-querier Mapping: map[80:32810 9095:32811]
    grpc_compression_test.go:44: 
        	Error Trace:	/home/runner/work/cortex/cortex/integration/grpc_compression_test.go:44
        	Error:      	Received unexpected error:
        	            	metric not found
        	            	github.com/cortexproject/cortex/integration/e2e.init
        	            		<autogenerated>:1
        	            	runtime.doInit1
        	            		/opt/hostedtoolcache/go/1.26.2/arm64/src/runtime/proc.go:8103
        	            	runtime.doInit
        	            		/opt/hostedtoolcache/go/1.26.2/arm64/src/runtime/proc.go:8070
        	            	runtime.main
        	            		/opt/hostedtoolcache/go/1.26.2/arm64/src/runtime/proc.go:258
        	            	runtime.goexit
        	            		/opt/hostedtoolcache/go/1.26.2/arm64/src/runtime/asm_arm64.s:1447
        	            	metric=cortex_ring_members service=distributor
        	            	github.com/cortexproject/cortex/integration/e2e.(*HTTPService).SumMetrics
        	            		/home/runner/work/cortex/cortex/integration/e2e/service.go:655
        	            	github.com/cortexproject/cortex/integration/e2e.(*HTTPService).WaitSumMetricsWithOptions
        	            		/home/runner/work/cortex/cortex/integration/e2e/service.go:611
        	            	github.com/cortexproject/cortex/integration.TestGRPCCompression.func1
        	            		/home/runner/work/cortex/cortex/integration/grpc_compression_test.go:44
        	            	testing.tRunner
        	            		/opt/hostedtoolcache/go/1.26.2/arm64/src/testing/testing.go:2036
        	            	runtime.goexit
        	            		/opt/hostedtoolcache/go/1.26.2/arm64/src/runtime/asm_arm64.s:1447
        	Test:       	TestGRPCCompression/ingester_client/gzip

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]
  • docs/configuration/v1-guarantees.md updated if this PR introduces experimental flags

…g metrics

Add e2e.WaitMissingMetrics option to the WaitSumMetricsWithOptions call
so the test retries when cortex_ring_members is not yet exposed by the
distributor, rather than failing immediately with 'metric not found'.

Signed-off-by: Ben Ye <benye@amazon.com>
@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label May 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer size/XS type/flaky-test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants