test: improve CI stability for performance and utility test suite#392
Open
aniket-shikhare-cstk wants to merge 9 commits into
Open
test: improve CI stability for performance and utility test suite#392aniket-shikhare-cstk wants to merge 9 commits into
aniket-shikhare-cstk wants to merge 9 commits into
Conversation
…cy folders Moved test coverage from test/asset and test/entry into the integration test suite (AssetQuery, ImageTransformation, ErrorHandling, UtilityMethods). Deleted the now-redundant test/asset and test/entry directories.
…t tests
Performance tests with timing assertions (expect(duration).toBeLessThan)
and concurrent load tests were intermittently failing on dev11 when 60+
sanity pipelines run simultaneously, slowing API response times.
- jest.retryTimes(2, { logErrorsBeforeRetry: true }) retries each failed
test up to 2 times
- afterEach(sleep 4000ms) provides a ~4s recovery window between attempts;
in jest-circus afterEach runs between retry attempts
Applies to:
- PerformanceBenchmarks.test.js (19 tests)
- StressTesting.test.js (14 tests)
- ConcurrentRequests.test.js (15 tests)
…ests Perf_UserAgentGeneration_Fast failed on dev11 pipeline (received 161ms vs expected < 100ms). The test does 1000 template string constructions — pure CPU work — so the threshold was unrealistically tight for a shared CI node running 60+ concurrent sanity pipelines. - Raise toBeLessThan(100) → toBeLessThan(500): still validates the operation is fast, but gives headroom under CPU contention - Add jest.retryTimes(2) + afterEach(4s sleep) to the describe block, consistent with the other performance test files, to handle momentary CPU spikes that drop after a short wait
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
jest.retryTimes(2)with a 4-secondafterEachdelay to performance-sensitive test files so transient failures caused by shared CI server load get a chance to recover before being counted as a real failureVersionUtilitytests from< 100ms/< 200msto< 500ms— the originals were calibrated for an idle machine and are too tight when the CI node is under concurrent loadFiles changed
test/integration/PerformanceTests/PerformanceBenchmarks.test.jsretryTimes(2)+ 4s inter-retry delaytest/integration/PerformanceTests/StressTesting.test.jsretryTimes(2)+ 4s inter-retry delaytest/integration/NetworkResilienceTests/ConcurrentRequests.test.jsretryTimes(2)+ 4s inter-retry delaytest/integration/UtilityTests/VersionUtility.test.jsretryTimes(2)+ raise CPU thresholds to 500msHow the retry delay works
In jest-circus (Jest 27+)
afterEachhooks run between retry attempts, so anafterEach(async () => sleep(4000))inside adescribeblock naturally inserts a 4-second pause before each re-run without needing any custom test runner.Test plan
npm run test:e2epasses locally