fix: reduce flaky tests by preventing container auto-removal and parallel --all interference#1753
Open
kasc0206 wants to merge 1 commit into
Open
fix: reduce flaky tests by preventing container auto-removal and parallel --all interference#1753kasc0206 wants to merge 1 commit into
kasc0206 wants to merge 1 commit into
Conversation
kasc0206
force-pushed
the
fix/flaky-tests-container-timing
branch
from
2b90ac2 to
3e83760
Compare
…llel --all interference Fix 5 flaky tests in Release mode: TestCLIStop (4 tests): - Change doLongRun to use autoRemove: false so containers persist even if they crash, preventing 'container not found' errors on subsequent inspect/stop calls. TestCLIRemove (3 tests): - Replace 'delete --all' and 'delete --all --force' with specific container names to prevent parallel test interference. TestCLIProgressAuto (1 test): - Guard ANSI assertion with isatty(STDERR_FILENO) check since progress falls back to plain text when output is redirected.
kasc0206
force-pushed
the
fix/flaky-tests-container-timing
branch
from
3e83760 to
1ef0fe3
Compare
Author
|
Hi @dcantah @crosbymichael, could you please review this PR when you have a moment? This fixes 5 flaky tests identified in Release mode testing:
The commit is SSH-signed and rebased onto the latest main. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix 5 flaky tests identified during Release mode testing (892 tests, 887 pass, 5 flaky).
Root Cause Analysis
TestCLIStop (4 failures):
doLongRundefaults toautoRemove: true(--rmflag).When a container crashes between creation and inspect/stop,
--rmauto-deletes it,causing subsequent
container inspect/container stopto fail with "container not found".TestCLIRemove (3 failures): Tests use
delete --all/delete --all --force,which operate on ALL containers. When multiple tests run in parallel, they delete
each other's containers.
TestCLIProgressAuto (1 failure):
testExplicitAnsiProgressexpects ANSI escapesequences on stderr, but ANSI output is only emitted when stderr is a TTY. When
running with output redirected (CI,
swift test > file), progress falls back to plain text.Changes
TestCLIStop.swiftdoLongRun(name:)→doLongRun(name:, autoRemove: false)TestCLIRemove.swiftdelete --all→delete <specific-names>to avoid cross-test interferenceTestCLIRemove.swiftdelete --all --force→delete --force <name>TestCLIProgressAuto.swiftisatty(STDERR_FILENO)checkNotes
upstream/mainto keep a clean history