Disable the Hypothesis deadline on the copartitioned-assignor property tests - #752
Open
wbarnha wants to merge 2 commits into
Open
Disable the Hypothesis deadline on the copartitioned-assignor property tests#752wbarnha wants to merge 2 commits into
wbarnha wants to merge 2 commits into
Conversation
The copartitioned-assignor property tests run on PyPy since the PyPy skips were removed. They carried a fixed 4000ms Hypothesis deadline, but PyPy's JIT warmup makes a single example's timing vary by over a second, so runs intermittently exceed it and Hypothesis reports DeadlineExceeded / FlakyFailure -- failing the (non-required) PyPy leg and turning master CI red even though the assignment produced is valid. These tests assert assignment correctness, not performance, so set deadline=None (as the DeadlineExceeded message itself recommends). The property assertions are unchanged and the job timeout still guards against a real hang. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HHPL4VFWQRQPpjR1gXSKyL
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #752 +/- ##
=======================================
Coverage 96.06% 96.06%
=======================================
Files 103 103
Lines 11072 11072
Branches 1191 1191
=======================================
Hits 10636 10636
Misses 345 345
Partials 91 91 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The
Python pypy3.11/Cython: falseCI leg fails intermittently ontests/meticulous/assignor/test_copartitioned_assignor.py, with an error that is about timing rather than about assignment correctness:These four property tests assert that the assignment produced is valid, not that it is produced within any particular time. PyPy's JIT warmup makes a single example's timing vary by well over a second, so a fixed 4000 ms deadline is tripped intermittently even though the assignment is correct. This sets
TEST_DEADLINE = None; the job's own timeout still guards against a real hang.Why this is worth landing on its own
This branch already existed with the fix but no PR, so the flake keeps failing unrelated PRs. It is not a hypothetical:
partitions=243, replicas=62, num_clients=686, num_removal_clients=1), 5061 ms and 5154 ms against the same 4000 ms deadline.faust/assignor/ortests/meticulous/.Both of those PRs currently carry
2f04895, a cherry-pick of this branch's commit, purely to get their CI green. Once this lands they can drop it, or it merges as a no-op since the change is identical.Testing
Full suite passes (2207 passed, 4 skipped). The four assignor property tests pass with the deadline removed, and the PyPy leg is green on both PRs carrying the cherry-pick.
Branch was 15 commits behind; master has been merged in so CI runs against the current base.
🤖 Generated with Claude Code
https://claude.ai/code/session_019oX4oGCQGgvPJHabjwfaBk
Generated by Claude Code