-
Notifications
You must be signed in to change notification settings - Fork 0
Unify time mocking strategy across the codebase #306
Copy link
Copy link
Open
Labels
testingCreate, improve or modify testsCreate, improve or modify tests
Description
Context
Time mocking has been flagged as a recurring pain point across multiple PRs:
- PR feat: add
qbftmodule #18 (QBFT): Tests were flaky due to reliance on real wall-clock time.cargo nextestwas introduced specifically for timeout/retry control. A customFakeClockwas built for QBFT to work around the problem. Several tests remain#[ignore]d with timing-related failures ("wrong decide round", "wrong prepared value"). - PR feat: add deadline #276 (Deadliner):
Utc::now()is called directly throughoutdeadline.rs, making the module untestable withtokio::time::pause()/advance(). Tests rely on real wall-clock time with short millisecond sleeps — fragile under CI load. The PR was merged with the agreement to revisit this as a separate issue.
Current state
The codebase has three unrelated time approaches coexisting:
| Approach | Where | Mockable? |
|---|---|---|
chrono::Utc::now() / chrono::Local::now() |
deadline.rs, retry.rs, peerinfo, cluster/definition.rs, p2p/k1.rs |
No (not controlled by tokio test-util) |
std::time::Instant / tokio::time::Instant |
peerinfo/protocol.rs, p2p/relay.rs, QBFT tests |
Partially (tokio variant responds to pause()/advance()) |
FakeClock (custom, crossbeam-based) |
core/qbft/fake_clock.rs |
Yes, but ad-hoc and only for QBFT |
Additionally, some modules use injectable time functions as a workaround:
retry.rshaswith_time()accepting aFn() -> DateTime<Utc>providerdeadline.rshasDeadlineFuncfor injectable deadline calculation
These are local solutions — there is no unified abstraction.
Problem
chrono::Utc::now()is invisible to tokio's test-util.tokio::time::pause()only controlstokio::time::Instantandtokio::time::sleep(). Any code usingchrono::Utc::now()will see real wall-clock time in tests, leading to flaky or non-deterministic behavior.FakeClockis QBFT-specific. It usescrossbeam::channelandstd::time::Instant— a completely different mechanism from tokio's async timers. It can't be reused for async code that usestokio::select!.- No consistent pattern for new code to follow. Each module has invented its own approach.
Proposed direction
Explore a unified time abstraction, potentially:
- Option A: Lean into
tokio::timethroughout. Replacechrono::Utc::now()withtokio::time::Instant-based calculations where possible. Usetokio::time::pause()/advance()in tests. Keepchronoonly for formatting/parsing, not as a clock source. - Option B: Create a small
Clocktrait (or crate-level abstraction as suggested in feat: add deadline #276) that wraps the time source. Production uses real time; tests inject a controllable clock. This is more flexible but adds a layer of indirection. - Option C: Hybrid — use tokio's built-in mocking for async timer code, and a simple injectable
Fn() -> DateTime<Utc>for code that needs wall-clock timestamps.
Affected areas
Key files using chrono::Utc::now() or direct time that would benefit from unification:
crates/core/src/deadline.rscrates/app/src/retry.rscrates/peerinfo/src/protocol.rs,config.rs,behaviour.rscrates/cluster/src/definition.rscrates/p2p/src/k1.rscrates/core/src/qbft/fake_clock.rs(custom mock)crates/app/src/privkeylock.rs
References
- feat: add
qbftmodule #18 — QBFT implementation (flaky tests,FakeClockintroduction) - feat: add deadline #276 — Deadliner (time mocking discussion)
- tokio test-util docs
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
testingCreate, improve or modify testsCreate, improve or modify tests