Add cooperative slow-test diagnostics for browser#10128
Conversation
Track active tests internally in the simplified browser and WASI output path, and report long-running tests with exponential backoff without relying on threads, processes, or named pipes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2eba6c9f-f63f-4726-a0ce-f90ed2404431
There was a problem hiding this comment.
Pull request overview
Adds cooperative slow-test diagnostics for browser/WASI environments.
Changes:
- Tracks active tests and reports slow tests with exponential backoff.
- Shares terminal progress configuration and localized messaging.
- Adds focused tests, API tracking, and browser documentation.
Show a summary per file
| File | Description |
|---|---|
SimplifiedConsoleOutputDeviceTests.cs |
Tests tracking and reporter lifecycle. |
ActiveTestTrackerTests.cs |
Tests thresholds, backoff, and UID handling. |
TerminalOutputDevice.Initialization.cs |
Uses shared progress configuration. |
TerminalOutputDevice.cs |
Removes relocated constants. |
SimplifiedConsoleOutputDeviceBase.cs |
Integrates tracking and cooperative reporting. |
ProgressReportingConfiguration.cs |
Centralizes environment-based thresholds. |
ActiveTestTracker.cs |
Implements active-test tracking and backoff. |
InternalAPI.Unshipped.txt |
Tracks new internal APIs. |
BrowserPlayground/README.md |
Documents slow-test diagnostics. |
Review details
- Files reviewed: 9/9 changed files
- Comments generated: 2
- Review effort level: Medium
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Note
🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.
Summary
Well-structured PR that adds cooperative slow-test diagnostics to the simplified console output device, enabling browser-wasm scenarios where timers/threads are unavailable. The design is sound: async polling with exponential backoff, clean separation of tracker logic, and good test coverage.
Findings
| Severity | Finding |
|---|---|
Potential race in OnTestSessionFinishingAsync — fields _slowTestReporterCancellationTokenSource and _slowTestReporterTask are read/nulled without atomicity guarantees. Use Interlocked.Exchange. |
|
| 💡 Low | GetDueDiagnostics() allocates on every poll even when no tests are due (empty list → array). Consider early-return with cached empty array or avoid the list allocation. |
| 💡 Low | The reporter won't restart if a second test session starts after the first finishes (flag not reset). Probably fine for current usage but worth a comment. |
Verdict
The race condition in session teardown is the only item worth addressing before merge — the rest are suggestions. Overall this is clean, well-tested code with good documentation.
Extract exponential threshold state for reuse by browser and WASI diagnostics, terminal heartbeat output, and CI slow-test reporters. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2eba6c9f-f63f-4726-a0ce-f90ed2404431
Close empty-result lifecycle tracking, serialize reporter teardown, revalidate diagnostics before emission, reduce cooperative polling allocations, and forward browser slow-test configuration through both launchers. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd
Resolve the active-progress test conflict with current main and propagate outcome-less test completion to simplified, terminal, Azure DevOps, and GitHub Actions active-test tracking without routing it through result classifiers. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (2)
src/Adapter/MSTest.TestAdapter/TestingPlatformAdapter/MSTestTestNodeConverter.cs:61
- This completion marker is still ignored by other active-test consumers.
HangDumpActivityIndicator.cs:132-146andCrashDumpSequenceLogger.cs:166-181only end tests for aTestNodeStateProperty, whileVideoRecorderSessionHandler.DataConsumer.cs:22-58leaves the UID in_inFlightwhen state is null. A programmatically dropped test will therefore remain “running” in hang/crash diagnostics, and video segment pruning can remain pinned to its start time. Handle this marker as an outcome-less completion in those consumers as well, with regression coverage.
testNode.Properties.Add(TestNodeExecutionCompletedProperty.CachedInstance);
src/Platform/Microsoft.Testing.Platform/OutputDevice/SimplifiedConsoleOutputDeviceBase.cs:516
- Revalidating only by UID has an ABA race. While this method waits for the output lock, the due test can complete and a new execution with the same UID can start; the new entry makes
IsActivereturn true, so the stale diagnostic (old name/elapsed time) is emitted even though the replacement has not reached the threshold. Same-UID restart is explicitly supported byConsumeAsync_EmptyResult_AllowsSameTestToReportProgressAgain. Carry a tracker generation/entry identity in the diagnostic and verify that exact entry before logging.
if (!_activeTestTracker.IsActive(diagnostic.Uid))
- Files reviewed: 25/25 changed files
- Comments generated: 1
- Review effort level: Medium
This comment has been minimized.
This comment has been minimized.
Verify the exact outcome-less completion marker in adapter tests and stabilize terminal reporter access with a runtime invariant check and non-null local alias. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd
This comment has been minimized.
This comment has been minimized.
Clear outcome-less completion from hang, crash, and video diagnostics, and validate slow-test snapshots against the exact active execution to avoid same-UID ABA reports. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd
Prove the test is actively tracked before completion, then verify completion prevents any additional slow-test diagnostic. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (1)
src/Adapter/MSTest.TestAdapter/TestingPlatformAdapter/MtpTestResultRecorder.cs:48
- This neutral completion update is not preserved by either server transport. In
--server dotnettestcli,TerminalOutputDevicereturns before handling it andDotnetTestDataConsumer.GetTestNodeDetailsdrops every node without aTestNodeStateProperty, so the SDK receives the preceding in-progress message but no message that clears it. The JSON-RPC path forwards the update, but both serializers classify state-less nodes asnode-type: group, turning the action node into a group update on the wire. Please add an explicit transport representation/filter for outcome-less completion and cover both server paths; otherwise programmatically dropped tests remain active or corrupt the remote node shape.
public Task RecordEmptyResultAsync(UnitTestElement testElement)
=> PublishAsync(MSTestTestNodeConverter.ToEmptyResultTestNode(testElement, _isTrxEnabled));
- Files reviewed: 33/33 changed files
- Comments generated: 0 new
- Review effort level: Medium
This comment has been minimized.
This comment has been minimized.
Serialize outcome-less completion as an action returning to discovered in both JSON formatter paths, preventing an in-progress node from becoming a group without recording an outcome. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd
This comment has been minimized.
This comment has been minimized.
Return outcome-less completion to discovered on the dotnet-test pipe and close the corresponding OpenTelemetry activity without recording a test outcome. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd
🧪 Test quality grade — PR #10128
This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with
|
Aligns InProgressTest in SlowTestReporterBase with the pattern used by ActiveTest in ActiveTestTracker (added in the same commit #10128), reducing boilerplate while preserving identical behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Browser and WASI test runs cannot use the HangDump extension because it depends on process APIs and named pipes. This adds a lightweight diagnostic path that identifies active tests which remain running without relying on unsupported browser threading or process facilities.
[slow] still running after ...lines using the existingMTP_PROGRESS_SLOW_TEST_SECONDSsetting (0disables reporting).Validation includes focused platform and extension unit tests, warning-free multi-target builds for the platform and both linked-source CI extensions, and publishing/running BrowserPlayground under Node. MSBuild validation was captured with binary logs.
Related to #2196, specifically #2196 (comment).