Skip to content

Add cooperative slow-test diagnostics for browser#10128

Merged
Evangelink merged 9 commits into
mainfrom
dev/amauryleve/browser-hang-diagnostics
Jul 24, 2026
Merged

Add cooperative slow-test diagnostics for browser#10128
Evangelink merged 9 commits into
mainfrom
dev/amauryleve/browser-hang-diagnostics

Conversation

@Evangelink

@Evangelink Evangelink commented Jul 22, 2026

Copy link
Copy Markdown
Member

Browser and WASI test runs cannot use the HangDump extension because it depends on process APIs and named pipes. This adds a lightweight diagnostic path that identifies active tests which remain running without relying on unsupported browser threading or process facilities.

  • Track active tests by UID in the shared simplified Browser/WASI output device.
  • Emit durable [slow] still running after ... lines using the existing MTP_PROGRESS_SLOW_TEST_SECONDS setting (0 disables reporting).
  • Share one exponential threshold/backoff state across Browser/WASI diagnostics, terminal heartbeat output, and the Azure DevOps/GitHub Actions slow-test reporter base.
  • Skip obsolete catch-up thresholds after delayed polls while reporting the actual elapsed duration.
  • Keep test starts silent so immediate per-test progress remains a separate concern.
  • Reuse the localized slow-test message without adding public API.
  • Document that cooperative reporting diagnoses asynchronous hangs, but cannot run while a synchronous test blocks the sole WebAssembly thread.

Validation includes focused platform and extension unit tests, warning-free multi-target builds for the platform and both linked-source CI extensions, and publishing/running BrowserPlayground under Node. MSBuild validation was captured with binary logs.

Related to #2196, specifically #2196 (comment).

Track active tests internally in the simplified browser and WASI output path, and report long-running tests with exponential backoff without relying on threads, processes, or named pipes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 2eba6c9f-f63f-4726-a0ce-f90ed2404431
Copilot AI review requested due to automatic review settings July 22, 2026 07:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds cooperative slow-test diagnostics for browser/WASI environments.

Changes:

  • Tracks active tests and reports slow tests with exponential backoff.
  • Shares terminal progress configuration and localized messaging.
  • Adds focused tests, API tracking, and browser documentation.
Show a summary per file
File Description
SimplifiedConsoleOutputDeviceTests.cs Tests tracking and reporter lifecycle.
ActiveTestTrackerTests.cs Tests thresholds, backoff, and UID handling.
TerminalOutputDevice.Initialization.cs Uses shared progress configuration.
TerminalOutputDevice.cs Removes relocated constants.
SimplifiedConsoleOutputDeviceBase.cs Integrates tracking and cooperative reporting.
ProgressReportingConfiguration.cs Centralizes environment-based thresholds.
ActiveTestTracker.cs Implements active-test tracking and backoff.
InternalAPI.Unshipped.txt Tracks new internal APIs.
BrowserPlayground/README.md Documents slow-test diagnostics.

Review details

  • Files reviewed: 9/9 changed files
  • Comments generated: 2
  • Review effort level: Medium

Comment thread samples/BrowserPlayground/README.md Outdated
@Evangelink
Evangelink marked this pull request as draft July 22, 2026 07:52
@github-actions

This comment has been minimized.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Summary

Well-structured PR that adds cooperative slow-test diagnostics to the simplified console output device, enabling browser-wasm scenarios where timers/threads are unavailable. The design is sound: async polling with exponential backoff, clean separation of tracker logic, and good test coverage.

Findings

Severity Finding
⚠️ Medium Potential race in OnTestSessionFinishingAsync — fields _slowTestReporterCancellationTokenSource and _slowTestReporterTask are read/nulled without atomicity guarantees. Use Interlocked.Exchange.
💡 Low GetDueDiagnostics() allocates on every poll even when no tests are due (empty list → array). Consider early-return with cached empty array or avoid the list allocation.
💡 Low The reporter won't restart if a second test session starts after the first finishes (flag not reset). Probably fine for current usage but worth a comment.

Verdict

The race condition in session teardown is the only item worth addressing before merge — the rest are suggestions. Overall this is clean, well-tested code with good documentation.

Comment thread src/Platform/Microsoft.Testing.Platform/OutputDevice/ActiveTestTracker.cs Outdated
Extract exponential threshold state for reuse by browser and WASI diagnostics, terminal heartbeat output, and CI slow-test reporters.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 2eba6c9f-f63f-4726-a0ce-f90ed2404431
Copilot AI review requested due to automatic review settings July 22, 2026 10:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 13/13 changed files
  • Comments generated: 3
  • Review effort level: Medium

Comment thread src/Platform/Microsoft.Testing.Platform/OutputDevice/ActiveTestTracker.cs Outdated
@Evangelink
Evangelink marked this pull request as ready for review July 24, 2026 08:46
@Evangelink
Evangelink enabled auto-merge (squash) July 24, 2026 08:46
Close empty-result lifecycle tracking, serialize reporter teardown, revalidate diagnostics before emission, reduce cooperative polling allocations, and forward browser slow-test configuration through both launchers.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd
Copilot AI review requested due to automatic review settings July 24, 2026 09:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 20/20 changed files
  • Comments generated: 1
  • Review effort level: Medium

Resolve the active-progress test conflict with current main and propagate outcome-less test completion to simplified, terminal, Azure DevOps, and GitHub Actions active-test tracking without routing it through result classifiers.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd
Copilot AI review requested due to automatic review settings July 24, 2026 10:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (2)

src/Adapter/MSTest.TestAdapter/TestingPlatformAdapter/MSTestTestNodeConverter.cs:61

  • This completion marker is still ignored by other active-test consumers. HangDumpActivityIndicator.cs:132-146 and CrashDumpSequenceLogger.cs:166-181 only end tests for a TestNodeStateProperty, while VideoRecorderSessionHandler.DataConsumer.cs:22-58 leaves the UID in _inFlight when state is null. A programmatically dropped test will therefore remain “running” in hang/crash diagnostics, and video segment pruning can remain pinned to its start time. Handle this marker as an outcome-less completion in those consumers as well, with regression coverage.
        testNode.Properties.Add(TestNodeExecutionCompletedProperty.CachedInstance);

src/Platform/Microsoft.Testing.Platform/OutputDevice/SimplifiedConsoleOutputDeviceBase.cs:516

  • Revalidating only by UID has an ABA race. While this method waits for the output lock, the due test can complete and a new execution with the same UID can start; the new entry makes IsActive return true, so the stale diagnostic (old name/elapsed time) is emitted even though the replacement has not reached the threshold. Same-UID restart is explicitly supported by ConsumeAsync_EmptyResult_AllowsSameTestToReportProgressAgain. Carry a tracker generation/entry identity in the diagnostic and verify that exact entry before logging.
                if (!_activeTestTracker.IsActive(diagnostic.Uid))
  • Files reviewed: 25/25 changed files
  • Comments generated: 1
  • Review effort level: Medium

@github-actions

This comment has been minimized.

Verify the exact outcome-less completion marker in adapter tests and stabilize terminal reporter access with a runtime invariant check and non-null local alias.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd
Copilot AI review requested due to automatic review settings July 24, 2026 10:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 26/26 changed files
  • Comments generated: 1
  • Review effort level: Medium

@github-actions

This comment has been minimized.

Clear outcome-less completion from hang, crash, and video diagnostics, and validate slow-test snapshots against the exact active execution to avoid same-UID ABA reports.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd
Copilot AI review requested due to automatic review settings July 24, 2026 11:22
Prove the test is actively tracked before completion, then verify completion prevents any additional slow-test diagnostic.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (1)

src/Adapter/MSTest.TestAdapter/TestingPlatformAdapter/MtpTestResultRecorder.cs:48

  • This neutral completion update is not preserved by either server transport. In --server dotnettestcli, TerminalOutputDevice returns before handling it and DotnetTestDataConsumer.GetTestNodeDetails drops every node without a TestNodeStateProperty, so the SDK receives the preceding in-progress message but no message that clears it. The JSON-RPC path forwards the update, but both serializers classify state-less nodes as node-type: group, turning the action node into a group update on the wire. Please add an explicit transport representation/filter for outcome-less completion and cover both server paths; otherwise programmatically dropped tests remain active or corrupt the remote node shape.
    public Task RecordEmptyResultAsync(UnitTestElement testElement)
        => PublishAsync(MSTestTestNodeConverter.ToEmptyResultTestNode(testElement, _isTrxEnabled));
  • Files reviewed: 33/33 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

Copilot AI review requested due to automatic review settings July 24, 2026 11:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 33/33 changed files
  • Comments generated: 1
  • Review effort level: Medium

@github-actions

This comment has been minimized.

Serialize outcome-less completion as an action returning to discovered in both JSON formatter paths, preventing an in-progress node from becoming a group without recording an outcome.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd
Copilot AI review requested due to automatic review settings July 24, 2026 11:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 36/36 changed files
  • Comments generated: 1
  • Review effort level: Medium

@github-actions

This comment has been minimized.

Return outcome-less completion to discovered on the dotnet-test pipe and close the corresponding OpenTelemetry activity without recording a test outcome.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 215e1000-a407-4d84-82a4-2d3696d566fd
Copilot AI review requested due to automatic review settings July 24, 2026 12:05
@github-actions

Copy link
Copy Markdown
Contributor

🧪 Test quality grade — PR #10128

GradeTestMutationNotesHow to improve
B (80–89) new AzureDevOpsSlowTestReporterTests.
ConsumeAsync_
ExecutionCompleted_
StopsTrackingAsync
2/3 killed Single assertion verifies no slow-test output; a mutation that skips InProgress tracking would survive. Add a prior ScanOnceAsync call after InProgress to confirm the test was being tracked before ExecutionCompleted stops it.
A (90–100) mod MtpTestResultRecorderTests.
MtpTestResultRecorder_
RecordEmptyResult_
PublishesExecutionCompletedNode
3/3 killed Asserts both presence of TestNodeExecutionCompletedProperty and absence of TestNodeStateProperty, covering the key contract.
A (90–100) new CrashDumpSequenceLoggerTests.
ConsumeAsync_
ExecutionCompleted_
WritesEndedRecord
3/3 killed Verifies file content with start/end predicates; timestamp is correctly left unchecked (non-deterministic).

This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /grade-tests.

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · sonnet46 56.6 AIC · ⌖ 4.23 AIC · ⊞ 10.3K · [◷]( · )

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 42/42 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@Evangelink
Evangelink disabled auto-merge July 24, 2026 13:26
@Evangelink
Evangelink merged commit a091361 into main Jul 24, 2026
32 of 35 checks passed
@Evangelink
Evangelink deleted the dev/amauryleve/browser-hang-diagnostics branch July 24, 2026 13:26
github-actions Bot added a commit that referenced this pull request Jul 24, 2026
Aligns InProgressTest in SlowTestReporterBase with the pattern used by
ActiveTest in ActiveTestTracker (added in the same commit #10128),
reducing boilerplate while preserving identical behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants