Handle flaky PR builds faster: real-time unstable-tests refresh + cross-PR detector

[mazhelez]

Why

PR builds on BCApps have been failing intermittently on known-unstable tests. The existing Test Tolerance feature is the right foundation (per-test, auditable reclassification of known-unstable failures), but its detection is slow: the unstable-tests-<branch> list only refreshed twice daily from CI/CD runs, so a newly-flaky test could block unrelated PRs for up to ~12h before it was tolerated.

This PR closes that latency gap with two additive, low-risk changes that reuse the existing tolerance machinery.

What changed

1. Refresh the unstable-tests list right after each CI/CD run (UpdateUnstableTests.yaml)

• Adds a workflow_run trigger on CI/CD completion that recomputes the list for just the branch that ran, reusing the exact existing sliding-window heuristic (self-healing intact).
• Scoped to official branches and to the trigger events the window already includes, so it does no wasteful work.
• Cuts detection latency from ~12h to minutes, and aligns the implementation with DESIGN.md, which already documents the list as maintained "after each CI/CD completion".
• Also fixes a latent single-branch JSON-array serialization bug in the matrix input.

2. Cross-PR unstable-test detector (DetectUnstableTestsFromPrBuilds.yaml)

Correlates failures across PR builds: a test failing on a single PR is ambiguous (could be that PR's own change), but the same test failing across multiple unrelated PRs in a short window is almost never caused by any one PR — it's an instability.

• Runs hourly (+ on demand). For each official branch it lists Pull Request Build runs completed in the last windowHours (default 3), resolves each run's PR number + base branch (same-repo and fork PRs), downloads test results from failed runs, and additively marks any test failing on ≥ minDistinctPrs distinct PRs (default 2) as unstable.
• Distinctness is counted by PR number, so retrying the same PR doesn't inflate the count.
• Logic is split into a pure, unit-tested core (Select-CrossPrUnstableTests) and a thin gh wrapper (Find-CrossPrUnstableTests), driven by UpdateUnstableTestsFromPrBuilds.ps1. Add-FailedTestsToUnstableTests now honors a per-test reason.
• Publishes the same unstable-tests-<branch> artifact PR builds already consume; because Receive-UnstableTestsArtifact always takes the latest artifact by name, the hourly detector and the twice-daily recompute interleave correctly (latest wins).

Self-healing & the confidence trade-off

Entries added by the detector are never removed by it; the scheduled recompute drops them if they stop failing in CI/CD, and the detector re-adds them if they keep recurring — the intended self-healing behavior.

The default threshold (minDistinctPrs = 2, no base-branch guard) is deliberately aggressive for speed. Worst case: a real regression freshly landed on a base branch fails across many PRs and could be auto-tolerated until the next recompute. This is accepted for now because exposure is time-bounded by self-healing; raising minDistinctPrs or adding a base-branch CI check would trade speed for lower masking risk.

Testing

• 46/46 Pester tests pass in TestTolerance.Test.ps1 (7 new tests covering threshold, distinct-PR dedupe, rerun dedupe, extensionId distinction, and reason formatting).
• Both new workflow YAML files parse; both new/modified PowerShell files parse cleanly; the gh wrapper short-circuits on unsupported branches with no API call.
• The live Actions REST API could not be exercised from the dev environment (org SAML blocks the local token), so the gh-facing paths are covered by mocked/pure unit tests and will run fully under github.token in CI.

Files

• .github/workflows/UpdateUnstableTests.yaml — new workflow_run trigger + branch scoping
• .github/workflows/DetectUnstableTestsFromPrBuilds.yaml — new detector workflow
• build/scripts/TestTolerance/TestTolerance.psm1 — Select-CrossPrUnstableTests, Find-CrossPrUnstableTests, per-test reason
• build/scripts/TestTolerance/UpdateUnstableTestsFromPrBuilds.ps1 — detector driver
• build/scripts/TestTolerance/TestTolerance.Test.ps1 — new unit tests
• docs/features/test-tolerance/DESIGN.md — documents both paths

[github-actions]

Could not find a linked ADO work item. Please link one by using the pattern 'AB#' followed by the relevant work item number. You may use the 'Fixes' keyword to automatically resolve the work item when the pull request is merged. E.g. 'Fixes AB#1234'