Skip to content

Quarantine flaky tests#4170

Draft
paulmedynski wants to merge 1 commit intomainfrom
dev/paul/quarantine-tests
Draft

Quarantine flaky tests#4170
paulmedynski wants to merge 1 commit intomainfrom
dev/paul/quarantine-tests

Conversation

@paulmedynski
Copy link
Copy Markdown
Contributor

@paulmedynski paulmedynski commented Apr 9, 2026

Summary

Quarantine 9 intermittently failing tests by adding [Trait("Category", "flaky")] so they are excluded from regular CI runs and only execute in quarantine pipeline steps.

Tests quarantined

Test File
DataStreamTest.ReadStream_ReadsStreamDataCorrectly tests/ManualTests/SQL/DataStreamTest/DataStreamTest.cs
AsyncTimeoutTest.TestDelayedAsyncTimeout tests/ManualTests/SQL/AsyncTest/AsyncTimeoutTest.cs
MetricsTest.ConnectionPoolGroupsCounter_Functional tests/ManualTests/TracingTests/MetricsTest.cs
ConnectionFailoverTests.NetworkError_..._RetryDisabled_... tests/UnitTests/SimulatedServerTests/ConnectionFailoverTests.cs
ConnectionFailoverTests.NetworkError_..._RetryEnabled_... tests/UnitTests/SimulatedServerTests/ConnectionFailoverTests.cs
ConnectionFailoverTests.TransientFault_..._ShouldConnectToPrimary tests/UnitTests/SimulatedServerTests/ConnectionFailoverTests.cs
ConnectionRoutingTests.NetworkTimeoutAtRoutedLocation_..._ShouldFail tests/UnitTests/SimulatedServerTests/ConnectionRoutingTests.cs
ConnectionPoolTest.BasicConnectionPoolingTest tests/ManualTests/SQL/ConnectionPoolTest/ConnectionPoolTest.cs
LocalizationTest.Localization_Tests tests/FunctionalTests/LocalizationTest.cs

Checklist

  • Tests updated (quarantine trait added)
  • Root causes investigated and tracked separately

Mark 9 intermittently failing tests as flaky so they are excluded
from regular CI runs (category!=flaky) and run only in quarantine
pipeline steps:

- DataStreamTest.ReadStream_ReadsStreamDataCorrectly
- AsyncTimeoutTest.TestDelayedAsyncTimeout
- MetricsTest.ConnectionPoolGroupsCounter_Functional
- ConnectionFailoverTests.NetworkError_WithUserProvidedPartner_RetryDisabled_ShouldConnectToFailoverPartner
- ConnectionFailoverTests.NetworkError_WithUserProvidedPartner_RetryEnabled_ShouldConnectToFailoverPartner
- ConnectionFailoverTests.TransientFault_WithUserProvidedPartner_ShouldConnectToPrimary
- ConnectionRoutingTests.NetworkTimeoutAtRoutedLocation_RetryDisabled_ShouldFail
- ConnectionPoolTest.BasicConnectionPoolingTest
- LocalizationTest.Localization_Tests
@github-project-automation github-project-automation bot moved this to To triage in SqlClient Board Apr 9, 2026
Copilot AI review requested due to automatic review settings April 9, 2026 13:09
@paulmedynski paulmedynski added this to the 7.1.0-preview1 milestone Apr 9, 2026
@paulmedynski paulmedynski moved this from To triage to In progress in SqlClient Board Apr 9, 2026
@paulmedynski paulmedynski added the Area\Tests Issues that are targeted to tests or test projects label Apr 9, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Quarantines a set of intermittently failing tests by marking them with [Trait("Category", "flaky")] so they are excluded from default CI test runs and executed only in dedicated “flaky” pipeline steps.

Changes:

  • Added Category=flaky trait to targeted unit, functional, and manual tests.
  • Ensures default test filters (category!=flaky) exclude these tests from regular runs.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/Microsoft.Data.SqlClient/tests/UnitTests/SimulatedServerTests/ConnectionRoutingTests.cs Marks the routed-location network timeout unit test as flaky.
src/Microsoft.Data.SqlClient/tests/UnitTests/SimulatedServerTests/ConnectionFailoverTests.cs Marks three simulated failover/partner routing tests as flaky.
src/Microsoft.Data.SqlClient/tests/ManualTests/TracingTests/MetricsTest.cs Marks the connection pool groups metrics counter manual test as flaky.
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/DataStreamTest/DataStreamTest.cs Marks the stream-reading manual test as flaky.
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ConnectionPoolTest/ConnectionPoolTest.cs Marks the basic connection pooling manual test as flaky.
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/AsyncTest/AsyncTimeoutTest.cs Marks the delayed async timeout manual test as flaky.
src/Microsoft.Data.SqlClient/tests/FunctionalTests/LocalizationTest.cs Marks the localization functional test as flaky.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area\Tests Issues that are targeted to tests or test projects

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

2 participants