Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,547 changes: 764 additions & 783 deletions .openpublishing.redirection.framework.json

Large diffs are not rendered by default.

8 changes: 8 additions & 0 deletions .openpublishing.redirection.fundamentals.json
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,14 @@
"source_path_from_root": "/docs/fundamentals/code-analysis/style-rules/ide0270.md",
"redirect_url": "/dotnet/fundamentals/code-analysis/style-rules/ide0029-ide0030-ide0270"
},
{
"source_path_from_root": "/docs/fundamentals/code-analysis/style-rules/ide0390.md",
"redirect_url": "/dotnet/fundamentals/code-analysis/style-rules/ide0390-ide0391"
},
{
"source_path_from_root": "/docs/fundamentals/code-analysis/style-rules/ide0391.md",
"redirect_url": "/dotnet/fundamentals/code-analysis/style-rules/ide0390-ide0391"
},
{
"source_path_from_root": "/docs/fundamentals/code-analysis/style-rules/ide1006.md",
"redirect_url": "/dotnet/fundamentals/code-analysis/style-rules/naming-rules"
Expand Down
8 changes: 2 additions & 6 deletions docs/core/testing/microsoft-testing-platform-features.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Use the following path based on your goal:

- Need to customize terminal output: [Terminal output](./microsoft-testing-platform-terminal-output.md) (built-in)
- Need TRX or Azure DevOps reports: [Test reports](./microsoft-testing-platform-test-reports.md) (extension)
- Need GitHub Actions-native output (log groups, annotations, and job summary): [GitHub Actions report](https://www.nuget.org/packages/Microsoft.Testing.Extensions.GitHubActionsReport) (extension, experimental)
- Need GitHub Actions-native output (log groups, annotations, and job summary): [GitHub Actions report](./microsoft-testing-platform-test-reports.md#github-actions-reports) (extension, experimental)
- Need coverage data: [Code coverage](./microsoft-testing-platform-code-coverage.md) (extension)
- Need crash or hang diagnostics: [Crash and hang dumps](./microsoft-testing-platform-crash-hang-dumps.md) (extension)
- Need to retry failed tests: [Retry](./microsoft-testing-platform-retry.md#retry) (extension)
Expand All @@ -54,11 +54,7 @@ These features require installing NuGet packages.

**[Test reports](./microsoft-testing-platform-test-reports.md)**

Generate test report files (TRX, Azure DevOps).

**[GitHub Actions report](https://www.nuget.org/packages/Microsoft.Testing.Extensions.GitHubActionsReport)** (experimental, introduced in MTP 2.3.0)

Emit GitHub Actions-native workflow commands so test runs produce a first-class experience: per-assembly log groups, failure annotations (surfaced in the workflow **Annotations** tab and, when the source location resolves, on the pull request's **Files changed** diff), a Markdown job summary appended to `GITHUB_STEP_SUMMARY`, and slow-test notices. The extension activates automatically when the `GITHUB_ACTIONS` environment variable is `true`, or elsewhere with the `--report-gh` switch. Each feature can be turned on or off individually with the `--report-gh-groups`, `--report-gh-annotations`, `--report-gh-step-summary`, and `--report-gh-slow-test-notices` options, and the slow-test threshold is set with `--report-gh-slow-test-threshold`. Register it manually with `builder.AddGitHubActionsProvider()`.
Generate test report files (TRX, HTML, JUnit, CTRF, Azure DevOps, GitHub Actions).

**[Code coverage](./microsoft-testing-platform-code-coverage.md)**

Expand Down
21 changes: 12 additions & 9 deletions docs/core/testing/microsoft-testing-platform-terminal-output.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,15 +43,18 @@ The progress bar is written based on the selected mode:

## Options

| Option | Description |
|---|---|
| `--no-progress` | Disables reporting progress to screen. Deprecated in MTP 2.3.0 in favor of `--progress off`. |
| `--progress` | Controls whether progress is shown. Valid values are `auto` (default), `on` (also accepts `true`, `enable`, `1`), and `off` (also accepts `false`, `disable`, `0`). Available in MTP starting with version 2.3.0. |
| `--no-ansi` | Disables outputting ANSI escape characters to screen. |
| `--ansi` | Controls whether ANSI escape characters are emitted. Valid values are `auto` (default), `on` (also accepts `true`, `enable`, `1`), and `off` (also accepts `false`, `disable`, `0`). Available in MTP starting with version 2.3.0. |
| `--output` | Specifies the output verbosity when reporting tests. Valid values are `Normal` and `Detailed`. Default is `Normal`. |
| `--show-stdout` | Determines when to show captured standard output of a test. Valid values are `All`, `Failed`, and `None`. Default is `All`. Available in MTP starting with version 2.2.1. |
| `--show-stderr` | Determines when to show captured error output of a test. Valid values are `All`, `Failed`, and `None`. Default is `All`. Available in MTP starting with version 2.2.1. |
| Option | MTP version | Description |
|---|---|---|
| `--no-progress` | — | Disables reporting progress to screen. Deprecated in MTP 2.3.0 in favor of `--progress off`. |
| `--progress` | 2.3.0 | Controls whether progress is shown. Valid values are `auto` (default), `on` (also accepts `true`, `enable`, `1`), and `off` (also accepts `false`, `disable`, `0`). |
| `--no-ansi` | — | Disables outputting ANSI escape characters to screen. |
| `--ansi` | 2.3.0 | Controls whether ANSI escape characters are emitted. Valid values are `auto` (default), `on` (also accepts `true`, `enable`, `1`), and `off` (also accepts `false`, `disable`, `0`). |
| `--output` | — | Specifies the output verbosity when reporting tests. Valid values are `Normal` and `Detailed`. Default is `Normal`. |
| `--show-stdout` | 2.2.1 | Determines when to show captured standard output of a test. Valid values are `All`, `Failed`, and `None`. Default is `All`. |
| `--show-stderr` | 2.2.1 | Determines when to show captured error output of a test. Valid values are `All`, `Failed`, and `None`. Default is `All`. |

> [!NOTE]
> A dash (—) in the **MTP version** column marks core options that aren't tied to a specific version because they've been available since the platform's initial releases.

> [!NOTE]
> Starting with MTP 2.3.0, when MTP detects that it runs inside an LLM or AI tool environment, it suppresses the startup banner and changes the default of `--show-stdout` and `--show-stderr` from `All` to `Failed` to reduce noise.
61 changes: 44 additions & 17 deletions docs/core/testing/microsoft-testing-platform-test-reports.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Microsoft.Testing.Platform (MTP) test reports
description: Learn about the MTP extensions that create test report files (TRX, HTML, JUnit, CTRF, Azure DevOps).
description: Learn about the MTP extensions that create test report files (TRX, HTML, JUnit, CTRF, Azure DevOps, GitHub Actions).
author: evangelink
ms.author: amauryleve
ms.date: 06/16/2026
Expand Down Expand Up @@ -143,23 +143,50 @@ builder.TestHost.AddAzureDevOpsProvider();

### Options

| Option | Description |
|---|---|
| `--report-azdo` | Enables the Azure DevOps report generator. Errors and warnings are written to the output in a format that Azure DevOps understands. |
| `--report-azdo-severity` | Severity to use for reported events. Valid values are `error` (default) and `warning`. |
| `--report-azdo-flaky-history` | Queries Azure DevOps test result history for the past N days (1-90) and annotates reported failures with flakiness context. Requires `--report-azdo`. |
| `--report-azdo-demote-known-flaky` | Demotes failures that are flaky enough in the Azure DevOps history window (default threshold is 25%) from errors to warnings. Requires `--report-azdo` and `--report-azdo-flaky-history`. |
| `--report-azdo-quarantine-file` | Path to a text file that lists quarantined test fully qualified names or glob patterns. Matching failures are reported as warnings. Requires `--report-azdo`. |
| `--report-azdo-summary` | Writes a Markdown job summary at the end of the test run and uploads it through `##vso[task.uploadsummary]`. An optional file path argument overrides the default location (`{testResultsDir}/azdo-summary-{tfm}.md`). Requires `--report-azdo`. |
| `--report-azdo-stackframe-filter` | Adds regex patterns, matched against the fully qualified type prefix of each stack frame, that are skipped when the extension locates the user's call site to annotate. The option is repeatable, up to 16 patterns, and each pattern is compiled with a 500-ms match timeout. These patterns are additive to the extension's built-in MSTest assertion-implementation prefixes. Requires `--report-azdo`. |
| `--report-azdo-upload-artifacts` | Uploads test result files and/or adds build tags to Azure DevOps. Valid values are `off` (default), `tags-only`, `files`, and `all`. |
| `--report-azdo-upload-artifact-include` | Includes files in the Azure DevOps artifact upload using glob patterns relative to the test results directory. Defaults to `**/*`. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. |
| `--report-azdo-upload-artifact-exclude` | Excludes files from the Azure DevOps artifact upload using glob patterns relative to the test results directory. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. |
| `--report-azdo-upload-artifact-name` | Overrides the Azure DevOps artifact container name. Defaults to `TestResults_{assemblyName}_{tfm}`. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. |
| `--publish-azdo-test-results` | Publishes test results live to the Azure DevOps **Tests** tab. |
| `--publish-azdo-run-name` | Sets a custom Azure DevOps test run name for live test-result publishing. Requires `--publish-azdo-test-results`. |
| Option | MTP version | Description |
|---|---|---|
| `--report-azdo` | 1.9.0 | Enables the Azure DevOps report generator. Errors and warnings are written to the output in a format that Azure DevOps understands. |
| `--report-azdo-severity` | 1.9.0 | Severity to use for reported events. Valid values are `error` (default) and `warning`. |
| `--report-azdo-flaky-history` | 2.3.0 | Queries Azure DevOps test result history for the past N days (1-90) and annotates reported failures with flakiness context. Requires `--report-azdo`. |
| `--report-azdo-demote-known-flaky` | 2.3.0 | Demotes failures that are flaky enough in the Azure DevOps history window (default threshold is 25%) from errors to warnings. Requires `--report-azdo` and `--report-azdo-flaky-history`. |
| `--report-azdo-quarantine-file` | 2.3.0 | Path to a text file that lists quarantined test fully qualified names or glob patterns. Matching failures are reported as warnings. Requires `--report-azdo`. |
| `--report-azdo-summary` | 2.3.0 | Writes a Markdown job summary at the end of the test run and uploads it through `##vso[task.uploadsummary]`. An optional file path argument overrides the default location (`{testResultsDir}/azdo-summary-{tfm}.md`). Requires `--report-azdo`. |
| `--report-azdo-stackframe-filter` | 2.3.0 | Adds regex patterns, matched against the fully qualified type prefix of each stack frame, that are skipped when the extension locates the user's call site to annotate. The option is repeatable, up to 16 patterns, and each pattern is compiled with a 500-ms match timeout. These patterns are additive to the extension's built-in MSTest assertion-implementation prefixes. Requires `--report-azdo`. |
| `--report-azdo-upload-artifacts` | 2.3.0 | Uploads test result files and/or adds build tags to Azure DevOps. Valid values are `off` (default), `tags-only`, `files`, and `all`. |
| `--report-azdo-upload-artifact-include` | 2.3.0 | Includes files in the Azure DevOps artifact upload using glob patterns relative to the test results directory. Defaults to `**/*`. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. |
| `--report-azdo-upload-artifact-exclude` | 2.3.0 | Excludes files from the Azure DevOps artifact upload using glob patterns relative to the test results directory. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. |
| `--report-azdo-upload-artifact-name` | 2.3.0 | Overrides the Azure DevOps artifact container name. Defaults to `TestResults_{assemblyName}_{tfm}`. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. |
| `--publish-azdo-test-results` | 2.3.0 | Publishes test results live to the Azure DevOps **Tests** tab. |
| `--publish-azdo-run-name` | 2.3.0 | Sets a custom Azure DevOps test run name for live test-result publishing. Requires `--publish-azdo-test-results`. |

> [!NOTE]
> The Azure DevOps extension became stable in MTP 1.9.0 (`--report-azdo` and `--report-azdo-severity`). All other options in the table — `--report-azdo-flaky-history`, `--report-azdo-demote-known-flaky`, `--report-azdo-quarantine-file`, `--report-azdo-summary`, `--report-azdo-stackframe-filter`, `--report-azdo-upload-artifacts`, `--report-azdo-upload-artifact-include`, `--report-azdo-upload-artifact-exclude`, `--report-azdo-upload-artifact-name`, `--publish-azdo-test-results`, and `--publish-azdo-run-name` — are available in MTP starting with version 2.3.0.
> The **MTP version** column lists the MTP release in which each option first became available in a stable build. The Azure DevOps extension itself became stable in MTP 1.9.0 with `--report-azdo` and `--report-azdo-severity`; the remaining options were added in MTP 2.3.0.

The extension automatically detects that it is running in continuous integration (CI) environment by checking the `TF_BUILD` environment variable.

## GitHub Actions reports

The GitHub Actions report emits GitHub Actions-native workflow commands so test runs produce a first-class experience on the runner: per-assembly log groups, failed and skipped test annotations (surfaced in the workflow **Annotations** tab and, when the source location resolves, on the pull request's **Files changed** diff), a Markdown job summary appended to the file referenced by `GITHUB_STEP_SUMMARY`, and slow-test notices.

The extension activates only when the run is on GitHub Actions (the `GITHUB_ACTIONS` environment variable is `true`) and the `--report-gh` switch is set; otherwise it does nothing. When active, each feature is enabled by default and can be turned off individually with its `--report-gh-*` option.

> [!NOTE]
> Available in MTP starting with version 2.3.0. This extension is experimental, and its options and output format might change in a future version.

### Manual registration

```csharp
var builder = await TestApplication.CreateBuilderAsync(args);
builder.AddGitHubActionsProvider();
```

### Options

| Option | MTP version | Description |
|---|---|---|
| `--report-gh` | 2.3.0 | Enables the GitHub Actions report generator so test runs emit workflow commands. Requires the run to be on GitHub Actions. |
| `--report-gh-groups` | 2.3.0 | Enables or disables per-assembly log groups. Valid values are `on` (default) and `off`. Requires `--report-gh`. |
| `--report-gh-annotations` | 2.3.0 | Enables or disables annotations for failed and skipped tests. Valid values are `on` (default) and `off`. Requires `--report-gh`. |
| `--report-gh-step-summary` | 2.3.0 | Enables or disables writing a Markdown job summary to the file referenced by `GITHUB_STEP_SUMMARY`. Valid values are `on` (default) and `off`. Requires `--report-gh`. |
| `--report-gh-slow-test-notices` | 2.3.0 | Enables or disables slow-test notices. Valid values are `on` (default) and `off`. Requires `--report-gh`. |
| `--report-gh-slow-test-threshold` | 2.3.0 | The duration a test can run before a slow-test notice is emitted. Accepts a bare number of seconds or a value with a unit suffix such as `90s`, `2m`, or `1.5h`. The default is `60s`. Requires `--report-gh`. |
11 changes: 8 additions & 3 deletions docs/core/testing/unit-testing-mstest-running-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ The MSTest runner is open source and builds on the [MTP](./microsoft-testing-pla

## Enable MTP in an MSTest project

It's recommended to use [MSTest SDK](./unit-testing-mstest-sdk.md) as it greatly simplifies your project configuration and updating the project, and it ensures a proper alignment of the versions of the platform (MTP) and its extensions.
Use [MSTest SDK](./unit-testing-mstest-sdk.md) to greatly simplify your project configuration, version management, and alignment of MTP and its extensions.

When you use `MSTest SDK`, by default you're opted in to using MTP.
`MSTest.Sdk` replaces `Microsoft.NET.Sdk` as the project SDK, and MTP is enabled by default:

```xml
<Project Sdk="MSTest.Sdk/4.1.0">
Expand All @@ -39,7 +39,12 @@ When you use `MSTest SDK`, by default you're opted in to using MTP.
</Project>
```

Alternatively, you can enable MSTest runner by adding the `EnableMSTestRunner` property and setting `OutputType` to `Exe` in your project file. You also need to ensure that you're using `MSTest 3.2.0` or newer. We strongly recommend you update to the latest MSTest version available.
> [!NOTE]
> `MSTest.Sdk` works only when your project can use `Microsoft.NET.Sdk` as its base SDK. If your project requires a different SDK, don't replace the `<Project Sdk="...">` value with `MSTest.Sdk`.
>
> For example, some ASP.NET Core integration test projects use `Microsoft.NET.Sdk.Web`. `MSTest.Sdk` derives from `Microsoft.NET.Sdk`, so it doesn't import the ASP.NET Core SDK targets that `Microsoft.NET.Sdk.Web` provides.

You can also configure MSTest manually instead of using `MSTest.Sdk`. Manual configuration is required when your project needs a different top-level SDK (for example, `Microsoft.NET.Sdk.Web`), but you can also choose it with the standard `Microsoft.NET.Sdk`. Add the `EnableMSTestRunner` property and set `OutputType` to `Exe` in your project file. Then add the MSTest references that your project requires. Ensure that you're using MSTest 3.2.0 or newer, and update to the latest MSTest version available.

Consider the following example project file:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,5 +31,5 @@ Compile and run the code.

## See also

- [A Closer Look at Platform Invoke](../../../framework/interop/consuming-unmanaged-dll-functions.md#a-closer-look-at-platform-invoke)
- [Marshaling Data with Platform Invoke](../../../framework/interop/marshalling-data-with-platform-invoke.md)
- [A Closer Look at Platform Invoke](../../../standard/native-interop/consuming-unmanaged-dll-functions.md#a-closer-look-at-platform-invoke)
- [Marshaling Data with Platform Invoke](../../../standard/native-interop/marshalling-data-with-platform-invoke.md)
Loading
Loading