From f3546a48c6eefb3668eac119d3d38c79ef761a52 Mon Sep 17 00:00:00 2001 From: wyattscarpenter Date: Mon, 23 Feb 2026 20:38:05 -0500 Subject: [PATCH 1/6] Improve documentation of test format There are some advanced edge cases that were indirectly described here or not described at all; this fixes that. This PR conflicts slightly with https://github.com/python/mypy/pull/19546 as it documents the current behavior, not the improved behavior that relies less on spaces. --- test-data/unit/README.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/test-data/unit/README.md b/test-data/unit/README.md index aaf774d1b62fa..eca566168e25f 100644 --- a/test-data/unit/README.md +++ b/test-data/unit/README.md @@ -29,10 +29,12 @@ Add the test in this format anywhere in the file: - `# E: abc...` indicates that this line should result in type check error with text "abc..." - note a space after `E:` and `flags:` -- `# E:12` adds column number to the expected error +- `# E:12` adds column number to the expected error. Example: `# E:12: Incompatible [etc...]`. +- ` # ` will normally terminate the error message that comes before it; + this allows for multiple comments on the same line. Example: `# E: [etc...] # This fails`. - use `\` to escape the `#` character and indicate that the rest of the line is part of -the error message -- repeating `# E: ` several times in one line indicates multiple expected errors in one line +the error message. Example: `# E: This message includes the \# character once :) # new comment`. +- repeating ` # E: ` several times in one line indicates multiple expected errors in one line - `W: ...` and `N: ...` works exactly like `E: ...`, but report a warning and a note respectively - lines that don't contain the above should cause no type check errors - optional `[builtins fixtures/...]` tells the type checker to use @@ -45,6 +47,8 @@ the test without having to change line numbers in `[out]` - an empty `[out]` section has no effect - to add tests for a feature that hasn't been implemented yet, append `-xfail` to the end of the test name +- to mark a test as to-be-skipped, append `-skip`. For more on the distinction + between xfail and skip, see https://docs.pytest.org/en/stable/how-to/skipping.html - to run just this test, use `pytest -n0 -k testNewSyntaxBasics` From 287f567a7b8634b4f5055a59a4e2ebfd9188ee04 Mon Sep 17 00:00:00 2001 From: wyattscarpenter Date: Mon, 23 Feb 2026 20:47:45 -0500 Subject: [PATCH 2/6] Use the placeholder format from the rest of the readme --- test-data/unit/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test-data/unit/README.md b/test-data/unit/README.md index eca566168e25f..9e2157a04e126 100644 --- a/test-data/unit/README.md +++ b/test-data/unit/README.md @@ -29,9 +29,9 @@ Add the test in this format anywhere in the file: - `# E: abc...` indicates that this line should result in type check error with text "abc..." - note a space after `E:` and `flags:` -- `# E:12` adds column number to the expected error. Example: `# E:12: Incompatible [etc...]`. +- `# E:12` adds column number to the expected error. Example: `# E:12: ...`. - ` # ` will normally terminate the error message that comes before it; - this allows for multiple comments on the same line. Example: `# E: [etc...] # This fails`. + this allows for multiple comments on the same line. Example: `# E: ... # This fails`. - use `\` to escape the `#` character and indicate that the rest of the line is part of the error message. Example: `# E: This message includes the \# character once :) # new comment`. - repeating ` # E: ` several times in one line indicates multiple expected errors in one line From 7292da60b5c4896784bca908be63ae195bc2bab0 Mon Sep 17 00:00:00 2001 From: wyattscarpenter Date: Mon, 23 Feb 2026 19:01:10 -0700 Subject: [PATCH 3/6] Don't be shy about describing # correctly --- test-data/unit/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/test-data/unit/README.md b/test-data/unit/README.md index 9e2157a04e126..2d0d106b8ed7e 100644 --- a/test-data/unit/README.md +++ b/test-data/unit/README.md @@ -30,10 +30,10 @@ Add the test in this format anywhere in the file: with text "abc..." - note a space after `E:` and `flags:` - `# E:12` adds column number to the expected error. Example: `# E:12: ...`. -- ` # ` will normally terminate the error message that comes before it; - this allows for multiple comments on the same line. Example: `# E: ... # This fails`. -- use `\` to escape the `#` character and indicate that the rest of the line is part of -the error message. Example: `# E: This message includes the \# character once :) # new comment`. +- ` # ` will terminate the error message that comes before it; + this allows for multiple comments on the same line. Example: `# E: ... # This type fails`. + Use `\` to escape a `#` character to include it in the error message. Example: + `# E: This error message includes the \# character once # new comment`. - repeating ` # E: ` several times in one line indicates multiple expected errors in one line - `W: ...` and `N: ...` works exactly like `E: ...`, but report a warning and a note respectively - lines that don't contain the above should cause no type check errors From 0f00e2e2f28a7d4066f2e33048fba3f5c172ba64 Mon Sep 17 00:00:00 2001 From: wyattscarpenter Date: Mon, 23 Feb 2026 21:52:33 -0500 Subject: [PATCH 4/6] Comment on backslash non-escaping --- test-data/unit/README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/test-data/unit/README.md b/test-data/unit/README.md index 2d0d106b8ed7e..353d35426a918 100644 --- a/test-data/unit/README.md +++ b/test-data/unit/README.md @@ -34,6 +34,11 @@ with text "abc..." this allows for multiple comments on the same line. Example: `# E: ... # This type fails`. Use `\` to escape a `#` character to include it in the error message. Example: `# E: This error message includes the \# character once # new comment`. + Note that there is no support by the test file format for using `\\` to escape a backslash; + the sequence `\\#` is interpreted as `\` followed by an escaped `#`, and `\\\#` as + two `\` followed by an escaped `#`. + Note also: in all other contexts, such as line-continuation, the backslash acts + as it normally would in a Python source file. - repeating ` # E: ` several times in one line indicates multiple expected errors in one line - `W: ...` and `N: ...` works exactly like `E: ...`, but report a warning and a note respectively - lines that don't contain the above should cause no type check errors From 27da6c2f2be4c175e2ef38ec90bb7e26f90fc2f0 Mon Sep 17 00:00:00 2001 From: wyattscarpenter Date: Mon, 23 Feb 2026 21:53:24 -0500 Subject: [PATCH 5/6] Fix "works" typo --- test-data/unit/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test-data/unit/README.md b/test-data/unit/README.md index 353d35426a918..1ee36d15ef822 100644 --- a/test-data/unit/README.md +++ b/test-data/unit/README.md @@ -40,7 +40,7 @@ with text "abc..." Note also: in all other contexts, such as line-continuation, the backslash acts as it normally would in a Python source file. - repeating ` # E: ` several times in one line indicates multiple expected errors in one line -- `W: ...` and `N: ...` works exactly like `E: ...`, but report a warning and a note respectively +- `W: ...` and `N: ...` work exactly like `E: ...`, but report a warning and a note respectively - lines that don't contain the above should cause no type check errors - optional `[builtins fixtures/...]` tells the type checker to use `builtins` stubs from the indicated file (see Fixtures section below) From fdb56264c129a653888cab88739edbc9dc609660 Mon Sep 17 00:00:00 2001 From: wyattscarpenter Date: Mon, 23 Feb 2026 21:57:54 -0500 Subject: [PATCH 6/6] Mention test file comments --- test-data/unit/README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/test-data/unit/README.md b/test-data/unit/README.md index 1ee36d15ef822..119df793f6c5b 100644 --- a/test-data/unit/README.md +++ b/test-data/unit/README.md @@ -42,6 +42,10 @@ with text "abc..." - repeating ` # E: ` several times in one line indicates multiple expected errors in one line - `W: ...` and `N: ...` work exactly like `E: ...`, but report a warning and a note respectively - lines that don't contain the above should cause no type check errors +- lines that begin with `--` are test-file-format comments, and will not appear in the tested python + source code +- some test files are run in a special way by the test runner; this is typically documented in + test-file-format comments at the top of the test file - optional `[builtins fixtures/...]` tells the type checker to use `builtins` stubs from the indicated file (see Fixtures section below) - optional `[out]` is an alternative to the `# E: ` notation: it indicates that