fix(pytest): surface diagnostic context when no tests are collected#2006
Open
YOMXXX wants to merge 1 commit into
Open
fix(pytest): surface diagnostic context when no tests are collected#2006YOMXXX wants to merge 1 commit into
YOMXXX wants to merge 1 commit into
Conversation
`rtk pytest` (no args) collapsed pytest's collection output to a single terse line "Pytest: No tests collected" whenever the summary parser couldn't find a `passed/failed/skipped` count. That happens in two real situations both reported in rtk-ai#1417: 1. `pytest` discovers nothing in the cwd and emits `no tests ran in X.XXs` instead of a counted summary line — the LLM/user is left guessing whether it's a cwd issue, a pytest config issue, or an RTK bug. 2. Collection itself fails (`ImportError` in a conftest, typo in a test file) and pytest emits `errors during collection` plus an `ERROR` block, then `no tests ran`. The actionable error was being thrown away. Add `diagnose_no_tests`: when the existing summary collapses to "No tests collected", scan the raw pytest output for the lines that explain *why* — `rootdir:`, `configfile:`, `testpaths:`, `collected N items`, any `ERROR …` line, the `no tests ran` / `no tests collected` / `errors during collection` markers — and emit them verbatim under a header so the LLM and the user can act on them. Caps at 15 lines (a tee-style overflow hint would be wrong here — diagnostics are bounded in practice). Falls back to the terse default when *no* recognisable marker is present (e.g. RTK's parser failed but tests actually ran), so we don't flood the output with arbitrary noise on a parser regression. Tests: - test_filter_no_tests_quiet_mode_falls_back_to_terse — quiet-mode output `no tests ran in 0.05s` carries no rootdir/configfile, so the terse verdict is the right answer; this fence prevents over-eager diagnostic surfacing from leaking the original raw line. - test_filter_no_tests_verbose_surfaces_rootdir_and_configfile — locks in that rootdir / configfile / testpaths / collected count / the `no tests ran` summary are all surfaced from a verbose pytest run. - test_filter_no_tests_collection_error_surfaced — covers situation (2) above: collection-time ImportError must reach the consumer. - The pre-existing test_filter_pytest_no_tests now asserts the new enriched output (collected 0 items + no tests ran both visible) while keeping the same verdict header for grep-ability. cargo fmt / clippy --all-targets / test --bin rtk: 1912 passed, 0 warnings. Fixes rtk-ai#1417
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
`rtk pytest` (no args) collapsed pytest's collection output to a single terse line — `Pytest: No tests collected` — whenever the summary parser couldn't find a `passed/failed/skipped` count. Reporter (#1417) saw this on a real project where `rtk pytest tests/test_cli_dispatch.py -v` worked but `rtk pytest` did not, with no way to tell whether it was a cwd issue, a pytest config issue, or an RTK bug.
Two real situations both hit this:
Fix
Add `diagnose_no_tests`: when the existing summary collapses to `No tests collected`, scan the raw pytest output for the lines that explain why — `rootdir:`, `configfile:`, `testpaths:`, `collected N items`, any `ERROR …` line, plus the `no tests ran` / `no tests collected` / `errors during collection` markers — and emit them verbatim under a header so the LLM and the user can act on them.
```
Pytest: no tests collected
═══════════════════════════════════════
rootdir: /Users/x/proj
configfile: pyproject.toml
testpaths: tests, integration
collected 0 items
no tests ran in 0.05s
```
Capped at 15 lines (a tee-style overflow hint would be wrong here — diagnostic lines are bounded in practice).
Falls back to the terse default when no recognisable marker is present (e.g. RTK's parser broke but tests actually ran), so we don't flood the output with arbitrary noise on a parser regression.
Behaviour
Test plan
Notes
Fixes #1417