Skip to content

Fix dmypy traversal of missing graph dependencies - #21818

Open
dnguy078 wants to merge 2 commits into
python:masterfrom
dnguy078:dnguy078/20260806_dmypy_missing_graph_deps
Open

Fix dmypy traversal of missing graph dependencies#21818
dnguy078 wants to merge 2 commits into
python:masterfrom
dnguy078:dnguy078/20260806_dmypy_missing_graph_deps

Conversation

@dnguy078

@dnguy078 dnguy078 commented Aug 6, 2026

Copy link
Copy Markdown

Fixes #20279.

Root cause

fix_module_deps() runs at the end of every follow-imports increment and
re-derives each state's edges so that state.dependencies only ever contains
IDs still present in the graph — anything dropped is moved to
state.suppressed. It does not rewrite state.ancestors.

find_reachable_changed_modules() traverses state.dependencies + ancestors
and dereferences graph[dep] unconditionally. Because ancestors are never
pruned, a stale ancestor ID survives into the next command and crashes the
daemon.

Concretely, with pkg.tests excluded from the build, the sequence is:

  1. dmypy recheck (first) drops pkg.tests from the graph as unreached, but
    pkg.tests.test_something keeps ancestors = ['pkg.tests'].
    fix_module_deps() cleans dependencies/suppressed and leaves that
    dangling ancestor edge in place.
  2. dmypy recheck (second) walks the edge and raises
    KeyError: 'pkg.tests'.

That two-step sequencing is why the issue reports the crash on repeated
rechecks rather than the first one.

The change

Skip IDs that are no longer in the graph when traversing dependency and
ancestor edges. direct_imports() gets the same guard: it performs the
identical graph[dep] dereference, and it is called from inside the
follow-imports worklist loop after fine_grained_manager.update() calls that
can remove modules and before fix_module_deps() runs at the end of the
increment, so its input is not guaranteed clean either. I could not construct a
repro for that path, so that half is defensive symmetry rather than a
demonstrated crash — happy to drop it if you would prefer the minimal diff.

A deeper alternative would be to have fix_module_deps() prune ancestors the
way it prunes dependencies. I did not do that here because ancestors also
feeds refresh_suppressed_submodules(), so pruning it is a wider behavioural
change than this crash warrants. Glad to take that route instead if you think
the invariant belongs there.

Test

testDaemonRecheckMissingGraphDependency reproduces the crash via a package
excluded with follow_imports = skip. Verified it fails without the fix with
exactly the reported KeyError: 'pkg.tests' (return code 2) and passes with
it. The daemon is started with --no-error-summary deliberately: the
checked N source files summary tracks the bundled typeshed and would make the
test fail spuriously after a typeshed sync.

Note that #20279 also has a report of the same KeyError reached through a
different route (mixing relative and absolute paths across projects). This
patch fixes the dangling-edge dereference itself, so it should cover that
route too, but the test here only exercises the excluded-package case.

Checks run locally

  • pytest -n0 -q mypy/test/testdaemon.py — 38 passed
  • pytest -n4 -q mypy/test/testfinegrained.py — 755 passed, 27 skipped
  • python runtests.py self — no issues in 341 source files
  • pre-commit run --files mypy/dmypy_server.py test-data/unit/daemon.test — passed

LLM assistance was used while investigating and preparing this change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dnguy078
dnguy078 marked this pull request as ready for review August 6, 2026 22:42
The "checked N source files" summary line varies with the bundled typeshed
version, which would make the test fail spuriously after a typeshed sync.
Start the daemon with --no-error-summary so the test asserts only on the
diagnostics that the fix actually affects.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dmypy 'KeyError' on recheck

1 participant