Hi, this follows from a discussion in aboutcode-org/vulnerablecode#2398, where I opened a PR with my team from CodeDay Labs to add mypy type checking to the project’s CI pipeline, as requested by issue aboutcode-org/vulnerablecode#226. @ziadhany suggested we discuss it here.
Approach
Rather than type-check a whole codebase at once (which fails on pre-existing errors), the approach is an incremental whitelist, where mypy only checks a curated list of already-clean files, and the list expands over time. We based this pattern on mypy’s existing-codebase guide. This whitelist uses the recommended foundational config settings as well. The goal is to catch type-related bugs earlier while allowing incremental adoption.
Concretely, per project it’s ~5 small pieces:
- mypy.ini (shared foundation settings) + per-project whitelist
- A make mypy target
- A check types step in CI
- Mypy pinned in setup.cfg [dev] extras + requirements-dev.txt
Answers to the Open Questions from Original Thread
The CI runtime impact is negligible. In the reference PR, the Check types step runs in 6 seconds (checking the current 50-file whitelist). Since the whitelist would grow towards full coverage over time, I also measured the worst case by running mypy across the entire codebase on GitHub Actions (via my fork) and it took 9 seconds.
Dependencies introduced: Only mypy is added directly. Besides mypy itself, it pulls in 5 transitive dependencies. Of those five, three (mypy_extensions, typing_extensions, and pathspec) are already present in requirements-dev.txt. The only two new ones are ast_serialize and librt, which are both small mypy-internal packages. All of these are dev/CI only with no impact on runtime.
On the question of whether it should be just in VulnerableCode or all projects, I think that I would treat VulnerableCode as a pilot or reference implementation. The whitelist approach makes adoption low risk because existing type errors outside the whitelist do not affect CI. The check only fails if a change introduces a type error in a file that has already been brought under the whitelist. Additionally, for a data-heavy project like VulnerableCode, mypy can provide significant value by catching type-related bugs earlier, making refactoring safer, and improving readability through type annotations. If the pilot proves successful, the same approach could be expanded to the other Python projects, with each project sharing the same foundational configuration while maintaining its own whitelist.
Hi, this follows from a discussion in aboutcode-org/vulnerablecode#2398, where I opened a PR with my team from CodeDay Labs to add mypy type checking to the project’s CI pipeline, as requested by issue aboutcode-org/vulnerablecode#226. @ziadhany suggested we discuss it here.
Approach
Rather than type-check a whole codebase at once (which fails on pre-existing errors), the approach is an incremental whitelist, where mypy only checks a curated list of already-clean files, and the list expands over time. We based this pattern on mypy’s existing-codebase guide. This whitelist uses the recommended foundational config settings as well. The goal is to catch type-related bugs earlier while allowing incremental adoption.
Concretely, per project it’s ~5 small pieces:
Answers to the Open Questions from Original Thread
The CI runtime impact is negligible. In the reference PR, the Check types step runs in 6 seconds (checking the current 50-file whitelist). Since the whitelist would grow towards full coverage over time, I also measured the worst case by running mypy across the entire codebase on GitHub Actions (via my fork) and it took 9 seconds.
Dependencies introduced: Only mypy is added directly. Besides mypy itself, it pulls in 5 transitive dependencies. Of those five, three (mypy_extensions, typing_extensions, and pathspec) are already present in requirements-dev.txt. The only two new ones are ast_serialize and librt, which are both small mypy-internal packages. All of these are dev/CI only with no impact on runtime.
On the question of whether it should be just in VulnerableCode or all projects, I think that I would treat VulnerableCode as a pilot or reference implementation. The whitelist approach makes adoption low risk because existing type errors outside the whitelist do not affect CI. The check only fails if a change introduces a type error in a file that has already been brought under the whitelist. Additionally, for a data-heavy project like VulnerableCode, mypy can provide significant value by catching type-related bugs earlier, making refactoring safer, and improving readability through type annotations. If the pilot proves successful, the same approach could be expanded to the other Python projects, with each project sharing the same foundational configuration while maintaining its own whitelist.