Skip to content

feat: add comprehensive type checking with mypy strict mode and pyright#20

Merged
mldangelo merged 2 commits intomainfrom
feat/enhanced-type-checking
Jan 11, 2026
Merged

feat: add comprehensive type checking with mypy strict mode and pyright#20
mldangelo merged 2 commits intomainfrom
feat/enhanced-type-checking

Conversation

@mldangelo
Copy link
Member

Summary

This PR enhances the type checking infrastructure by:

  • Upgrading mypy to strict mode with additional error codes (ignore-without-code, redundant-cast, truthy-bool, truthy-iterable, unused-awaitable)
  • Adding pyright as a second type checker for comprehensive coverage, running in strict mode
  • Updating CI to run both type checkers in parallel via a matrix strategy for efficient execution
  • Fixing source code type issues to pass both strict checkers

Why Two Type Checkers?

mypy and pyright catch different categories of issues:

Checker Strengths
mypy Standard Python type checker, excellent for protocol validation, widely adopted
pyright Microsoft's type checker, faster execution, catches additional edge cases, better IDE integration

Running both provides defense-in-depth: errors caught by one may be missed by the other.

Configuration Details

mypy (strict mode + extras):

  • All flags from --strict enabled
  • Additional error codes for comprehensive coverage
  • Override for posthog.* module to ignore missing imports

pyright (strict mode with practical exceptions):

  • typeCheckingMode = "strict"
  • Disabled checks for unknown types from third-party libraries lacking full stubs
  • Compatible with mypy's type: ignore comments

Changes

  • pyproject.toml: Enhanced mypy config, added pyright config and dependency
  • .github/workflows/test.yml: Matrix strategy for parallel type checker execution
  • src/promptfoo/cli.py: Fixed CompletedProcess generic type parameter
  • src/promptfoo/telemetry.py: Added type: ignore for untyped posthog methods
  • AGENTS.md: Updated documentation with new type checking setup
  • uv.lock: Added pyright dependency

Test plan

  • Both uv run mypy src/promptfoo/ and uv run pyright src/promptfoo/ pass locally
  • Ruff linting and formatting checks pass
  • CI runs both type checkers in parallel
  • All existing tests continue to pass

🤖 Generated with Claude Code

mldangelo and others added 2 commits January 11, 2026 02:46
- Upgrade mypy to strict mode with additional error codes (ignore-without-code,
  redundant-cast, truthy-bool, truthy-iterable, unused-awaitable)
- Add pyright as a second type checker for comprehensive coverage
- Configure pyright in strict mode with practical exceptions for
  third-party libraries lacking full stubs
- Update CI to run both type checkers in parallel via matrix strategy
- Fix CompletedProcess generic type parameter in cli.py
- Add type: ignore comments for untyped posthog methods in telemetry.py
- Update AGENTS.md documentation with new type checking setup

The dual type checker approach catches different categories of issues:
- mypy: The standard Python type checker, excellent for protocol validation
- pyright: Microsoft's type checker, catches additional edge cases

Both run in strict mode to enforce rigorous typing standards.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@mldangelo mldangelo merged commit 8b10925 into main Jan 11, 2026
11 checks passed
mldangelo added a commit that referenced this pull request Feb 24, 2026
…ht (#20)

- Upgrade mypy to strict mode with additional error codes (ignore-without-code,
  redundant-cast, truthy-bool, truthy-iterable, unused-awaitable)
- Add pyright as a second type checker for comprehensive coverage
- Configure pyright in strict mode with practical exceptions for
  third-party libraries lacking full stubs
- Update CI to run both type checkers in parallel via matrix strategy
- Fix CompletedProcess generic type parameter in cli.py
- Add type: ignore comments for untyped posthog methods in telemetry.py
- Update AGENTS.md documentation with new type checking setup

The dual type checker approach catches different categories of issues:
- mypy: The standard Python type checker, excellent for protocol validation
- pyright: Microsoft's type checker, catches additional edge cases

Both run in strict mode to enforce rigorous typing standards.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
@mldangelo mldangelo deleted the feat/enhanced-type-checking branch February 24, 2026 23:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant