Skip to content

perf: cache MatrixAccessor properties to avoid redundant recomputation#616

Open
MaykThewessen wants to merge 1 commit intoPyPSA:masterfrom
MaykThewessen:perf/cache-matrix-accessor-properties
Open

perf: cache MatrixAccessor properties to avoid redundant recomputation#616
MaykThewessen wants to merge 1 commit intoPyPSA:masterfrom
MaykThewessen:perf/cache-matrix-accessor-properties

Conversation

@MaykThewessen
Copy link

Summary

  • Change vlabels, clabels, A, c, b, sense, lb, ub, vtypes, and Q from @property to @cached_property in MatrixAccessor
  • Add all new cached properties to clean_cached_properties() for proper invalidation
  • Add test verifying caching behavior and cache invalidation

Motivation

During to_highspy() (and other direct API exporters), MatrixAccessor properties are accessed multiple times — both directly and indirectly. For example, M.A internally accesses M.clabels and M.vlabels, and M.c accesses M.flat_vars. Each access recomputes from scratch, including rebuilding sparse matrices and flattening DataFrames.

For large models (~593K variables, ~1.38M constraints), this redundant recomputation adds measurable overhead per solve call. Caching eliminates this since the underlying model data doesn't change between property accesses within a single solve.

The cache is already properly invalidated at the start of Model.solve() and Model._mock_solve() via existing clean_cached_properties() calls.

Context

See discussion in #198 (comment) for profiling data from a real-world 52-chunk DC-OPF optimization.

Test plan

  • Existing test_matrices.py tests pass (verify shapes, values, masked models)
  • New test_matrices_properties_are_cached verifies identity caching and invalidation
  • test_optimization.py highs-direct tests pass (24/25 — one pre-existing failure in test_modified_model)
  • test_io.py tests pass

Note: test_modified_model fails on upstream master as well — it's a pre-existing issue unrelated to this change.

🤖 Generated with Claude Code

Change vlabels, clabels, A, c, b, sense, lb, ub, vtypes, and Q from
@Property to @cached_property in MatrixAccessor, and add them to
clean_cached_properties() for proper invalidation.

Previously, each access to these properties recomputed from scratch.
During to_highspy(), properties like vlabels and clabels are accessed
multiple times (directly and indirectly through A), causing redundant
sparse matrix operations and DataFrame lookups. For large models
(~593K vars, ~1.38M constraints), this adds ~15s of overhead per solve.

The cache is already properly invalidated at the start of solve() and
_mock_solve() via existing clean_cached_properties() calls.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant