perf(matrices): cache MatrixAccessor properties with cached_property#716
Open
MaykThewessen wants to merge 1 commit into
Open
perf(matrices): cache MatrixAccessor properties with cached_property#716MaykThewessen wants to merge 1 commit into
MaykThewessen wants to merge 1 commit into
Conversation
Convert the remaining `@property` methods on `MatrixAccessor` (`c`, `Q`, `sol`, `dual`) to `@cached_property`. After PyPSA#630, vlabels/clabels/A/b/sense/lb/ub are materialised once in `__init__`, but these four properties still recomputed on every access. Each `MatrixAccessor` is short-lived (rebuilt on every `model.matrices` call), so cache invalidation is moot: the instance never sees a mutated model. Also fix a latent double-build in `expressions.BaseExpression._map_solution`: `m.matrices.sol` followed by `m.matrices.vlabels` constructed two accessors (each running `_build_vars`+`_build_cons`). Bind once to `M`. Benchmark on a 360k-var / 1.2k-constraint model (Python 3.12, scipy 1.17, numpy 2.4): cold (first) warm (repeat) master: 1.0 ms 1.1 ms (recomputed) patch: 1.2 ms 0.4 us (~2500x on warm) Repeat access happens in `solvers.Highs._build_solver_model` (M.c / M.Q both read at lines 1278/1299 and again in Gurobi/Mosek paths at 1586-1587, 2860-2861) and anywhere a caller assigns `M = model.matrices` then reads multiple times.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Convert the remaining
@propertymethods onMatrixAccessor(c,Q,sol,dual) tofunctools.cached_property. After #630,vlabels,clabels,A,b,sense,lb,ub,vtypesare already materialised once in__init__. The four remaining properties still recompute on every access, which shows up in solver paths that read the same attribute twice on oneMatrixAccessorinstance.Also fix a small adjacent issue in
expressions.BaseExpression._map_solution:m.matrices.solfollowed bym.matrices.vlabelsquietly constructed twoMatrixAccessorinstances (each running_build_vars+_build_cons). Binding to a localM = m.matricesreuses one.Why this is safe
Each
MatrixAccessoris short-lived:Model.matricesis itself a@propertythat returns a fresh instance on every access (linopy/model.py:350-352). So the cache lives at most as long as the local variable a caller holds, and the accessor never sees a mutated model. No newclean_cached_propertiesplumbing is needed.Where the gain shows up
Multiple solver builders read the same property twice within one
M = model.matrices:solvers.Highs._build_solver_model—M.cat line 1278,M.Qat 1299; both can be re-read in user code that inspects the accessor after solve.solvers.Gurobi._build_solver_model—M.Qat 1586, used again at 1587 in the QP branch.solvers.Mosek._build_solver_model—M.Qat 2860, again at 2861.Repeat reads of
M.sol/M.dualare also common in post-solve introspection.Benchmark
Model: 360,600 variables, 1,200 constraints, solved with HiGHS (Python 3.12, scipy 1.17, numpy 2.4):
Repro script:
Tests
pytest test/test_matrices.py test/test_objective.py test/test_solution_lookup.py test/test_model.py test/test_constraints.py test/test_linear_expression.py test/test_quadratic_expression.py→ 399 passedpytest test/test_optimization.py -k "not modified_model and not netcdf"→ 241 passed, 2 skipped(Pre-existing master failures unrelated to this PR:
test_modified_model[*]andtest_model_to_netcdf_*— both fail onupstream/masterwith the same scipy/numpy stack.)Files touched
linopy/matrices.py(+4@cached_property, +1 import)linopy/expressions.py(1 line: bindm.matricesto local before reading two attributes)