Skip to content

perf: replace np.vectorize with vectorized string ops for label names#617

Open
MaykThewessen wants to merge 2 commits intoPyPSA:masterfrom
MaykThewessen:perf/vectorize-label-names
Open

perf: replace np.vectorize with vectorized string ops for label names#617
MaykThewessen wants to merge 2 commits intoPyPSA:masterfrom
MaykThewessen:perf/vectorize-label-names

Conversation

@MaykThewessen
Copy link

Summary

  • Add vectorized_label_names() helper that uses np.char.add() for simple prefix-based label names
  • Replace np.vectorize(print_variable/print_constraint) in to_highspy(), to_gurobipy(), and to_mosek()
  • Falls back to np.vectorize for explicit_coordinate_names=True mode (which requires per-label lookups)

Motivation

np.vectorize is documented as a convenience wrapper, not a performance tool — it calls a Python function per element. For the default (non-explicit-names) path, the printer functions are trivial string concatenation (f"x{var}", f"c{cons}") that can be expressed as np.char.add(prefix, labels.astype(str)).

The improvement is modest on modern numpy/Python (~1.2x for 2M labels), but the vectorized approach is also cleaner and avoids the np.vectorize footgun for future maintainers.

Context

See discussion in #198 (comment) for the broader performance analysis.

Test plan

  • test_io.py::test_to_highspy passes (verifies HiGHS direct API export)
  • test_optimization.py highs-direct tests pass (24/25 — one pre-existing failure)
  • Verified output equality between old and new approaches on 593K + 1.38M labels

🤖 Generated with Claude Code

MaykThewessen and others added 2 commits March 13, 2026 22:15
Replace np.vectorize(print_variable/print_constraint) calls in
to_highspy(), to_gurobipy(), and to_mosek() with a vectorized_label_names()
helper that uses np.char.add for simple prefix-based names (the common case).

np.vectorize is a Python-level loop wrapper that calls a scalar function
per element. For the default (non-explicit-coordinate-names) path, the
printer is just f"x{var}" / f"c{cons}" — pure string concatenation that
can be expressed as np.char.add(prefix, labels.astype(str)).

For explicit_coordinate_names mode (which requires per-label lookups via
get_label_position()), the function falls back to np.vectorize.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@MaykThewessen
Copy link
Author

Benchmark Results

Tested on Python 3.14, numpy 2.x, macOS ARM64 with realistic LP sizes (593K variable labels + 1.38M constraint labels):

593K cols + 1.38M rows:
  np.vectorize: 0.560s
  np.char.add:  0.454s
  Speedup:      1.2x

The speedup is modest (~1.2x) because:

  1. Python 3.14 has significantly improved function call overhead, making np.vectorize faster than in older versions
  2. The dominant cost is labels.astype(str) conversion, which both approaches share

The bigger performance win comes from PR #616 (caching MatrixAccessor properties) — vlabels and clabels were being recomputed 3-4 times per solve call, each involving DataFrame flattening + vector creation. That redundant recomputation was the real source of the ~15s overhead we observed in our profiling.

This PR is still worthwhile for code clarity (np.char.add makes the intent explicit) and avoiding the np.vectorize footgun, but the performance delta is smaller than initially estimated.

@FBumann FBumann mentioned this pull request Mar 14, 2026
3 tasks
@FBumann
Copy link
Collaborator

FBumann commented Mar 14, 2026

@MaykThewessen Thanks for your contribution.
To document the performance improvements you experienced, we need a way of replicating them.
Could you add some sort of benchmark script that lets us reproduce your claims? Take #564 as a reference.
We also are working on a benchmarking suite here: #567
Maybe one of those models can be used.
The same goes for #616 #618 #619

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants