perf: scale the objective before handing the model to the solver - #104
Conversation
CBC judges improvements against absolute tolerances of about 1e-7. Prices are given in currency per Wh, so the raw objective coefficients land close to that bound: the peak attenuation tie-breakers sit around 1e-9 per W, and the cost terms are not far above. Improvements that small are indis- tinguishable from numerical noise, so the solver both explores a degenerate search tree and stops on solutions that are not optimal. Multiplying the assembled objective by a constant does not change its argmax, but it lifts the coefficients into a range the solver can resolve. The reported objective value is recalculated from the solution by get_clean_objective_value(), so the API response stays in the original unit and no expected test result changes because of the scaling itself. Measured over all test cases, each run with the stored strategy and with every charging strategy (14 cases x 5 settings, interleaved A/B, min of 7): total solve time 12.78 s -> 10.07 s (0.79x) worst case, none 1.31 s -> 0.79 s (0.60x) worst case, grid peaks 2.17 s -> 1.03 s (0.47x) No solution got economically worse, several got better. The largest gain is 021-min-pv-use-case, where the previous model returned a solution 0.59% below the optimum. Case 018-high-soc-initial got slower (0.13 s -> 0.23 s with attenuate_grid_peaks), which is branching luck rather than a trend. The new test case pins the recovered optimum: it fails without the scaling.
The peak attenuation strategies create one ramp variable per time step, but the ramp constraints only cover the steps that have a predecessor. The variable of the first step is therefore never constrained, it only carries the ramp penalty and is driven to zero. Index the ramps by the step they lead into instead, which removes one variable per leveled grid side without changing the model.
…ed case The scenario that proves the objective scaling is 021 solved without a charging strategy. Adding it as a data driven case duplicated 2839 lines of fixture that differ from the existing file in one field. Read the request from that file instead and assert the objective in a dedicated test module, as AGENTS.md suggests for behaviour the data driven harness cannot express. The assertion is a lower bound rather than an equality: a future model that finds an even better solution here should not fail this test. Also drops the duplicated rationale at the call site, the comment on OBJECTIVE_SCALE covers it.
…t moves Both tests on this branch fail against current main. `test_objective_scaling` asserted 021 reaches 1.4327495658 with the charging strategy forced to `none`. That number is not reachable and never was: with `none` the case returns 1.4243782720 at every scale from 1e0 to 1e8, and 1.4327495637 is what 021 returns under its stored `attenuate_demand_peaks` strategy *unscaled*. The threshold was read off a different strategy and the wrong side of the change, so the test could only ever fail. The 0.59 percent recovery it was written to pin does not reproduce either. Main has moved since the measurement, and #88 tightening the indicator big-Ms already removed that suboptimal stop. Measured across all 19 cases x 5 charging strategies, scaled against unscaled, the true objective now differs by at most 6e-7 anywhere. The speed claim does hold: 14.34 s against 10.78 s in total, 0.75x. So the test now pins the guarantee the change can actually make, on three cases x three strategies: a uniform positive factor cannot move the argmax, so the solver must not stop anywhere worse than it does unscaled. It compares the model objective rather than the reported one, because `get_clean_objective_value` credits battery gain as `s[T-1] - s[0]` and every one of these cases charges more in the first step once scaled. That reads as a loss of about half a percent where the true objective is in fact a shade better, which is the same `s[0]` quirk documented in AGENTS.md and the reason #88 had to refresh 013. `019` is refreshed for exactly that reason. Its true objective moves by +2.2e-07, from 2.7861544729 to 2.7861546924, while first step charging goes from 666.67 Wh to 724.44 Wh and the reported value therefore drops 0.71 percent. No money is given up. `012` and `021` show the same effect but only under the peak strategies, which the golden suite does not exercise for those cases, so their stored responses are unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI runs on linux, where the scaled and unscaled solves of 019 without a charging strategy land 4e-9 apart. Asserting at 1e-9 measured that noise rather than the change: CBC converges to about 1e-7 in objective units, so nothing tighter than that says anything about the model. A part per million is still three orders below the smallest move worth catching, the 0.4 percent that the strategy weight table records at x10. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
The risk of undercutting the solver's precision floor is a good finding ... ... but hang on, we shall have a look at the max. spread of double precision as well. Let me check with Claude .... |
... in order to make sure that the scaling of penalties is consistent and considers all real price inputs to the model.
Objective term hierarchy — updated after the
|
| Tier | Term | Formula | Multiple of penalty_base |
Scaled coefficient range | Intent |
|---|---|---|---|---|---|
| 1 — hard-avoidance | SOC limit exceedance | prc_soc_exc_pen = penalty_base·1000 |
×1000 | 1.6e5 – 3.0e8 | must basically never happen |
| Unmet charge goal | prc_e_goal_pen = penalty_base·100 |
×100 | 1.6e4 – 3.0e7 | strong avoid | |
| Unmet charge demand | prc_p_goal_pen = penalty_base·100 (fixed — was ·(max dt/3600)·100) |
×100 | 1.6e4 – 3.0e7 | strong avoid, now equal weight to the goal penalty above, as intended | |
| Import limit exceedance | prc_e_grid_imp_pen = penalty_base·100 |
×100 | 1.6e4 – 3.0e7 | strong avoid | |
| Export limit exceedance | prc_e_grid_exp_pen·(1−t·1e-5) = penalty_base·100·(≈1) |
×100 | 1.6e4 – 3.0e7 | strong avoid | |
| 2 — real economics | Grid import cost | p_N[t] |
(raw, and now also feeds penalty_base via max_import_price) |
~-1 – 3e5 | actual currency |
| Grid export revenue | p_E[t] |
(raw, now also feeds penalty_base via max_export_price) |
~0 – 3e5 | actual currency | |
| Final SOC value | bat.p_a |
(raw, now also feeds penalty_base directly) |
~1.7e2 (up to whatever p_a is configured) |
actual currency | |
| Demand-rate charge | grid.prc_p_exc_imp |
(raw; prc_p_exc_imp / horizon_hours now also feeds penalty_base) |
unbounded, but tier 1 now tracks it | actual currency | |
| 3 — tie-break nudges | Peak leveling | prc_p_peak = penalty_base·1e-3 |
×1e-3 | 0.16 – 300 | shape, not cost |
| Charge-before-export | min_import_price·2e-5·(T−t) |
×2e-5·(T−t) | ~1e-3 – tens | shape, not cost | |
| Discharge-before-import | min_import_price·5e-6·(T−t) |
×5e-6·(T−t) | ~1e-4 – tens | shape, not cost | |
| Charging priority | min_import_price·5e-5·(T−t)·c_priority |
×5e-5·(T−t)·c_priority | ~1e-3 – tens | shape, not cost | |
| Ramp leveling | prc_p_ramp = penalty_base·1e-5 |
×1e-5 | 1.6e-3 – 3.0 | shape, not cost |
Net effect
Tier 1 now scales with the largest of any real economic term the model can see — not just the
energy price series — so a large demand-rate tariff or an unusually high p_a inflates
penalty_base (and therefore every tier-1 penalty) right along with it, keeping tier 1 ≫ tier 2
regardless of which real-cost term happens to dominate for a given request. Within tier 1 itself,
prc_p_goal_pen and prc_e_goal_pen are now always equal, so the choice between sacrificing an
unmet charge demand vs. an unmet charge goal is decided by the model's deliberate tie-breakers
(the (1 + (T−t)/T) front-loading factor), not by an accidental side effect of the input time
series' step length.
|
I hope I didn't break anything in regards to parallel PRs, I didn't check against that one yet. My fix above was about making sure precision losses coming from poorly scaled penalties do not affect the optimization results, not really about performance. |
…time step The demand charge is set by the single worst time step, so 1 Wh kept out of the binding step is worth 3600/dt of it. Converting the rate over the horizon instead made penalty_base up to T times too small. The SOC floor penalty is charged per time step, which hides that for a violation lasting the whole horizon, but one confined to a single short step is charged once and wins: 1000 Wh of load in a 36 s step at the end of a 23 h horizon drained the battery to 48952.6 against a floor of 50000. Also gate the term on is_grid_demand_rate_active. p_max_imp and prc_p_exc_imp are independent optional fields, so a rate without an import threshold used to inflate every penalty for a term that is not in the model at all.
…s out The repository already keeps documentation in docs/. Add the two properties the tier table does not show: the SOC penalty is charged per time step, and the min_import_price tie breakers invert under negative prices.
|
Pushed two commits fixing what a review of the last two commits turned up. The demand rate conversion was the wrong quantity. The horizon version looked right because
Table moved into Three new tests, all red before the fix, suite is 39 green. Open point: the new conversion raises |
|
Measured the open point. The larger No stored case carries
Worth noting that 020 with no strategy takes over a minute either way once a demand rate is present, well past the 10 s time limit from #116. Separate issue, but the demand rate path looks like the expensive one. |
Interesting ... did you verify or is this what Claude said? |
|
Re-measured the gain over the stored cases against current
Total is better than the 0.79x in the description, but the shape is different: the gain is entirely in the peak strategies and All 95 solves come back Optimal. Three report a lower objective, up to 0.71 percent, and four report a higher one, up to 8.1 percent, but that is the So nothing regressed economically, and the reported 5 to 8 percent improvements on None of this covers the |
2 participants
The grid peak shaping work (#93, #96, #102) made the peak attenuation strategies noticeably slower: measured against the commit before them,
attenuate_grid_peakscosts about 1.3x median and up to 3x on single cases, while the default strategies are unaffected. Looking for the cause turned up something more general than the peak code.The problem
CBC judges improvements against absolute tolerances of about
1e-7. Prices enter the model in currency per Wh, so the raw objective coefficients land close to that bound. The peak attenuation tie-breakers are the extreme case at roughly1e-9per W, but the cost terms are not far above it either. Improvements that small are indistinguishable from numerical noise, so the solver explores a degenerate search tree and stops on solutions that are not actually optimal.The change
Multiply the assembled objective by a constant before handing it to the solver. This does not change the argmax, it only moves the coefficients into a range the solver can resolve.
get_clean_objective_value()recomputes the reported value from the solution, so the API response stays in the original unit and nothing in the contract changes.The second commit removes a ramp variable that the peak strategies create for the first time step but never constrain. It carries the ramp penalty, is driven to zero, and contributes nothing.
Measurements
All test cases, each run with its stored strategy and with every charging strategy (14 cases x 5 settings), old and new interleaved in one process, min of 7 runs:
noneattenuate_grid_peaksNo solution came out economically worse. Several came out better, the largest being
021-min-pv-use-case, where the previous model returned a solution 0.59% below the optimum.018-high-soc-initialgot slower (0.13 s to 0.23 s withattenuate_grid_peaks); that is branching luck on one instance, not a trend.The gain is not limited to the peak strategies, the default path benefits as much. It does not remove the extra work the peak strategies add to the model (one peak variable,
T - 1ramp variables and3T - 2constraints per leveled side), it makes the solver handle the whole model better.Test
tests/test_objective_scaling.pypins the recovered optimum. It reads the request from the existing021case, solves it without a charging strategy, and asserts a lower bound on the objective, so a future model that finds an even better solution there does not fail it. Onmainthe same request returns1.4243782673against a threshold of1.4327495658.