Fix CI: correct H∞ bound in the reduction tests and de-flake the scaling-invariance tests - #154
Merged
Merged
Conversation
ControlSystemsBase 1.21.0 reworked the `hinfnorm` kernel
(`_infnorm_two_steps_ct`), and the frequency-weighted-reduction test at
test/test_reduction.jl:65 started failing on master as a result. The old
bound was calibrated against a wrong value, not against the true norm:
CSB 1.20.4: norm(G, Inf) = 0.31417 at ω = Inf
CSB 1.21.0: norm(G, Inf) = 0.45826 at ω = 139.44
`G = sysi*(sys - sysr)` has a state dimension of 23 with near-cancelled
poles from `fudge_inv` close to the imaginary axis. On that system the old
algorithm got stuck at the lower bound `opnorm(D)`, hence the ω = Inf. The
peak gain over a 200k-point grid from 1e-4 to 1e4 rad/s is 0.458258 and
`hinfnorm2` (via DescriptorSystems, unchanged across the bump) gives
0.458196, so 1.21.0 is the correct value and the bound is raised to 0.5.
The reduction itself is unaffected: `norm(sys-sysr, Inf)`, the DC-gain
residual property and the comparison against `baltrunc` are identical on
both versions.
The three scaling-invariance tests compared `hinfnorm2(minreal(sys1-sys2))`
against an absolute tolerance. `minreal` occasionally leaves a near-cancelled
pole close to the imaginary axis, in which case `ghinfnorm` returns Inf and
the test fails; measured at roughly 5% of the random `ssrand(1,1,9)` draws.
They now compare the frequency responses on a grid, which is what the
section comments already said they did. Worst relative difference over 200
draws: 5.0e-13 (baltrunc2), 1.4e-11 (baltrunc_coprime), 5.0e-13
(baltrunc_unstab), so the 1e-8 tolerance keeps a wide margin while being
three orders tighter than the old absolute 1e-5.
Full test suite: 1414 passed, 6 broken, 0 failed against CSB 1.21.0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #154 +/- ##
=======================================
Coverage 91.90% 91.90%
=======================================
Files 20 20
Lines 3065 3065
=======================================
Hits 2817 2817
Misses 248 248
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI on master is red on a single assertion in
test/test_reduction.jl:The bad bound
This is not a regression in this package — #153 only touched
Project.toml. It is a dependency bump: the last passing run resolvedControlSystemsBase v1.20.4, the failing one resolvedv1.21.0, which reworked_infnorm_two_steps_ct, the kernel behindhinfnorm/norm(·, Inf).The test system is hardcoded, so the failure is fully deterministic. Pinning ControlSystemsBase back to 1.20.4 in an otherwise identical environment restores the old value:
norm(G, Inf)InfThe brute-force peak gain of
G = sysi*(sys - sysr)over a 200k-point grid from 1e-4 to 1e4 rad/s is 0.458258, andhinfnorm2(via DescriptorSystems, unchanged across the bump) gives 0.458196 on both versions. So 1.21.0 is right and the old value was a bad under-estimate:Ghas a state dimension of 23 with near-cancelled poles fromfudge_invsitting close to the imaginary axis, and on that system the old algorithm got stuck at the lower boundopnorm(D)— hence the reported ω = Inf.The bound of 0.4 was therefore calibrated against a wrong number rather than against the true norm, and is raised to 0.5. The reduction itself is unchanged across the bump:
norm(sys-sysr, Inf), the DC-gain residual property and the comparison againstbaltruncall give identical values on both versions.The flaky scaling-invariance tests
Separately, the three scaling-invariance tests compared
hinfnorm2(minreal(sys1 - sys2))[1]against an absolute tolerance.sys_siso = ssrand(1,1,9,proper=true)is unseeded, and for roughly 5% of drawsminrealleaves a near-cancelled pole close enough to the imaginary axis thatghinfnormreturnsInfand the test fails (2/50 and 2/30 in two measurement loops). This is unrelated to the ControlSystemsBase change — it happened to pass in the failing run — but it would have bitten eventually.They now compare the frequency responses on a grid, which is what the section comments already claimed they did. Worst relative difference over 200 random draws:
baltrunc2baltrunc_coprimebaltrunc_unstabso the
1e-8relative tolerance keeps a wide margin while being three orders of magnitude tighter than the old absolute1e-5.sys_sisois left unseeded so the tests keep sampling new systems.Verification
Infnow passes it, 3/3 runs.test_reduction.jlin 4 fresh processes: 45 passed, 4 broken, exit 0.🤖 Generated with Claude Code