Skip to content

Fix CI: correct H∞ bound in the reduction tests and de-flake the scaling-invariance tests - #154

Merged
baggepinnen merged 1 commit into
masterfrom
fix-reduction-test-hinfnorm-bound
Aug 24, 2026
Merged

Fix CI: correct H∞ bound in the reduction tests and de-flake the scaling-invariance tests#154
baggepinnen merged 1 commit into
masterfrom
fix-reduction-test-hinfnorm-bound

Conversation

@baggepinnen

Copy link
Copy Markdown
Member

CI on master is red on a single assertion in test/test_reduction.jl:

Expression: norm(sysi * (sys - sysr), Inf) < 0.4
 Evaluated: 0.45825876482743627 < 0.4

The bad bound

This is not a regression in this package — #153 only touched Project.toml. It is a dependency bump: the last passing run resolved ControlSystemsBase v1.20.4, the failing one resolved v1.21.0, which reworked _infnorm_two_steps_ct, the kernel behind hinfnorm / norm(·, Inf).

The test system is hardcoded, so the failure is fully deterministic. Pinning ControlSystemsBase back to 1.20.4 in an otherwise identical environment restores the old value:

ControlSystemsBase norm(G, Inf) reported ω
1.20.4 0.31417 Inf
1.21.0 0.45826 139.44

The brute-force peak gain of G = sysi*(sys - sysr) over a 200k-point grid from 1e-4 to 1e4 rad/s is 0.458258, and hinfnorm2 (via DescriptorSystems, unchanged across the bump) gives 0.458196 on both versions. So 1.21.0 is right and the old value was a bad under-estimate: G has a state dimension of 23 with near-cancelled poles from fudge_inv sitting close to the imaginary axis, and on that system the old algorithm got stuck at the lower bound opnorm(D) — hence the reported ω = Inf.

The bound of 0.4 was therefore calibrated against a wrong number rather than against the true norm, and is raised to 0.5. The reduction itself is unchanged across the bump: norm(sys-sysr, Inf), the DC-gain residual property and the comparison against baltrunc all give identical values on both versions.

The flaky scaling-invariance tests

Separately, the three scaling-invariance tests compared hinfnorm2(minreal(sys1 - sys2))[1] against an absolute tolerance. sys_siso = ssrand(1,1,9,proper=true) is unseeded, and for roughly 5% of draws minreal leaves a near-cancelled pole close enough to the imaginary axis that ghinfnorm returns Inf and the test fails (2/50 and 2/30 in two measurement loops). This is unrelated to the ControlSystemsBase change — it happened to pass in the failing run — but it would have bitten eventually.

They now compare the frequency responses on a grid, which is what the section comments already claimed they did. Worst relative difference over 200 random draws:

check worst over 200 draws
baltrunc2 5.0e-13
baltrunc_coprime 1.4e-11
baltrunc_unstab 5.0e-13

so the 1e-8 relative tolerance keeps a wide margin while being three orders of magnitude tighter than the old absolute 1e-5. sys_siso is left unseeded so the tests keep sampling new systems.

Verification

  • A daemon session that deterministically reproduces the unlucky draw returning Inf now passes it, 3/3 runs.
  • test_reduction.jl in 4 fresh processes: 45 passed, 4 broken, exit 0.
  • Full suite against ControlSystemsBase 1.21.0: 1414 passed, 6 broken, 0 failed (the failing CI run had 1413 passed + 1 failed).

🤖 Generated with Claude Code

ControlSystemsBase 1.21.0 reworked the `hinfnorm` kernel
(`_infnorm_two_steps_ct`), and the frequency-weighted-reduction test at
test/test_reduction.jl:65 started failing on master as a result. The old
bound was calibrated against a wrong value, not against the true norm:

    CSB 1.20.4:  norm(G, Inf) = 0.31417  at ω = Inf
    CSB 1.21.0:  norm(G, Inf) = 0.45826  at ω = 139.44

`G = sysi*(sys - sysr)` has a state dimension of 23 with near-cancelled
poles from `fudge_inv` close to the imaginary axis. On that system the old
algorithm got stuck at the lower bound `opnorm(D)`, hence the ω = Inf. The
peak gain over a 200k-point grid from 1e-4 to 1e4 rad/s is 0.458258 and
`hinfnorm2` (via DescriptorSystems, unchanged across the bump) gives
0.458196, so 1.21.0 is the correct value and the bound is raised to 0.5.
The reduction itself is unaffected: `norm(sys-sysr, Inf)`, the DC-gain
residual property and the comparison against `baltrunc` are identical on
both versions.

The three scaling-invariance tests compared `hinfnorm2(minreal(sys1-sys2))`
against an absolute tolerance. `minreal` occasionally leaves a near-cancelled
pole close to the imaginary axis, in which case `ghinfnorm` returns Inf and
the test fails; measured at roughly 5% of the random `ssrand(1,1,9)` draws.
They now compare the frequency responses on a grid, which is what the
section comments already said they did. Worst relative difference over 200
draws: 5.0e-13 (baltrunc2), 1.4e-11 (baltrunc_coprime), 5.0e-13
(baltrunc_unstab), so the 1e-8 tolerance keeps a wide margin while being
three orders tighter than the old absolute 1e-5.

Full test suite: 1414 passed, 6 broken, 0 failed against CSB 1.21.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.90%. Comparing base (5fafc2e) to head (fa381b5).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #154   +/-   ##
=======================================
  Coverage   91.90%   91.90%           
=======================================
  Files          20       20           
  Lines        3065     3065           
=======================================
  Hits         2817     2817           
  Misses        248      248           
Flag Coverage Δ
unittests 91.90% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@baggepinnen
baggepinnen merged commit 162e9b7 into master Aug 24, 2026
2 checks passed
@baggepinnen
baggepinnen deleted the fix-reduction-test-hinfnorm-bound branch August 24, 2026 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants