test: Assert TPP Programmed tracks generation bumps - #336
Conversation
Add a Chainsaw e2e for NSO#266: after a TrafficProtectionPolicy spec change, skew must be queryable via Programmed lag, then Programmed must catch up to the new generation. Silent unbounded lag fails the test.
Chainsaw invokes scripts with /usr/bin/sh (dash on Ubuntu). Bash here-strings caused an immediate Syntax error: redirection unexpected and failed the new programmed-generation test in CI. Key features: - replace <<< / seq with set -- and while loops - keep skew-then-converge assertions unchanged
|
CI failure was |
Chainsaw scripts do not inherit the test namespace as kubectl's default. Getting gen-waf without -n hit default and failed under set -u with "parameter not set" before the skew probes ran. Key features: - pass -n "$NAMESPACE" on get/patch/catch dumps
|
Next failure was namespace scoping: |
Single-cluster e2e converges Programmed before the first poll, so requiring an intermediate obs<gen window fails every run. Assert that Programmed tracks the post-edit generation instead; lag under real multi-edge delay is covered by the apply-lag SLO alert. Key features: - wait for generation bump + Programmed observedGeneration catch-up - fix catch logs to the e2e extension-server Deployment
|
Root cause of the latest failure: in single-cluster e2e, Programmed was already Dropped the flaky intermediate-skew assert (forcing it would mean scaling the shared extension server and breaking parallel tests). The test now gates the real contract: after a gen bump, Programmed must report |
2 participants
Summary
NSO#266’s regression gap was that a TPP spec change could lag on the edge with no test asserting the lag is observable. Create-path already checks
Programmed=True; this adds a generation-bump scenario: after a real edit,Programmedmust lag (FalseorobservedGeneration < generation) so skew is queryable, then catch up to the new generation — an unbounded silent tail fails.Wire-level inverted-paranoia remediation stays in
trafficprotectionpolicy-neutralize-inverted; this test owns the Programmed/generation contract from ecv’s acceptance checklist on #266.Test plan
task test-infra:test-e2e -- ./test/e2e/trafficprotectionpolicy-programmed-generationagainst the two-cluster env (task test-infra:up+ WAF dataplane)./test/e2eRelated to #266