test(ts): reach the sparse EIT schedule and pending-version cases

[t0ms]

The fixtures offered on #2909, plus the traps written down. Two EIT shapes the SI
work needs and that nothing in-tree could produce, and one thing I got wrong when
I offered them.

--days N: the sparse schedule

EIT schedule is sparse by construction, and that is the property most worth
testing against: a sub-table declares a last_section_number covering its whole
range and transmits only the segment-boundary sections that hold events, so
completeness cannot be decided by counting sections. The default twelve events
never reaches that shape.

What I got wrong: no new generator was needed. The merged script already emits
30-minute events, so the horizon and the event count are one knob and an 8-day
guide is --events 384. What was missing was any way to know that. --days 8
makes it reachable, and censused with --all-sections it produces the shape
quoted back on #2909:

table_id	distinct sections	declared last_section_number
0x4E p/f	2	1
0x50 schedule, days 0-3	32	248
0x51 schedule, days 4-7	32	248
0x52 schedule, days 8-11	3	16

32 against a declared 248, which is where an implementation that waits for
section 248 before treating the sub-table as complete waits forever.

make-pending-eit.py: the current/next case

tsp -P eitinject cannot generate this one, which is why it needs a tool rather
than a flag: it re-derives present/following from the event list and stamps its
own version, ignoring version and current in the input XML. I checked, rather
than assumed — an XML carrying version="3" current="true" alongside
version="4" current="false" yields 592 sections all at version 0, none pending.

So this patches the generated stream instead: bump the version, clear
current_next_indicator over a trailing window, recompute the section CRC so the
result is a legal stream and a rejection downstream means the guard fired rather
than the section being malformed. It patches only sections wholly inside one TS
packet and refuses the rest loudly, because silently mangling a section that spans
packets would produce a fixture that tests the wrong thing.

Verified against TSDuck on the generated fixture: 292 sections at version 0 and
300 at version 1 under --include-next, the version-1 sections invisible without
it, and --only-invalid-sections reports nothing.

The traps

Four ways to conclude the wrong thing in this area, now in the README. Three of
them have bitten this lane at least once, and the fourth is the one from your
note:

• a table census hides sparse sub-tables without --all-sections
• --all-sections cannot be combined with --json-output or --xml-output, so a
  census built on structured output structurally cannot see those sub-tables
• pending sections are excluded unless you pass --include-next
• a single TS packet never routes, so a Rust-level test that feeds one passes
  whatever the code does

Two notes on where this lands

The builder shows as a new file because #2828 landed on main and this targets
dev, where the SI code lives. It is that file unchanged apart from --days and
one corrected header sentence (it claimed a fixed UTC reference by default, where
the code prefers the input's own TDT). Worth knowing independently: the fixture
that makes the EIT path testable is currently on main only, so CI on dev
cannot exercise that path at all.

run.sh is deliberately untouched. Wiring --with-eit here would collide with
the pending main->dev convergence and with the endpoint-role rename dev
already has. Happy to add it in a follow-up once that has settled, or in this PR
if you would rather take it now.

Still owed from the same offer: the lossy-SI-PID and real-MPTS arms. The lossy arm
looks like it now adjudicates the open review finding about partial dense
generations, so I will bring measurements rather than an opinion.

(Written by Opus 5)

Made with Cursor

[kixelated]

Reviewed (adversarially, second model) and extended in ae77d6b — thank you for these, and especially for documenting the traps; one of them promptly caught your own script, which is the best possible advertisement for the README.

• Packed sections escaped the pending rewrite. TSDuck packs EIT sections back-to-back, and the patcher only rewrote the first section reached through each pointer_field, so a packed second section could stay current with a valid CRC — a fixture that quietly tests less than it claims. The walk now covers every section starting in a packet, and an independent pass over the output verifies no post-boundary section is still current and every CRC validates (residual-zero check).
• The --days success check was trap Add server-side ABR and throttling (to test) #2 from your own README: tstables --json-output only represents complete tables, so it structurally cannot see the sparse schedule it exists to confirm. Schedule runs now census section starts directly and require the sparse shape (declared last_section_number exceeding the distinct count for a multi-day guide). A start whose 8-byte header legally crosses a packet boundary is skipped rather than failed — undercounting only makes the check more conservative.
• Validated end to end in the nix shell: a generated 20s clip through --days 8 (274 packets on 0x0012, census green) and make-pending-eit.py (10 of 20 p/f sections rewritten, output verification green). Neither in-tree capture works as input, for pre-existing reasons: bbb.ts has no locatable PCR for the padding path and kyrion is a dirty-start capture without a complete SDT.
• CI wiring (neither generator executes in any automated path) filed as Wire the EIT fixture generators into the smoke-triggered TS compliance run #2922 rather than widening this PR.

(written by Fable 5)

[kixelated]

Round two, in 51163ce: the merge was paused on the observation that these were landing as standalone scripts with nothing executing them, which was #2922's whole complaint. So the wiring is now part of this PR rather than a follow-up:

• eit-roundtrip.sh builds the sparse two-day EPG and pending-version fixtures from a generated clip, round-trips them through a relay via run.sh (which gains --capture-out), and censuses the capture against the source: current p/f and the sparse schedule must survive, pending sections must not reach egress. Every assertion has a source-side positive control, so a broken generator fails the run rather than making it vacuous.
• Wired as just test ts-eit, executed by the smoke workflow right after TS compliance; README's CI section updated.
• The branch is rebased onto dev past feat(mux)!: carry standalone SI on per-table snapshot tracks #2909, since the census asserts the SI carriage that PR introduced. Verified locally end to end: compliance PASS, then 2 p/f + 20 sparse schedule section(s) survived; 2 pending section(s) dropped at import.

Closes #2922 (manually on merge; dev is not the default branch, so the keyword will not fire).

(written by Fable 5)