serviceability-instruction: golden fixtures + CI guard (RFC-26 RF)#4059
serviceability-instruction: golden fixtures + CI guard (RFC-26 RF)#4059juan-malbeclabs wants to merge 3 commits into
Conversation
e0235a0 to
9240816
Compare
70541a5 to
57fe622
Compare
9240816 to
9e23736
Compare
57fe622 to
7ab5da2
Compare
155cbc7 to
d934652
Compare
7ab5da2 to
92e417d
Compare
d934652 to
bde6207
Compare
92e417d to
b0a7627
Compare
ben-dz
left a comment
There was a problem hiding this comment.
Two things worth changing before merge; nothing blocking. (1) The new fixtures-check job in rust.yml is a strict duplicate of the pre-existing sdk-fixture-drift job in sdk.yml — same make check-fixtures, both triggered on every PR/push with no path filters — so the drift check runs twice per PR for zero added coverage, and the new copy runs on an oversized ubuntu-24.04-16c-64gb runner with no timeout-minutes. Delete one, or move the check and delete the other. (2) The set_global_config fixture is built from SetGlobalConfigArgs::default(), emitting an all-zero body that pins only the length — it can't catch field-order/endianness/signedness/Option-encoding regressions, which is the whole point of a wire-format golden; use distinct nonzero values. Low: the fixture JSON records neither program_id nor the input args, so a future parity harness must read the Rust source to reproduce inputs. Fixtures are deterministic (regenerated locally, byte-identical) and otherwise correctly capture the builder output for the representative set.
| components: clippy, rustfmt | ||
| - uses: Swatinem/rust-cache@v2 | ||
| - run: make rust-lint | ||
| fixtures-check: |
There was a problem hiding this comment.
This fixtures-check job runs make check-fixtures, byte-for-byte the same command as the pre-existing sdk-fixture-drift job in .github/workflows/sdk.yml:55-63. make check-fixtures regenerates all SDK generators (not just serviceability) and git diffs the goldens, so the two jobs are strict duplicates — both trigger on every PR and push to main/hotfix with no path filters, running the drift check twice per PR for zero added coverage. This copy is also worse-provisioned: a paid ubuntu-24.04-16c-64gb runner (the existing one fits on ubuntu-latest) and no timeout-minutes: 15 guard, so a hung download runs to the 6-hour default on 16 cores. Pick one home: delete this job, or move the check into rust.yml and delete sdk-fixture-drift in the same PR. If it stays, drop to ubuntu-latest and add a timeout — the work is four small cargo builds plus a git diff, and rust-cache@v2 won't even warm-cache the generator crates' separate target/ dirs without a workspaces: setting.
There was a problem hiding this comment.
Done — deleted the fixtures-check job. sdk-fixture-drift in sdk.yml already runs make check-fixtures on ubuntu-latest with timeout-minutes: 15, so this was a strict duplicate with no added coverage; keeping the pre-existing, better-provisioned one. (ebe5e54)
|
|
||
| // set_global_config (3): the config PDA plus all eight resource pools. | ||
| let set_global_config = | ||
| dzi::globalconfig::set_global_config(&program_id, &payer, SetGlobalConfigArgs::default()); |
There was a problem hiding this comment.
SetGlobalConfigArgs::default() produces an all-zero body — the emitted data_hex is the tag byte plus 29 zero bytes. A wire-format golden should pin field order, endianness, signedness, and Option Some/None encoding, but an all-zero payload catches none of these (a None option byte is indistinguishable from a zeroed field; reordering the NetworkV4 blocks or swapping the two u32 ASNs is invisible). This is the builder the PR highlights as exercising all eight resource pools, yet its fixture is the least discriminating. Build the args with distinct nonzero values per field, as generate_global_config already does for the account fixture.
There was a problem hiding this comment.
Fixed — set_global_config now builds SetGlobalConfigArgs with distinct nonzero values per field (mirroring generate_global_config): local_asn: 65000, remote_asn: 65001, the three tunnel/multicast NetworkV4 blocks, next_bgp_community: Some(10042), and multicast_publisher_block. The body grew from 30 to 32 bytes and data_hex is now 03e8fd0000e9fd0000...013a27..., so the golden pins field order, endianness, both u32 ASNs, and the Some-encoding of the optional community rather than only the length. (ebe5e54)
| } | ||
|
|
||
| #[derive(Serialize)] | ||
| struct IxFixtureMeta { |
There was a problem hiding this comment.
IxFixtureMeta records variant/data_hex/accounts but not instruction.program_id, and the fixed program id (pubkey_from_byte(255)) plus per-builder arg values live only in this Rust source. A later Go/Python/TS parity harness must read the generator to reproduce inputs, and a program-id mismatch surfaces only as opaque PDA-key diffs. Add a program_id field (and optionally an args/fields block, as the account fixtures carry) so the goldens are self-describing. Cheaper to fix now, before the parity PR locks in the fixture shape.
There was a problem hiding this comment.
Added — IxFixtureMeta now records program_id (read straight off ix.program_id), so every ix_*.json golden is self-describing on the program id and a parity harness no longer has to read the generator to catch a mismatch. Left the per-builder arg block out for now since no consumer reads these yet (the parity harness is still future work); happy to add a fields block in the parity PR if it's useful there. (ebe5e54)
…+ CI guard (RFC-26 RF)
Extend the serviceability fixture generator to depend on the builder crate and
emit, per instruction, ix_<name>.bin (tag + borsh wire bytes) and ix_<name>.json
({ variant, data_hex, accounts:[{pubkey,is_signer,is_writable}] }) with fixed,
deterministic inputs. Covers a representative set exercising the trickiest
layouts: create_device (variable dz_prefix), delete_device (atomic close),
create_link, create_subscribe_user (optional feed), create_user (length-detected),
clear_topology / assign_topology_node_segments (batched), set_global_config.
Add `make generate-fixtures-check` (regenerate + fail on drift) and a
`fixtures-check` job in the rust CI workflow.
Refs #4026, RFC-26.
…nerator feed borrow The clear_topology builder now marks the topology account writable (the processor decrements its reference_count), so regenerate ix_clear_topology.json. Also fix the generator to pass feed as Option<&Pubkey> (Some(&d)), matching the current create_subscribe_user signature.
…iscriminating global_config fixture, self-describing goldens - Delete duplicate fixtures-check job from rust.yml; sdk-fixture-drift in sdk.yml already runs make check-fixtures (on ubuntu-latest with a timeout). - Build set_global_config args from distinct nonzero values instead of ::default(), so the golden pins field order, endianness, both u32 ASNs, and the Some-encoding of next_bgp_community rather than only the body length. - Record instruction.program_id in the ix_*.json goldens so a future parity harness need not read the generator to reproduce inputs.
b0a7627 to
ebe5e54
Compare
ben-dz
left a comment
There was a problem hiding this comment.
All three findings from the prior review are addressed in ebe5e54 and verified in the current code: the duplicate fixtures-check CI job is deleted from rust.yml (sdk.yml's better-provisioned sdk-fixture-drift retained); set_global_config now builds SetGlobalConfigArgs with distinct nonzero values so the golden pins field order, both u32 ASNs, and the optional community's Some-encoding instead of an all-zero body; and IxFixtureMeta records program_id in every ix_*.json. The passing sdk-fixture-drift job confirms the goldens regenerate byte-identically. lgtm.
Summary
RFC-26 RF: golden fixtures for the instruction builders + a CI drift guard.
doublezero-serviceability-instructionand emit, per instruction,ix_<name>.bin(wire bytes = tag + borsh) andix_<name>.json({ variant, data_hex, accounts: [{pubkey, is_signer, is_writable}] }) from fixed, deterministic inputs.create_device(variable dz_prefix),delete_device(atomic close),create_link,create_subscribe_user(optional feed),create_user(length-detected),clear_topology/assign_topology_node_segments(batched),set_global_config(config PDA + all 8 pools).make generate-fixtures-check(regenerate + fail on drift, scoped to the emitted.bin/.json) and afixtures-checkjob in therustworkflow.These fixtures capture byte-for-byte the current SDK trailing convention (they will drive Go/Python/TS parity in a later phase).
Testing Verification
make generate-fixtures-checkpasses (regenerated output is byte-identical to the committed fixtures); a hand-edited fixture makes it fail as expected.Closes #4026. Part of RFC-26 (rfcs/rfc26-rust-instruction-builder-library.md).
PR stack (RFC-26 builder library)
Stacked PRs, merge in order (each based on the previous one's branch):
R10 (commands/* migration + program-test) is the final PR and will carry the global changelog entry.