Skip to content

serviceability-instruction: golden fixtures + CI guard (RFC-26 RF)#4059

Open
juan-malbeclabs wants to merge 3 commits into
mainfrom
feat/rfc26-rf-fixtures
Open

serviceability-instruction: golden fixtures + CI guard (RFC-26 RF)#4059
juan-malbeclabs wants to merge 3 commits into
mainfrom
feat/rfc26-rf-fixtures

Conversation

@juan-malbeclabs

Copy link
Copy Markdown
Contributor

Summary

RFC-26 RF: golden fixtures for the instruction builders + a CI drift guard.

  • Extend the serviceability fixture generator to depend on doublezero-serviceability-instruction and emit, per instruction, ix_<name>.bin (wire bytes = tag + borsh) and ix_<name>.json ({ variant, data_hex, accounts: [{pubkey, is_signer, is_writable}] }) from fixed, deterministic inputs.
  • Representative set covering the trickiest layouts: create_device (variable dz_prefix), delete_device (atomic close), create_link, create_subscribe_user (optional feed), create_user (length-detected), clear_topology / assign_topology_node_segments (batched), set_global_config (config PDA + all 8 pools).
  • make generate-fixtures-check (regenerate + fail on drift, scoped to the emitted .bin/.json) and a fixtures-check job in the rust workflow.

These fixtures capture byte-for-byte the current SDK trailing convention (they will drive Go/Python/TS parity in a later phase).

Testing Verification

  • make generate-fixtures-check passes (regenerated output is byte-identical to the committed fixtures); a hand-edited fixture makes it fail as expected.

Closes #4026. Part of RFC-26 (rfcs/rfc26-rust-instruction-builder-library.md).


PR stack (RFC-26 builder library)

Stacked PRs, merge in order (each based on the previous one's branch):

  1. serviceability-instruction: scaffold pure instruction-builder crate (RFC-26 R0) #4049 — R0 scaffold + exemplars
  2. serviceability-instruction: device domain builders (RFC-26 R1) #4050 — R1 device
  3. serviceability-instruction: link domain builders (RFC-26 R2) #4051 — R2 link
  4. serviceability-instruction: user domain builders (RFC-26 R3) #4052 — R3 user
  5. serviceability-instruction: location/exchange/contributor builders (RFC-26 R4) #4053 — R4 location/exchange/contributor
  6. serviceability-instruction: multicastgroup + allowlist builders (RFC-26 R5) #4054 — R5 multicastgroup + allowlists
  7. serviceability-instruction: tenant + permission builders (RFC-26 R6) #4055 — R6 tenant + permission
  8. serviceability-instruction: topology + feed builders (RFC-26 R7) #4056 — R7 topology + feed
  9. serviceability-instruction: accesspass + resource builders (RFC-26 R8) #4057 — R8 accesspass + resource
  10. serviceability-instruction: globalstate/globalconfig/allowlist/index/migrate builders (RFC-26 R9) #4058 — R9 globalstate/config/allowlist/index/migrate
  11. this PR — RF fixtures + CI guard

R10 (commands/* migration + program-test) is the final PR and will carry the global changelog entry.

@juan-malbeclabs
juan-malbeclabs force-pushed the feat/rfc26-r9-admin-builders branch from e0235a0 to 9240816 Compare July 21, 2026 17:02
@juan-malbeclabs
juan-malbeclabs force-pushed the feat/rfc26-rf-fixtures branch from 70541a5 to 57fe622 Compare July 21, 2026 17:02
@juan-malbeclabs
juan-malbeclabs force-pushed the feat/rfc26-r9-admin-builders branch from 9240816 to 9e23736 Compare July 21, 2026 17:19
@juan-malbeclabs
juan-malbeclabs force-pushed the feat/rfc26-rf-fixtures branch from 57fe622 to 7ab5da2 Compare July 21, 2026 17:19
@juan-malbeclabs
juan-malbeclabs force-pushed the feat/rfc26-r9-admin-builders branch 4 times, most recently from 155cbc7 to d934652 Compare July 23, 2026 23:10
@juan-malbeclabs
juan-malbeclabs force-pushed the feat/rfc26-rf-fixtures branch from 7ab5da2 to 92e417d Compare July 23, 2026 23:10
@juan-malbeclabs
juan-malbeclabs force-pushed the feat/rfc26-r9-admin-builders branch from d934652 to bde6207 Compare July 24, 2026 13:49
Base automatically changed from feat/rfc26-r9-admin-builders to main July 24, 2026 20:06
@juan-malbeclabs
juan-malbeclabs force-pushed the feat/rfc26-rf-fixtures branch from 92e417d to b0a7627 Compare July 24, 2026 20:31
@juan-malbeclabs
juan-malbeclabs requested a review from ben-dz July 24, 2026 20:38

@ben-dz ben-dz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two things worth changing before merge; nothing blocking. (1) The new fixtures-check job in rust.yml is a strict duplicate of the pre-existing sdk-fixture-drift job in sdk.yml — same make check-fixtures, both triggered on every PR/push with no path filters — so the drift check runs twice per PR for zero added coverage, and the new copy runs on an oversized ubuntu-24.04-16c-64gb runner with no timeout-minutes. Delete one, or move the check and delete the other. (2) The set_global_config fixture is built from SetGlobalConfigArgs::default(), emitting an all-zero body that pins only the length — it can't catch field-order/endianness/signedness/Option-encoding regressions, which is the whole point of a wire-format golden; use distinct nonzero values. Low: the fixture JSON records neither program_id nor the input args, so a future parity harness must read the Rust source to reproduce inputs. Fixtures are deterministic (regenerated locally, byte-identical) and otherwise correctly capture the builder output for the representative set.

Comment thread .github/workflows/rust.yml Outdated
components: clippy, rustfmt
- uses: Swatinem/rust-cache@v2
- run: make rust-lint
fixtures-check:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixtures-check job runs make check-fixtures, byte-for-byte the same command as the pre-existing sdk-fixture-drift job in .github/workflows/sdk.yml:55-63. make check-fixtures regenerates all SDK generators (not just serviceability) and git diffs the goldens, so the two jobs are strict duplicates — both trigger on every PR and push to main/hotfix with no path filters, running the drift check twice per PR for zero added coverage. This copy is also worse-provisioned: a paid ubuntu-24.04-16c-64gb runner (the existing one fits on ubuntu-latest) and no timeout-minutes: 15 guard, so a hung download runs to the 6-hour default on 16 cores. Pick one home: delete this job, or move the check into rust.yml and delete sdk-fixture-drift in the same PR. If it stays, drop to ubuntu-latest and add a timeout — the work is four small cargo builds plus a git diff, and rust-cache@v2 won't even warm-cache the generator crates' separate target/ dirs without a workspaces: setting.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — deleted the fixtures-check job. sdk-fixture-drift in sdk.yml already runs make check-fixtures on ubuntu-latest with timeout-minutes: 15, so this was a strict duplicate with no added coverage; keeping the pre-existing, better-provisioned one. (ebe5e54)


// set_global_config (3): the config PDA plus all eight resource pools.
let set_global_config =
dzi::globalconfig::set_global_config(&program_id, &payer, SetGlobalConfigArgs::default());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SetGlobalConfigArgs::default() produces an all-zero body — the emitted data_hex is the tag byte plus 29 zero bytes. A wire-format golden should pin field order, endianness, signedness, and Option Some/None encoding, but an all-zero payload catches none of these (a None option byte is indistinguishable from a zeroed field; reordering the NetworkV4 blocks or swapping the two u32 ASNs is invisible). This is the builder the PR highlights as exercising all eight resource pools, yet its fixture is the least discriminating. Build the args with distinct nonzero values per field, as generate_global_config already does for the account fixture.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — set_global_config now builds SetGlobalConfigArgs with distinct nonzero values per field (mirroring generate_global_config): local_asn: 65000, remote_asn: 65001, the three tunnel/multicast NetworkV4 blocks, next_bgp_community: Some(10042), and multicast_publisher_block. The body grew from 30 to 32 bytes and data_hex is now 03e8fd0000e9fd0000...013a27..., so the golden pins field order, endianness, both u32 ASNs, and the Some-encoding of the optional community rather than only the length. (ebe5e54)

}

#[derive(Serialize)]
struct IxFixtureMeta {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IxFixtureMeta records variant/data_hex/accounts but not instruction.program_id, and the fixed program id (pubkey_from_byte(255)) plus per-builder arg values live only in this Rust source. A later Go/Python/TS parity harness must read the generator to reproduce inputs, and a program-id mismatch surfaces only as opaque PDA-key diffs. Add a program_id field (and optionally an args/fields block, as the account fixtures carry) so the goldens are self-describing. Cheaper to fix now, before the parity PR locks in the fixture shape.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added — IxFixtureMeta now records program_id (read straight off ix.program_id), so every ix_*.json golden is self-describing on the program id and a parity harness no longer has to read the generator to catch a mismatch. Left the per-builder arg block out for now since no consumer reads these yet (the parity harness is still future work); happy to add a fields block in the parity PR if it's useful there. (ebe5e54)

…+ CI guard (RFC-26 RF)

Extend the serviceability fixture generator to depend on the builder crate and
emit, per instruction, ix_<name>.bin (tag + borsh wire bytes) and ix_<name>.json
({ variant, data_hex, accounts:[{pubkey,is_signer,is_writable}] }) with fixed,
deterministic inputs. Covers a representative set exercising the trickiest
layouts: create_device (variable dz_prefix), delete_device (atomic close),
create_link, create_subscribe_user (optional feed), create_user (length-detected),
clear_topology / assign_topology_node_segments (batched), set_global_config.

Add `make generate-fixtures-check` (regenerate + fail on drift) and a
`fixtures-check` job in the rust CI workflow.

Refs #4026, RFC-26.
…nerator feed borrow

The clear_topology builder now marks the topology account writable (the
processor decrements its reference_count), so regenerate ix_clear_topology.json.
Also fix the generator to pass feed as Option<&Pubkey> (Some(&d)), matching the
current create_subscribe_user signature.
…iscriminating global_config fixture, self-describing goldens

- Delete duplicate fixtures-check job from rust.yml; sdk-fixture-drift in
  sdk.yml already runs make check-fixtures (on ubuntu-latest with a timeout).
- Build set_global_config args from distinct nonzero values instead of
  ::default(), so the golden pins field order, endianness, both u32 ASNs, and
  the Some-encoding of next_bgp_community rather than only the body length.
- Record instruction.program_id in the ix_*.json goldens so a future parity
  harness need not read the generator to reproduce inputs.
@juan-malbeclabs
juan-malbeclabs force-pushed the feat/rfc26-rf-fixtures branch from b0a7627 to ebe5e54 Compare July 25, 2026 15:05
@juan-malbeclabs
juan-malbeclabs requested a review from ben-dz July 25, 2026 15:08

@ben-dz ben-dz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three findings from the prior review are addressed in ebe5e54 and verified in the current code: the duplicate fixtures-check CI job is deleted from rust.yml (sdk.yml's better-provisioned sdk-fixture-drift retained); set_global_config now builds SetGlobalConfigArgs with distinct nonzero values so the golden pins field order, both u32 ASNs, and the optional community's Some-encoding instead of an all-zero body; and IxFixtureMeta records program_id in every ix_*.json. The passing sdk-fixture-drift job confirms the goldens regenerate byte-identically. lgtm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants