refactor: Split node roles into independent capability labels - #441
Merged
Conversation
Every DaemonSet's node-affinity used to key off one label, galactic.datumapis.com/node, mixing genuinely exclusive primary roles (edge vs. route-reflector vs. gateway) with capabilities that can and do coexist on the same node. That conflation was already live and wrong: fabric-router enumerated roles by hand and silently never included nat66, so NAT66 shard nodes never got the underlay BGP session their own SID advertisement depends on; nat66 and the route-reflector role each required their own node enum value, which made it structurally impossible for a node to be both that role and the tenant-serving role at once -- the one configuration those two actually use in every real deployment. This replaces the single overloaded label with independent labels per concern, matching how galactic.datumapis.com/fabric already worked: - galactic.datumapis.com/node is now a two-value enum (compute, edge) for the roles that really are mutually exclusive. "edge" now means the actual network-edge/gateway role (galactic-gateway) rather than the old per-node tenant role, resolving a long-standing collision with the pre-existing "edge XDP NAT+LB gateway" terminology used everywhere else in this repo; the tenant/compute role that used to be called "edge" is now "compute". - galactic.datumapis.com/galactic-route-reflector and galactic.datumapis.com/fabric are independent boolean flags, so a route-reflector node can also be a compute node, and any role that needs the underlay just adds the fabric label rather than needing every consumer's affinity list kept in sync. - nat66 is no longer its own node value at all -- every compute node now runs galactic-nat66 unconditionally, since that's the only configuration ever used. - galactic.datumapis.com/fabric-route-reflector is reserved for a future fabric-router-rr and documented as not yet implemented. config/galactic-router's overlay directories are renamed to match (edge -> default, tenant-control -> rr) and flattened to mirror plain Kustomize base/overlay conventions instead of bespoke naming. The containerlab lab's node labels, taints, and per-site fabric-router narrowing overlays are updated to the new scheme, including two overlays whose stale values: lists this same work caught live via a real crash-loop (fabric-router/base and fabric-control/iad, both still referencing retired label values) -- fixing that also surfaced a latent deploy-fabric.sh bug where a redeploy silently kept serving stale copied manifests, now fixed with the same rm -rf guard the other deploy scripts already use. See docs/node-labels.md for the full label reference and the reasoning behind which concerns got their own label versus stayed on the shared enum. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
scotwells
approved these changes
Aug 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Galactic used one node label to decide which DaemonSets ran where, mixing genuinely exclusive roles with capabilities that needed to coexist — this silently broke real configurations, like a NAT66 shard node that also needed the tenant role. Node roles now use a two-value label for the roles that are actually mutually exclusive (
compute,edge), plus independent flags for capabilities that aren't (fabric, route reflector), matching the fabric label's existing pattern.edgealso changes meaning here — it now means the network-edge gateway role instead of the old per-node tenant role, resolving a naming collision that already existed elsewhere in this repo. The containerlab lab, including two live-caught crash-loop bugs from stale label references, is updated to match.Note
task verify's cross-site ping check (verify:ns10/verify:ns20) has a separate, pre-existing regression unrelated to this change (a hex/base62 VPC-identifier mismatch introduced by an earlier commit, tracked separately). It isn't expected to pass on this branch either.Test plan
kubectl kustomizebuilds cleanly forgalactic-router,galactic-cni,galactic-gateway,galactic-nat66, andfabric-routerwith the new labels