ateomnet: enable IPv6 forwarding in worker pod netns - #979
ateomnet: enable IPv6 forwarding in worker pod netns#979SURAJ KUMAR (krsnaSuraj) wants to merge 4 commits into
Conversation
EnableIPv4Forwarding now also writes /proc/sys/net/ipv6/conf/all/forwarding so actor IPv6 traffic (including DNS queries) is routed between the actor veth and pod eth0 instead of being dropped by ip6_forward() on dual-stack / IPv6-only clusters. Factor the sysctl write into writeSysctlIfUnset preserving the original read-only remount/restore behavior, and add unit coverage for its fast paths. Fixes: agent-substrate#945
f74fea7 to
330d38e
Compare
IPv6 sysctls are absent on kernels with IPv6 disabled (e.g. some containers set net.ipv6.conf.* only when IPv6 is enabled). Treat a missing path as 'nothing to enable' instead of forcing a remount and failing, matching the documented behavior.
|
Any chance of a review on this? It's now part of the IPv6 e2e chain — #1059 (the signal-only draft) rides on the worker-pod forwarding + CI is green ( |
|
Two asks before merge. The premise checks out first: writing 1. Commit 2's branch is never executed. A coverage run puts the 2. Illustrative diff — builds, vet clean, passesdiff --git a/internal/ateomnet/net.go b/internal/ateomnet/net.go
index a2a95017..1f0fbc92 100644
--- a/internal/ateomnet/net.go
+++ b/internal/ateomnet/net.go
@@ -192,11 +192,11 @@ func PodIPv4() (net.IP, error) {
return nil, fmt.Errorf("pod eth0 has no IPv4 address")
}
-// EnableIPv4Forwarding enables IPv4 forwarding in the current network namespace.
-// It also enables IPv6 forwarding so actor IPv6 traffic (including DNS queries
-// on IPv6-capable clusters) is routed between the veth and eth0 instead of
-// being dropped by ip6_forward().
-func EnableIPv4Forwarding() error {
+// EnableForwarding enables IPv4 and IPv6 forwarding in the current network
+// namespace, so actor traffic (including DNS queries on IPv6-capable clusters)
+// is routed between the veth and eth0 instead of being dropped by ip_forward()
+// or ip6_forward().
+func EnableForwarding() error {
// Forwarding is required because actor packets now enter the worker pod via
// the host-side veth and then leave through the pod's eth0. Without this, the
// kernel would not route traffic between those interfaces even though both
@@ -595,7 +595,7 @@ func SetupActorNetwork(ctx context.Context, cfg NetworkConfig) (retErr error) {
return fmt.Errorf("while configuring actor veth in interior netns: %w", err)
}
- if err := EnableIPv4Forwarding(); err != nil {
+ if err := EnableForwarding(); err != nil {
return err
}
if err := InstallActorNftablesRules(cfg.EgressRedirectPort); err != nil {
diff --git a/internal/ateomnet/write_sysctl_test.go b/internal/ateomnet/write_sysctl_test.go
index b4cb1db1..d687932d 100644
--- a/internal/ateomnet/write_sysctl_test.go
+++ b/internal/ateomnet/write_sysctl_test.go
@@ -65,6 +65,21 @@ func TestWriteSysctlIfUnset(t *testing.T) {
}
})
+ t.Run("missing_path_is_noop", func(t *testing.T) {
+ // A node under a directory that does not exist stands in for
+ // /proc/sys/net/ipv6/... on a kernel with IPv6 disabled. The other
+ // subtests' paths can be created, so they return at the os.WriteFile
+ // fast path; this is the only one that reaches the os.Stat branch,
+ // which is what procfs always does in production.
+ p := filepath.Join(dir, "no-such-dir", "forwarding")
+ if err := writeSysctlIfUnset(p); err != nil {
+ t.Fatalf("writeSysctlIfUnset on a missing path: %v", err)
+ }
+ if _, err := os.Stat(p); !os.IsNotExist(err) {
+ t.Fatalf("expected %s to stay absent, stat err = %v", p, err)
+ }
+ })
+
t.Run("zero_is_rewritten", func(t *testing.T) {
p := filepath.Join(dir, "zero")
if err := os.WriteFile(p, []byte("0\n"), 0o644); err != nil {Rebased on current main here: |
Yuan Gao (ygao-g)
left a comment
There was a problem hiding this comment.
LGTM.
Confirmed by measurement rather than by reading: in a netns replica of the worker topology, actor IPv6 egress across the veth is 100% packet loss with net.ipv6.conf.all.forwarding=0 and 0% with it set. On a running dual-stack kind cluster the worker pod netns reads ipv6 all.forwarding=0 and ipv4 ip_forward=1, so this is the write that actually changes behaviour — the existing IPv4 one is already a no-op in a pod.
One sequencing note for whoever merges: #1057 gives the actor veth an IPv6 address and a default route, and without this PR that route is a black hole. Neither PR declares the dependency, and no current CI lane would catch it — the e2e clusters are IPv4-only by default, so the IPv6 path never runs.
The helper has enabled both address families since IPv6 forwarding was added; the name now says so. The single call site in SetupActorNetwork is updated along with the doc comment.
The os.Stat/IsNotExist fallback was never executed by the unit tests: every existing subtest's temp path could be created, so each returned at the os.WriteFile fast path. Point the new subtest at a node under a directory that does not exist — what procfs always does in production — and assert the file stays absent.
|
Both asks are addressed; the delta since your approval is exactly two commits:
Local verification: On the sequencing note: agreed, the dependency should be declared. GitHub has no native PR-to-PR dependency, so the practical options are a merge-order note in both PR bodies — I'm adding one here stating #979 must merge before #1057; worth mirroring in #1057 — or a CI is green on the updated head (check-changes + CLA). Happy to re-request review if you want a fresh look at the two commits; otherwise I'll take your approval as covering them. |
ateomnet: enable IPv6 forwarding in the worker pod netns
Fixes: #945
Problem
EnableForwarding(formerlyEnableIPv4Forwarding) only wrote/proc/sys/net/ipv4/ip_forward. It had noIPv6 counterpart, so
net.ipv6.conf.all.forwardingstayed at 0 in the workerpod network namespace. On dual-stack / IPv6-only clusters, every IPv6 packet
an actor sends — including its own DNS queries — is dropped by the kernel's
ip6_forward(), because the packet enters on the actor veth and must beforwarded out through
eth0.Reproduced as
TestActorEgressreturning 504 on IPv6-only clusters (4/4 runs).Fix
The helper is renamed to
EnableForwardingsince it now covers both addressfamilies, and it writes
/proc/sys/net/ipv6/conf/all/forwarding.conf.all.forwarding=1sets theforwarding state for all existing interfaces (and is the default for new
ones), so a single write covers both the veth and
eth0.The write is factored into a
writeSysctlIfUnsethelper that preserves theoriginal behavior:
1, return early (no remount, no write);/proc/sys(the worker holdsCAP_SYS_ADMIN and uses no user namespace), write
1\n, and restorero.Test
internal/ateomnet/write_sysctl_test.gocovers the helper's fast paths(already-set is untouched, unset/zero is written) with a temp file, no root
required. The privileged remount path is exercised by the existing netns
integration tests (
withTestNetNS).Compatibility
sysctl is a no-op where IPv6 is disabled or the file is absent — the helper
returns nil on missing path).
InstallActorNftablesRulesIPv4-only TODO remains accurate — this PR onlyrestores the kernel forwarding path so IPv6 packets reach the pod's eth0.
Verification
gofmt -lclean.go build ./internal/ateomnet/passes.go test ./internal/ateomnet/ -run TestWriteSysctlIfUnsetpasses.go test ./internal/ateomnet/passes.Merge order
#1057 gives the actor veth an IPv6 address and a default route; without this
PR's forwarding write that route is a black hole. This PR must merge before
#1057. No current CI lane catches the ordering (default e2e clusters are
IPv4-only); tracked in #1094 (opt-in dual-stack kind e2e).