Skip to content

hack: fix DNS on IPv6-only kind clusters - #958

Open
Yuan Gao (ygao-g) wants to merge 2 commits into
agent-substrate:mainfrom
ygao-g:kind-ipv6-coredns
Open

hack: fix DNS on IPv6-only kind clusters#958
Yuan Gao (ygao-g) wants to merge 2 commits into
agent-substrate:mainfrom
ygao-g:kind-ipv6-coredns

Conversation

@ygao-g

@ygao-g Yuan Gao (ygao-g) commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Part of #246.

  • Tests pass
  • Appropriate changes to documentation are included in the PR (none needed — the new IPV6_DNS_UPSTREAM knob is documented in the script's --help)

On a fresh IP_FAMILY=ipv6 cluster nothing resolves from inside a pod and no actor boots: CoreDNS runs dnsPolicy: Default and inherits the node's IPv4 resolver, which a v6-only pod cannot reach. Two Corefile clauses fix it — a forward to an IPv6 upstream, overridable with IPV6_DNS_UPSTREAM, and a hosts entry for kind-registry, which step 3 wires into containerd on the node but atelet pulls from its own pod netns. IPv4 and dual-stack pods reach both names through kind's own DNS translation, so this is gated on ipv6 and those families are untouched.

A probe pod checks both legs before the script exits, so a misconfiguration is named here rather than surfacing much later as an actor that will not boot. The first commit is a prerequisite: on a native Linux daemon the old unconditional kubeconfig rewrite traded a working [::1] for a refused localhost.

No CI lane covers this (#939 is gated on this PR), so it was checked by hand: four consecutive clean ipv6 bring-ups, three negative controls that each exit 1 with their own message (unreachable upstream, unpatched Corefile, stale registry address), and an ipv4 run confirming the Corefile is left alone. atenet-egress still crashloops on v6-only, for an unrelated Envoy bind bug.

🤖 Generated with Claude Code

@ygao-g
Yuan Gao (ygao-g) force-pushed the kind-ipv6-coredns branch 2 times, most recently from 45bac3d to ecf1fc6 Compare August 18, 2026 16:00
@ygao-g Yuan Gao (ygao-g) changed the title hack/create-kind-cluster.sh: fix DNS on IPv6-only kind clusters hack: fix DNS on IPv6-only kind clusters Aug 18, 2026
@ygao-g
Yuan Gao (ygao-g) force-pushed the kind-ipv6-coredns branch 3 times, most recently from e812e99 to 3350f55 Compare August 18, 2026 22:07
@ygao-g Yuan Gao (ygao-g) added kind/bug Something isn't working / bugfixes area/network labels Aug 18, 2026
@ygao-g
Yuan Gao (ygao-g) marked this pull request as ready for review August 19, 2026 00:32
@ygao-g

Yuan Gao (ygao-g) commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator Author

Testing

Each check is paired with a control that must fail — the failure mode this guards against is a probe that passes without proving anything.

  • IPv6-only. IP_FAMILY=ipv6 hack/create-kind-cluster.sh--deploy-ate-system--deploy-demo-counter: atelet pulls the gVisor tarball from storage.googleapis.com over IPv6 from its own pod netns, the actor image pulls through kind-registry, and a golden actor runs and checkpoints. 11 of 11 fresh clusters exit 0 on a single probe pod. hack/run-e2e-kind.sh: demo 19/0/5 with a skip set identical to the IPv4 control, networking 7/2 — both failures are atenet-egress crashlooping on v6-only for an unrelated Envoy bind.
  • Controls. IPV6_DNS_UPSTREAM=2001:db8::1 exits 1 in ~28s, naming each replica that did not answer. Pointing the hosts entry at an address the registry is not on exits 1 with the Corefile says [...] but the registry is at [...] — the hack: IPv6-only kind clusters resolve a stale registry address #1049 case, told apart from a registry that is down. The stock Corefile fails the probe; dropping only the hosts block leaves external names working and the registry fetch failing.
  • IPv4 regression. Step 4.5 and the verifier do not run, the Corefile keeps forward . /etc/resolv.conf, and the demo stays green. dual takes the same path.

CI has no IPv6 job and this PR does not add one — #939 is a signal-only draft that produces that coverage on demand.

@ygao-g

Copy link
Copy Markdown
Collaborator Author

Ping — this one is a two-file hack/ change, and it unblocks the IPv6-only CI job in #939, which carries it verbatim and can't merge first.

Worth knowing that the branch has moved since it was last looked at. Two commits went on, both fixes for defects I hit while trying to run it rather than responses to review: the DNS gate was reporting green with one of kind's two CoreDNS replicas unable to resolve, and the IPv6-only kubeconfig repoint was replacing a working apiserver address with a refused one on a native-Linux Docker host, which killed every kubectl after it.

Comment thread hack/create-kind-cluster.sh Outdated
Comment thread hack/create-kind-cluster.sh Outdated
Comment thread hack/create-kind-cluster.sh Outdated
Comment thread hack/create-kind-cluster.sh Outdated
Comment thread hack/verify-ipv6-dns.sh Outdated
kind writes an IPv6 kubeconfig pointing at [::1] and the script rewrote it
to localhost unconditionally, for the VM-hosted-daemon case where the port
is forwarded to the v4 loopback. On a native Linux daemon [::1] already
works and localhost is not a name for it -- Debian gives ::1 the names
ip6-localhost and ip6-loopback -- so the rewrite traded a working address
for a refused one and every later kubectl in the script died at connect.

It now probes [::1], falls back to localhost only if that fails, and errors
if neither answers, so a transient apiserver hiccup cannot strand the
kubeconfig on an address that never worked. First of two commits fixing
IPv6-only kind bring-up.
On a fresh IP_FAMILY=ipv6 cluster nothing resolves from inside a pod and
no actor boots: CoreDNS runs dnsPolicy: Default and inherits the node's
IPv4 resolver, which a v6-only pod cannot reach. Two Corefile clauses fix
it -- a forward to an IPv6 upstream, overridable with IPV6_DNS_UPSTREAM,
and a hosts entry for the local registry, which step 3 wires into
containerd on the node but atelet pulls from its own pod netns. IPv4 and
dual-stack pods reach both names through kind's own DNS translation, so
this is gated on ipv6 and those families are untouched.

A probe pod checks both legs before the script exits, because a silent
misconfiguration here does not surface until an actor fails to boot much
later. Second of two commits fixing IPv6-only kind bring-up.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/network kind/bug Something isn't working / bugfixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant