Skip to content

Let a NIC-less sandbox reach out with no configuration - #57

Merged
CMGS merged 10 commits into
mainfrom
pr2-none-lane-egress
Jul 29, 2026
Merged

Let a NIC-less sandbox reach out with no configuration#57
CMGS merged 10 commits into
mainfrom
pr2-none-lane-egress

Conversation

@doge-rgb

@doge-rgb doge-rgb commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

A net=none sandbox has no NIC at all: its egress is spliced over vsock to a
host-side proxy. That path worked, but only for a guest that had been told about
it — an unconfigured curl inside the sandbox still failed, which makes the
lane useless as a default.

Bake the proxy into the guest. The base image sets the proxy variables on
silkd's unit, so the lane is usable the moment a sandbox is claimed.

Forward them into every exec — but only where they help. sysutil::base_env
deliberately does not inherit silkd's environment, so silkd now forwards exactly
the six proxy variables — and only when the guest has no NIC beyond lo/sit0.
On a lane with its own NIC the loopback relay's host side is closed unless a
policy is armed, so forwarding there would steer every env-honoring client into
a dead proxy while the direct NIC works. Children that inherit silkd's
environment instead of starting from base_env (git, language servers) follow
the same lane rule via align_proxy_env. The proxy values are snapshotted once
(the unit environment is fixed at service start); the allow-list and the lane
gate are both pinned by test.

Let a node name what its proxy may reach. The proxy's SSRF guard refuses
private destinations, which also refuses the internal services a sandbox
legitimately needs. egress_internal_allow names the prefixes to re-admit —
node-wide: every pool and tenant on the node gets the same re-admission. The
guard's shape is pinned by test: re-admitting the named ranges must not re-admit
the guest bridges, which are themselves ULA/RFC1918 — a blanket "permit private"
would let one sandbox reach another and the host's own gateway. The fully-open
form is pinned separately so that choosing it stays deliberate.

Verified on a 20-node fleet: from a guest whose ip link lists only lo and
sit0, an unconfigured curl reaches a public host, an internal v4 host and a
v6-only internal host — all without proxy arguments.

go test ./... green; golangci-lint (pinned v2.12.2) clean on GOOS=linux and
GOOS=darwin; silkd cargo fmt --check, cargo clippy --all-targets -- -D warnings and cargo test clean.

doge-rgb added 4 commits July 30, 2026 00:33
The proxy refuses every private and special-purpose target so a guest
cannot turn it into an SSRF. That also refuses the corporate services a
sandbox legitimately needs: internal names resolve to a ULA v6 (and an
RFC1918 v4 the node cannot route anyway), both of which the guard reads
as internal.

Name the prefixes instead of adding a "permit private" switch. The guest
bridges are themselves ULA and RFC1918, so a blanket permit would hand
every sandbox a route to its neighbours and to the host's own gateway
address — the two things the guard exists to prevent. An explicit list
admits the corporate ranges and nothing adjacent to them.

The list is validated at load, so a malformed prefix fails startup rather
than surfacing as a refused dial hours later. Empty is the default and
behaves exactly as before.
A node that lists the whole address space turns the guard off on purpose,
loopback and cloud metadata included. That is a deliberate configuration
for fleets that put a policy-enforcing forward proxy in front; the test
records what it opens so the choice stays visible.
…s out

A none-lane guest has no NIC at all — that is what makes the lane fast
(1000 VMs in 5s at a flat 200/s, against 86s and a collapsing rate for the
same count on the egress lane, where each VM takes a bridge port). Its
only way out is silkd's loopback relay on 3128, which splices to the
host's egress proxy over vsock.

Nothing reached it before because nothing told a client it was there. The
variables go on silkd's own unit rather than /etc/environment or a
profile script: every exec in the guest is a child of silkd, so this is
the one place that covers a bare `curl` as well as a login shell.

On the egress lane the NIC carries traffic directly and the relay simply
goes unused, so one image serves both.
An exec starts from a clean environment on purpose — silkd's own settings
are not the guest's. But that also drops the one thing a guest cannot
discover for itself. On the no-network lane there is no NIC at all; the
only way out is the loopback relay silkd runs, and an unconfigured client
never thinks to look for it. Baking the variables into silkd's unit was
not enough for the same reason: they reached silkd and stopped there.

Forward exactly the six proxy names, and only when set. Everything else
stays uninherited, and a request's own env still layers on top.
@CMGS
CMGS force-pushed the pr2-none-lane-egress branch from 7e6e17d to 296acbc Compare July 29, 2026 16:33
CMGS added 6 commits July 30, 2026 00:49
silkd forwards the baked proxy variables only when the guest has no NIC
(nothing in /sys/class/net beyond lo/sit0): on an egress lane without an
armed policy the loopback relay's host side is closed, and the variables
would steer every env-honoring client into it while the direct NIC works.
The forwarding test injects its environment instead of mutating the
process's (set_var is unsafe and racy under parallel tests). Also: use
documentation-style prefixes in the dialer tests, drop the dead
egressDialer var and a subset-duplicate test, extract validateEgressAllow
(gocyclo 31 > 30 broke CI), move the new tests above the file's helpers,
and cut comments to the why.
git and language servers inherit silkd's unit environment rather than
starting from base_env, so the baked proxy variables reached them on
every lane; align_proxy_env scrubs them when the guest has a NIC (git
clone would otherwise dial a closed relay) and leaves them inherited on
the none lane, which needs them. The proxy values are snapshotted once —
the unit environment is fixed at service start — and base_env returns
only borrowed statics, dropping the per-exec String allocations.
egress.md drops the manual -x incantation for the baked-env story and
documents egress_internal_allow (node-wide, post-NAT64, named prefixes
not permit-private); silkd.md notes the lane-gated env forwarding.
sysutil reuses net::has_egress instead of a second, weaker NIC probe,
which also honors SILKD_NET.
The Android init service sets only SILKD_PORT, so the none lane's
zero-config egress never got its proxy variables there; mirror the
base:24.04 unit. The git e2e lane test mutated SILKD_NET with set_var,
whose safety note stopped holding once align_proxy_env made every
parallel git test read it; net.override_egress_for_tests injects the
lane through an atomic instead.
The env read allocated a String on every spawn; the variable is
operator config fixed at service start, and tests now inject the lane
through the atomic instead.
@CMGS
CMGS merged commit 081e584 into main Jul 29, 2026
2 checks passed
@CMGS
CMGS deleted the pr2-none-lane-egress branch July 29, 2026 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants