feat(gatewayapi): calico-system policy for namespaced data-plane proxies#4970
Open
electricjesus wants to merge 3 commits into
Open
feat(gatewayapi): calico-system policy for namespaced data-plane proxies#4970electricjesus wants to merge 3 commits into
electricjesus wants to merge 3 commits into
Conversation
electricjesus
force-pushed
the
seth/gatewayapi-proxy-allow-policy
branch
from
d3d69a5 to
311699b
Compare
Since deploy.type=GatewayNamespace (tigera#4690) the data-plane envoy proxies run in each Gateway's own namespace, not calico-system. The only calico-system gateway policy selects the controller/certgen pods in calico-system, so the proxies match nothing and have no policy punching through a default-deny tier in the namespaces they now run in. Add a GlobalNetworkPolicy selecting the EG proxy pods (label gateway.envoyproxy.io/owning-gateway-name) so it covers every Gateway namespace with no re-render: DNS + xDS(18000)/Wasm(18002) egress to the controller in calico-system, and all inbound TCP so a managed Gateway serves traffic out of the box under a default-deny tier. Backend egress is left to the user, matching the controller policy.
electricjesus
force-pushed
the
seth/gatewayapi-proxy-allow-policy
branch
from
311699b to
fe034ab
Compare
electricjesus
changed the title
rene-dekker
reviewed
Jul 7, 2026
| // not calico-system, so they need their own policy in the calico-system tier. EnvoyProxy | ||
| // stamps gateway.envoyproxy.io/owning-gateway-name on every proxy pod, so use it as the selector. | ||
| ProxyPolicyName = networkpolicy.CalicoComponentPolicyPrefix + "envoy-gateway-proxy" | ||
| EnvoyProxyPolicySelector = "has(gateway.envoyproxy.io/owning-gateway-name)" |
Member
There was a problem hiding this comment.
Could we set some calico labels on these objects and use those for policy?
Related: do the envoy related components have standard labels that we set in utils/component.go? If not,that is a gap.
Per review: instead of selecting the data-plane proxy pods on Envoy Gateway's gateway.envoyproxy.io/owning-gateway-name label, which we do not own and which could change upstream without notice, stamp our own k8s-app=calico-gateway-api-proxy label on the proxy pods through the EnvoyProxy pod spec and point the calico-system-tier proxy policy at that. Any user-supplied pod labels from a custom EnvoyProxy are kept.
Put the Calico-owned k8s-app labeling for all three envoy-gateway components in one documented place. The operator-rendered controller and certgen pods go through setGatewayComponentLabel; the runtime-created proxy pods, which the operator never renders, go through the EnvoyProxy pod spec (ensureGatewayProxyLabel). Documents why this lives in the gateway render and not the standard labeler in component.go, which keys off the object name and cannot reach a runtime-created pod.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Bug fix.
Since #4690 we run a single envoy-gateway controller in
calico-systemwithdeploy.type=GatewayNamespace. The data-plane proxies now run in each Gateway's own namespace, not incalico-system.We render one gateway policy in the
calico-systemtier:calico-system.envoy-gateway. It lives incalico-systemand selects the controller and certgen pods. The proxies run elsewhere and match no policy in that tier. When a Gateway namespace runs a default-deny tier, the proxy there has nothing to let its traffic through. This is the same kind of miss as the conformance MetalLB pool that stayed pinned totigera-gateway(projectcalico/calico#13095).This adds a
GlobalNetworkPolicy,calico-system.envoy-gateway-proxy, that selects the proxy pods byk8s-app == 'calico-gateway-api-proxy'. The operator stamps that Calico-owned label on the proxy pods through the EnvoyProxy pod spec, so the policy keys off a label we control rather than Envoy Gateway's owngateway.envoyproxy.io/owning-gateway-name, which we do not own and which could change upstream. A GNP covers any Gateway namespace, including ones created later, with no re-render. It allows:calico-system. The proxy dials the controller (seeenvoyproxy/gatewayinternal/infrastructure/kubernetes/proxy/resource.goandinternal/xds/bootstrap). 18001 is the ratelimit path, not a proxy path, so it is left out.The same Calico-owned labelling now covers all three envoy-gateway components (controller, certgen, proxy) from one documented place in the gateway render. It is not set through the standard labeller in
pkg/controller/utils/component.goon purpose: that labeller keys off the object name, and the proxy pods are created by the envoy-gateway controller at runtime, so the operator never renders them to label.How I tested it
I built this operator image and ran it on an OSS master cluster (eBPF, namespaced mode). I put a Gateway, an HTTPRoute, and an nginx backend in a normal user namespace, scoped a Calico default-deny to the proxy pod, and curled the proxy from inside the cluster.
I also tried the narrower ingress idea (allow only 19001, then Pass). Under default-deny it times out: listener traffic falls through to the user's deny and the Gateway stops serving. So allowing all inbound TCP is the right default. The cost is that an Allow is terminal in this tier, so a user cannot narrow proxy ingress with their own policy. Scaling the operator back up showed it reverts any drift on the GNP and the Gateway recovers.
I re-ran the whole repro after switching the selector to the Calico-owned
k8s-applabel (OSS master, eBPF, namespaced mode). The operator stampsk8s-app: calico-gateway-api-proxyon the proxy pod, the rendered GNPcalico-system.envoy-gateway-proxyselects it, and the table holds: baseline 200; under a Gateway-namespace default-deny the proxy returns 503 (reachable and routing, only the backend hop denied); allowing proxy to backend gives 200. Scaling the operator down and deleting the GNP drops the proxy to a full timeout, and scaling it back up re-renders the GNP and the Gateway recovers. One thing to watch when testing this by hand: the 503 lands after Envoy's ~10s upstream timeout, so a curl timeout under 10s reads as a connect failure even though the proxy is reachable..One thing to know
The GNP covers the proxy's own needs: DNS, the control-plane link, and ingress. It does not open egress to backends, because backends are arbitrary user workloads. A user who runs default-deny in a Gateway namespace has to allow the proxy to reach their backend themselves. Until they do, the proxy is up and configured but returns 503 on the upstream. This matches how the controller policy already works, and is worth a line in the docs.
Release Note
For PR author
make gen-files(n/a, no API change).make gen-versions(n/a).