Migrate from nginx Ingress to Gateway API (Envoy Gateway)

[dapperdivers]

Context

Migrate from nginx ingress to Gateway API. This is the first candidate for vCluster-based PR testing per our SDLC pipeline (#2565).

Why

• Gateway API is the Kubernetes standard going forward (GA)
• More expressive routing (header-based, weighted, traffic splitting)
• Better separation of concerns (infra manages Gateway, apps manage HTTPRoutes)
• Cilium has native Gateway API support
• nginx ingress is in maintenance mode

SDLC Workflow for This Change

1. Create feature branch
2. flux-local validates the Kustomization changes on PR
3. Deploy to dev vCluster for runtime testing
4. Verify routes work, TLS terminates, traffic flows
5. Merge to main → Flux reconciles production
6. Migrate apps incrementally (not big bang)

Implementation Options

• Option A: Envoy Gateway (onedr0p's choice, mature, feature-rich)
• Option B: Cilium Gateway API (native, no extra controller, already have Cilium)

Tasks

• Research Envoy Gateway vs Cilium Gateway for our stack
• Deploy chosen controller to dev vCluster
• Create GatewayClass + Gateway resources
• Migrate echo-server first (canary)
• Migrate internal apps one-by-one
• Migrate external apps
• Remove ingress-nginx
• Update cert-manager integration

Related

• Deploy Envoy Gateway for Gateway API migration #2556 — Phase 2 Gateway API issue
• Define cluster SDLC pipeline: flux-local → vCluster → production #2565 — SDLC pipeline definition