Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ Things not in any single grep-able file:
- **Wildcard DNS**: `*.live.k8s.phl.io` → the Envoy LB `45.79.246.168`. DNS is managed in OpenTofu at [CodeForPhilly/ops](https://github.com/CodeForPhilly/ops) → `tofu/dns`; a host with no specific record simply follows the wildcard, so new apps need no DNS change at all.
- **Apex domains in tree**: `balancerproject.org`, `choosenativeplants.com` (+ `www.`), `codeforphilly.org` (+ `www.`), `penn-chime.phl.io`, `vaultwarden.phl.io`, `bitwarden.phl.io`. Apex ACME challenges only work once DNS points at Envoy — plan cutover and cert issuance together for these. `choosenativeplants.com` is at **Namecheap**, not Cloud DNS, so it can't be moved from the ops repo.
- **A new hostname is briefly down between DNS and cert.** The cert can't issue until the hostname resolves to Envoy (Let's Encrypt has to reach the solver), and Envoy's HTTPS listener doesn't program until the cert Secret exists — meanwhile HTTP 301s into a listener that isn't there. Roughly 60–90s. Keep TTLs at 60s.
- **No cnpg / shared-cluster** on this cluster yet. If a database is needed, it ships per-app (e.g. vaultwarden runs its own PostgreSQL StatefulSet via the gissilabs chart; chime + third-places similar).
- **cnpg is landing.** `_infra/cloudnative-pg/` installs the operator (chart v0.28.0) and a `shared-cluster` Cluster, mirroring cfp-sandbox-cluster. It has **no backup configuration** — do not let it hold the only copy of anything until an object store lands (CodeForPhilly/balancer-main#526). Existing apps still ship their own database per-app (vaultwarden runs a PostgreSQL StatefulSet via the gissilabs chart; chime + third-places similar); nothing has been migrated onto the shared cluster yet.

## Guardrails

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[holomapping]
holosource = "cloudnative-pg-chart"
root = "charts/cloudnative-pg"
files = "**"
14 changes: 14 additions & 0 deletions .holo/lenses/cloudnative-pg.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
[hololens]
container = "ghcr.io/hologit/lenses/helm3:latest"

[hololens.input]
root = "_infra/cloudnative-pg/operator"
files = "**"

[hololens.output]
merge = "replace"

[hololens.helm]
namespace = "cloudnative-pg"
release_name = "cloudnative-pg"
include_crds = true
3 changes: 3 additions & 0 deletions .holo/sources/cloudnative-pg-chart.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[holosource]
url = "https://github.com/cloudnative-pg/charts.git"
ref = "refs/tags/cloudnative-pg-v0.28.0"
4 changes: 4 additions & 0 deletions _infra/cloudnative-pg/namespaces.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
apiVersion: v1
kind: Namespace
metadata:
name: cloudnative-pg
26 changes: 26 additions & 0 deletions _infra/cloudnative-pg/shared-cluster.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: shared-cluster
namespace: cloudnative-pg
spec:
instances: 2
imageName: ghcr.io/cloudnative-pg/postgis:18-3-system-trixie

storage:
# Explicit, unlike sandbox, which takes the cluster default. `-retain` keeps
# the Linode volume if the PVC is ever deleted — on this cluster that is the
# difference between an incident and an outage.
storageClass: linode-block-storage-retain
size: 20Gi

managed:
roles:
- name: balancer
login: true
passwordSecret:
name: balancer-db-credentials

# No `backup:` stanza yet — it needs an object store bucket plus
# credentials that do not exist. This cluster must not hold the only copy of
# production data until that lands. See CodeForPhilly/balancer-main#526.
50 changes: 50 additions & 0 deletions balancer/app/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: balancer

# balancer-main v1.1.7 owns its own HTTPRoute (paths, backends, ports) and no
# longer ships an Ingress. This repo owns the Gateway — hostname, TLS, issuer —
# in _gateways/balancer.yaml. The route attaches to it by name and declares no
# hostnames, so it inherits the Gateway's listener hostname.
#
# We deliberately do not take upstream's gateway-listeners.yaml, and v1.1.7
# removed it: it declared a ListenerSet, which Envoy Gateway v1.7.3 does not
# reconcile ("XListenerSet CRD not found, skipping XListenerSet watch"). It
# would apply cleanly and then be silently ignored.
resources:
- manifests/namespace.yaml
- manifests/deployment.yaml
- manifests/service.yaml
- manifests/httproute.yaml

# Stays on the running version. v1.1.7 is a deploy-manifest release with no
# application changes, so bumping the image here would ship app code under cover
# of a routing change. Bump it deliberately, on its own.
images:
- name: ghcr.io/codeforphilly/balancer-main/app
newTag: "1.1.5"

# v1.1.7's Deployment does `envFrom: configMapRef: balancer-config` alongside the
# existing `secretRef: balancer-config` (our SealedSecret, holding SECRET_KEY,
# SQL_*, and the API keys). A ConfigMap and a Secret may share a name.
#
# balancer.env is empty upstream, and the app reads ALLOWED_HOSTS rather than
# HOSTNAME — so this ConfigMap carries nothing. It exists solely so configMapRef
# resolves; without it the pods fail with CreateContainerConfigError.
configMapGenerator:
- name: balancer-config
envs:
- manifests/balancer.env

generatorOptions:
disableNameSuffixHash: true

patches:
- target:
kind: Namespace
name: balancer
patch: |-
- op: replace
path: /metadata/name
value: balancer
16 changes: 16 additions & 0 deletions balancer/cnpg/database.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
apiVersion: postgresql.cnpg.io/v1
kind: Database
metadata:
name: balancer
namespace: cloudnative-pg
spec:
name: balancer
owner: balancer
cluster:
name: shared-cluster
# Django's api.0005_embeddings migration creates a vector(384) column.
# cnpg-controller runs CREATE EXTENSION as the postgres superuser; the
# `balancer` role has no superuser perms and cannot enable extensions itself.
extensions:
- name: vector
ensure: present
8 changes: 8 additions & 0 deletions balancer/cnpg/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# No namespace — database.yaml declares its own (cloudnative-pg). cnpg requires
# the Database CR to live in the same namespace as the Cluster it targets, so we
# keep it out of the balancer-app rewrite.
resources:
- database.yaml
53 changes: 7 additions & 46 deletions balancer/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -1,50 +1,11 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: balancer

# balancer-main v1.1.7 owns its own HTTPRoute (paths, backends, ports) and no
# longer ships an Ingress. This repo owns the Gateway — hostname, TLS, issuer —
# in _gateways/balancer.yaml. The route attaches to it by name and declares no
# hostnames, so it inherits the Gateway's listener hostname.
#
# We deliberately do not take upstream's gateway-listeners.yaml, and v1.1.7
# removed it: it declared a ListenerSet, which Envoy Gateway v1.7.3 does not
# reconcile ("XListenerSet CRD not found, skipping XListenerSet watch"). It
# would apply cleanly and then be silently ignored.
# No namespace here — each sub-kustomization sets its own:
# - app/ rewrites resources to `balancer`
# - cnpg/ leaves the Database CR in `cloudnative-pg`, where cnpg requires it
# kustomize emits a single stream; k8s-normalize routes by each resource's
# metadata.namespace at deploy time. Same split as cfp-sandbox-cluster.
resources:
- manifests/namespace.yaml
- manifests/deployment.yaml
- manifests/service.yaml
- manifests/httproute.yaml

# Stays on the running version. v1.1.7 is a deploy-manifest release with no
# application changes, so bumping the image here would ship app code under cover
# of a routing change. Bump it deliberately, on its own.
images:
- name: ghcr.io/codeforphilly/balancer-main/app
newTag: "1.1.5"

# v1.1.7's Deployment does `envFrom: configMapRef: balancer-config` alongside the
# existing `secretRef: balancer-config` (our SealedSecret, holding SECRET_KEY,
# SQL_*, and the API keys). A ConfigMap and a Secret may share a name.
#
# balancer.env is empty upstream, and the app reads ALLOWED_HOSTS rather than
# HOSTNAME — so this ConfigMap carries nothing. It exists solely so configMapRef
# resolves; without it the pods fail with CreateContainerConfigError.
configMapGenerator:
- name: balancer-config
envs:
- manifests/balancer.env

generatorOptions:
disableNameSuffixHash: true

patches:
- target:
kind: Namespace
name: balancer
patch: |-
- op: replace
path: /metadata/name
value: balancer
- app
- cnpg