feat(balancer): declare the cnpg balancer role and Database

[TineoC]

Step 4 (continued) of CodeForPhilly/balancer-main#526. Creates an empty balancer database on shared-cluster with pgvector enabled, so the RDS dump has somewhere to land.

Still no cutover. The app's sealed SQL_HOST is untouched; production keeps reading from AWS RDS after this merges.

Stacked on #182. It branches from that PR, so the diff below includes its commit. Merge #182 first and this rebases to just the second commit. Draft until then — and until the credentials Secret exists (see blocker below).

What changed

balancer/ splits into app/ + cnpg/, matching cfp-sandbox-cluster. The Database CR must stay in the cloudnative-pg namespace — cnpg requires it beside the Cluster it targets — so it cannot sit under the kustomization that rewrites everything to balancer. The existing balancer/kustomization.yaml moves to balancer/app/ unchanged, and its holomapping moves with it (.holo/branches/k8s-manifests/balancer/app/manifests.toml). The new top-level kustomization just composes the two, setting no namespace of its own.

balancer/cnpg/database.yaml — Database/balancer, owner balancer, with:

extensions:
  - name: vector
    ensure: present

That answers the "is pgvector available on the CfP database?" question from the original Slack thread. It has to go through Database.spec.extensions rather than a CREATE EXTENSION — the balancer role has no SUPERUSER, so cnpg-controller runs it as postgres. Sandbox verified vector 0.8.2 this way.

_infra/cloudnative-pg/shared-cluster.yaml — adds the managed.roles entry deferred out of #182.

Blocker before this can merge

balancer-db-credentials must be sealed into cloudnative-pg.secrets/ first. Until that Secret exists, cnpg reports a role reconcile error. It needs cluster access, so it is not something I can deliver as a PR.

Gotcha from cfp-sandbox-cluster#162, worth having in hand: cnpg does not pick up a newly-created passwordSecret on its periodic resync. A two-hour wait produced nothing; this reconciled in seconds:

kubectl annotate cluster -n cloudnative-pg shared-cluster \
  cnpg.io/reconciliationLoop="$(date +%s)" --overwrite

Verification after deploy

kubectl -n cloudnative-pg get database balancer -o jsonpath='{.status.applied}'         # true
kubectl -n cloudnative-pg get database balancer -o jsonpath='{.status.extensions[0].applied}'  # true
kubectl -n cloudnative-pg exec shared-cluster-1 -- psql -d balancer -tAc "select extname, extversion from pg_extension"
kubectl -n balancer get pods    # unchanged, still pointed at RDS

Last line is the one that matters: nothing about the running app should move.

Next, once this is in

pg_dump RDS → pg_restore into this database → per-table n_live_tup parity check against RDS → only then re-seal balancer-config with the cnpg host and bump the image off 1.1.5. And an object store + ScheduledBackup before this cluster holds the only copy of anything.