feat(otel): use dedicated least-privilege monitoring user for OTel Collector sidecar

[udsmicrosoft]

Summary

Closes #344.

The OTel Collector sidecar (added in #342) connected to PostgreSQL using
PGUSER/PGPASSWORD from the <cluster>-app secret — the CNPG application
user, which has broad read/write access to application data. This PR switches
the sidecar to a dedicated least-privilege monitoring role that only has
membership in the built-in pg_monitor role.

What changed

1. Dedicated role — when monitoring is enabled, the operator declares an
   otel_monitor PostgreSQL role via CNPG's managed-roles reconciler
   (spec.managed.roles): LOGIN, member of pg_monitor, nothing else.
2. Dedicated secret — the operator generates and owns a basic-auth secret
   <cluster>-otel-monitor (username=otel_monitor, random password). The
   password is generated once and preserved across reconciles. CNPG consumes it
   as the role's passwordSecret (which requires the secret username to match
   the role name exactly).
3. Sidecar wiring — the sidecar-injector plugin gains an otelMonitorSecret
   parameter and sources PGUSER/PGPASSWORD from that secret instead of
   <cluster>-app.
4. Sync support — SyncCnpgCluster now reconciles spec.managed.roles so
   toggling monitoring on an existing cluster creates/removes the role. The
   Azure-Fleet managed-services path was refactored (addAzureFleetManagedServices)
   to preserve managed roles instead of overwriting the whole spec.managed.
5. RBAC — the operator ClusterRole (Helm 05_clusterrole.yaml and kustomize
   role.yaml) gains create;update;patch on secrets.

Design notes

• The desired role sets the CNPG/CRD-defaulted fields (connectionLimit=-1,
  inherit=true, ensure=present) explicitly so the sync diff stays stable and
  doesn't re-patch every reconcile.
• pg_hba is currently trust, so the password isn't strictly enforced today —
  but scoping the sidecar to a pg_monitor-only role still delivers the
  least-privilege, audit-attribution, and blast-radius benefits, and is
  future-proof if auth is tightened.

Testing

• go test ./... (operator module) — all pass
• go test ./... (sidecar-injector plugin module) — all pass
• go vet / gofmt clean; make manifests generate produce no diff
• New unit tests: monitor secret name/role helpers, managed-role generation,
  managed-role sync (add/remove/preserve-services/no-op), Azure-Fleet role
  preservation, secret reconcile (create + password stability) and delete, and
  plugin config/sidecar credential sourcing.

[documentdb-triage-tool]

🤖 Auto-triaged by documentdb-triage-tool.

Applied: go, test, enhancement
Project fields suggested: Component controllers · Priority P2 · Effort L · Status In Progress
Confidence: 0.85 (mixed)

Reasoning

component from path globs (controllers, test, manifests); effort from diff stats (568+33 LOC, 17 files); LLM: Multi-file, cross-component security improvement introducing a dedicated least-privilege monitoring role, new secret management, RBAC updates, and sidecar wiring — touches controllers, manifests, and the kubectl-plugin sidecar-injector.

If a label is wrong, remove it manually and ping @patty-chow so the rules can be tuned. The bot will not re-label items that already have component labels.

[Copilot]

Pull request overview

This PR updates the DocumentDB operator and CNPG sidecar-injector plugin to use a dedicated least-privilege PostgreSQL role (otel_monitor) and a dedicated per-cluster Secret (<cluster>-otel-monitor) for OTel Collector sidecar credentials, instead of reusing the broad CNPG application user credentials.

Changes:

• Introduces otel_monitor role + <cluster>-otel-monitor basic-auth Secret generation/cleanup in the operator and wires CNPG managed roles + plugin params to use it.
• Extends CNPG cluster sync to reconcile managed roles independently (so monitoring can be toggled on/off for existing clusters).
• Updates the sidecar-injector plugin to require and consume the new otelMonitorSecret parameter for PGUSER/PGPASSWORD.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
operator/src/internal/otel/config.go	Adds constants/helpers for monitoring role name and Secret naming.
operator/src/internal/otel/config_test.go	Adds unit tests for the new role/secret helpers.
operator/src/internal/controller/physical_replication.go	Refactors Azure Fleet managed-services setup to preserve existing spec.managed (e.g., roles).
operator/src/internal/controller/physical_replication_test.go	Adds tests for Azure Fleet managed-services behavior and role preservation.
operator/src/internal/controller/documentdb_controller.go	Reconciles the new monitoring Secret and cleans it up when monitoring is disabled; adds Secrets RBAC annotation.
operator/src/internal/controller/documentdb_controller_test.go	Adds tests for monitoring Secret create/stability and delete behavior.
operator/src/internal/cnpg/cnpg_sync.go	Syncs otelMonitorSecret plugin param and adds managed-roles reconciliation logic.
operator/src/internal/cnpg/cnpg_sync_test.go	Adds tests for managed-role add/remove/preserve/no-op sync scenarios.
operator/src/internal/cnpg/cnpg_patch.go	Adds JSON patch paths for /spec/managed and /spec/managed/roles.
operator/src/internal/cnpg/cnpg_cluster.go	Declares the managed otel_monitor role and passes otelMonitorSecret plugin param when monitoring is enabled.
operator/src/internal/cnpg/cnpg_cluster_test.go	Adds tests validating managed role + plugin param when monitoring enabled/disabled.
operator/src/config/rbac/role.yaml	Grants Secrets write verbs; adjusts ClusterRole rules (kustomize path).
operator/documentdb-helm-chart/templates/05_clusterrole.yaml	Grants Secrets write verbs and documents the new per-cluster monitoring Secret.
operator/cnpg-plugins/sidecar-injector/internal/lifecycle/lifecycle.go	Requires otelMonitorSecret for sidecar injection and sources PG creds from it.
operator/cnpg-plugins/sidecar-injector/internal/lifecycle/lifecycle_test.go	Adds tests asserting the sidecar uses the monitoring Secret for PG creds.
operator/cnpg-plugins/sidecar-injector/internal/config/config.go	Adds otelMonitorSecret parameter parsing into typed configuration.
operator/cnpg-plugins/sidecar-injector/internal/config/config_test.go	Adds unit test coverage for parsing the new OTel parameters.

[xgerman]

[Suggestion] Since this adds required Secret write/delete permissions, please call out upgrade ordering in release notes/changelog (apply updated RBAC before or with controller rollout). In local kind testing, running the new controller against old RBAC repeatedly failed with cannot create resource "secrets" until ClusterRole was updated.

[xgerman]

Summary

[Required] No blocking issues found in this PR.

I validated the key behavior changes locally (including kind):

• dedicated <cluster>-otel-monitor secret lifecycle (create/preserve/delete)
• spec.managed.roles reconciliation for otel_monitor
• sidecar credential wiring (PGUSER/PGPASSWORD) to otelMonitorSecret
• monitoring disable/enable cleanup and re-create path

Targeted tests run clean:

• go test ./... (sidecar-injector module)
• go test ./internal/otel ./internal/cnpg ./internal/controller (operator module)

I left one non-blocking inline suggestion about documenting RBAC upgrade ordering, based on local rollout behavior with stale ClusterRole.