feat(openshift): add mTLS for Prometheus metrics#3473
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
/kind feature |
There was a problem hiding this comment.
Pull request overview
Adds OpenShift-specific mutual TLS support for Prometheus scraping of Tekton component metrics endpoints by annotating metrics Services for serving cert issuance, mounting TLS materials into workloads, and updating ServiceMonitors to scrape over HTTPS.
Changes:
- Introduces OpenShift manifest transformers to inject serving-cert annotations, mount metrics TLS Secret + client-CA ConfigMap, and set
METRICS_PROMETHEUS_TLS_*env vars. - Updates OpenShift component reconcilers to apply the new metrics mTLS transformers to Deployments/StatefulSets and patch ServiceMonitors.
- Extends ServiceMonitor namespace rewriting to also patch the namespace segment in
tlsConfig.serverName.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/reconciler/openshift/tektontrigger/extension.go | Applies metrics mTLS transformers to Triggers controller. |
| pkg/reconciler/openshift/tektonresult/extension.go | Applies metrics mTLS transformers to Results watcher/API. |
| pkg/reconciler/openshift/tektonpruner/extension.go | Applies metrics mTLS transformers to Pruner controller. |
| pkg/reconciler/openshift/tektonpipeline/extension.go | Applies metrics mTLS transformers to Pipelines controllers and refines monitoring transforms. |
| pkg/reconciler/openshift/tektonconfig/extension.go | Ensures the metrics client CA ConfigMap is synced into the target namespace. |
| pkg/reconciler/openshift/tektonchain/extension.go | Applies metrics mTLS transformers to Chains metrics Service / controller. |
| pkg/reconciler/openshift/openshiftpipelinesascode/extension.go | Applies metrics mTLS transformers and updates PAC ServiceMonitors for HTTPS scraping. |
| pkg/reconciler/openshift/common/transformer.go | Extends ServiceMonitor namespace transformer to also update tlsConfig.serverName. |
| pkg/reconciler/openshift/common/metricstls.go | New: common transformers/helpers for metrics mTLS (Service annotation/port rename, workload mounts/env, ServiceMonitor TLS wiring). |
| pkg/reconciler/openshift/common/metricsca.go | New: sync logic for Prometheus client CA into component namespaces. |
| cmd/openshift/operator/kodata/openshift-monitoring/00-monitoring.yaml | Updates Pipelines controller ServiceMonitor endpoint to HTTPS with TLS config. |
| cmd/openshift/operator/kodata/openshift-monitoring/01-trigger-monitoring.yaml | Updates Triggers ServiceMonitor endpoint to HTTPS with TLS config. |
| cmd/openshift/operator/kodata/openshift-monitoring/02-chains-monitoring.yaml | Updates Chains ServiceMonitor endpoint to HTTPS with TLS config. |
| cmd/openshift/operator/kodata/openshift-monitoring/03-pipeline-webhook-monitoring.yaml | Explicitly sets webhook ServiceMonitor scheme to HTTP. |
| cmd/openshift/operator/kodata/openshift-monitoring/05-results-monitoring.yaml | Updates Results ServiceMonitors endpoints to HTTPS with TLS config. |
| cmd/openshift/operator/kodata/openshift-monitoring/06-pruner-monitoring.yaml | Updates Pruner ServiceMonitor endpoint to HTTPS with TLS config. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Add the client-CA ConfigMap volume. | ||
| clientCAVol := corev1.Volume{ | ||
| Name: metricsClientCAVolume, | ||
| VolumeSource: corev1.VolumeSource{ | ||
| ConfigMap: &corev1.ConfigMapVolumeSource{ | ||
| LocalObjectReference: corev1.LocalObjectReference{ | ||
| Name: MetricsClientCAConfigMap, | ||
| }, | ||
| }, |
There was a problem hiding this comment.
The ordering guarantee is the fix. EnsureMetricsClientCA runs in TektonConfig.PreReconcile, which executes before any child component (TektonPipeline, TektonResult, etc.) is reconciled and before their pods are created. The ConfigMap is always present by the time pods are scheduled.
If the ConfigMap is deleted after install, TektonConfig's periodic reconciliation recreates it. In the small window between deletion and recreation, a pod restart will block — which is intentional. A pod without the client-CA bundle cannot perform mTLS client verification at all, so failing loudly (pod unschedulable) is preferable to starting silently with broken mTLS. This is consistent with how other mandatory ConfigMaps (e.g. CA bundles) are handled throughout the operator.
44e4a21 to
07c10f6
Compare
Secure component metrics endpoints with mutual TLS on OpenShift. Each reconciler syncs the Prometheus client CA bundle from kube-system/extension-apiserver-authentication into the component namespace as a metrics-client-ca ConfigMap. OpenShift's serving-cert controller is triggered via annotation on each metrics Service to provision a per-component TLS Secret. Two new manifest transformers (InjectMetricsServingCert, ApplyMetricsTLS) wire the Secret and ConfigMap as volumes and inject METRICS_PROMETHEUS_TLS_* env vars so that the knative/pkg prometheus.Server enables mTLS with require client auth. ServiceMonitor resources are updated with scheme: https, scrapeClass tls-client-certificate-auth, and tlsConfig.serverName. The existing UpdateServiceMonitorTargetNamespace transformer is extended to also patch the namespace segment inside serverName at runtime. Relates-To: SRVKP-8172 Signed-off-by: Jawed khelil <jkhelil@redhat.com> Assisted-by: Claude Sonnet 4.6 (via Cursor) Co-authored-by: Cursor <cursoragent@cursor.com>
|
New changes are detected. LGTM label has been removed. |
Secure component metrics endpoints with mutual TLS on OpenShift. Each reconciler syncs the Prometheus client CA bundle from kube-system/extension-apiserver-authentication into the component namespace as a metrics-client-ca ConfigMap.
OpenShift's serving-cert controller is triggered via annotation on each metrics Service to provision a per-component TLS Secret. Two new manifest transformers (InjectMetricsServingCert, ApplyMetricsTLS) wire the Secret and ConfigMap as volumes and inject METRICS_PROMETHEUS_TLS_* env vars so that the knative/pkg prometheus.Server enables mTLS with require client auth.
ServiceMonitor resources are updated with scheme: https, scrapeClass tls-client-certificate-auth, and tlsConfig.serverName. The existing UpdateServiceMonitorTargetNamespace transformer is extended to also patch the namespace segment inside serverName at runtime.
Relates-To: SRVKP-8172
Assisted-by: Claude Sonnet 4.6 (via Cursor)
Changes
Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:
make test lintbefore submitting a PRSee the contribution guide for more details.
Release Notes