Skip to content

TRT-2568: Revert "CCO-788: Remove kube-rbac-proxy container from metrics"#982

Merged
openshift-merge-bot[bot] merged 1 commit intoopenshift:masterfrom
stbenjam:revert-976-payload-4.22.0-0.nightly-2026-03-08-004901
Mar 8, 2026
Merged

TRT-2568: Revert "CCO-788: Remove kube-rbac-proxy container from metrics"#982
openshift-merge-bot[bot] merged 1 commit intoopenshift:masterfrom
stbenjam:revert-976-payload-4.22.0-0.nightly-2026-03-08-004901

Conversation

@stbenjam
Copy link
Member

@stbenjam stbenjam commented Mar 8, 2026

Revertomatic Revert

Field Value
Original PR #976
JIRA TRT-2568
Payload 4.22.0-0.nightly-2026-03-08-004901
Confidence 95 (HIGH)

Why

This PR is causing blocking job failures in the 4.22 nightly amd64 payload. All serial and techpreview-serial blocking jobs fail with:

[sig-instrumentation][Late] Platform Prometheus targets should not be accessible without auth [Serial]

Error: failed to ensure scraping target of pod openshift-cloud-credential-operator/cco-metrics requires auth: context deadline exceeded

The PR removed kube-rbac-proxy from the CCO metrics endpoint and reimplemented RBAC via controller-runtime. The Prometheus auth test checks that metrics endpoints require authentication, and the CCO metrics endpoint now fails this check.

Failing Jobs

  • aws-ovn-serial-1of2
  • aws-ovn-serial-2of2
  • aws-ovn-techpreview-serial-1of3
  • aws-ovn-techpreview-serial-2of3
  • aws-ovn-techpreview-serial-3of3
  • overall-analysis-all (downstream)

cc @jstuever

This reverts commit 6b2f075, reversing
changes made to eb0673e.
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 8, 2026
@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Mar 8, 2026

@stbenjam: This pull request references CCO-788 which is a valid jira issue.

Details

In response to this:

Revertomatic Revert

Field Value
Original PR #976
Payload 4.22.0-0.nightly-2026-03-08-004901
Confidence 95 (HIGH)

Why

This PR is causing blocking job failures in the 4.22 nightly amd64 payload. All serial and techpreview-serial blocking jobs fail with:

[sig-instrumentation][Late] Platform Prometheus targets should not be accessible without auth [Serial]

Error: failed to ensure scraping target of pod openshift-cloud-credential-operator/cco-metrics requires auth: context deadline exceeded

The PR removed kube-rbac-proxy from the CCO metrics endpoint and reimplemented RBAC via controller-runtime. The Prometheus auth test checks that metrics endpoints require authentication, and the CCO metrics endpoint now fails this check.

Failing Jobs

  • aws-ovn-serial-1of2
  • aws-ovn-serial-2of2
  • aws-ovn-techpreview-serial-1of3
  • aws-ovn-techpreview-serial-2of3
  • aws-ovn-techpreview-serial-3of3
  • overall-analysis-all (downstream)

cc @jstuever

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@stbenjam
Copy link
Member Author

stbenjam commented Mar 8, 2026

/payload-job 4.22 nightly aws-ovn-serial-1of2

@openshift-ci openshift-ci bot requested review from dlom and jstuever March 8, 2026 13:15
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 8, 2026

@stbenjam: trigger 0 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

@stbenjam
Copy link
Member Author

stbenjam commented Mar 8, 2026

/payload-job 4.22 nightly aws-ovn-techpreview-serial-1of3

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 8, 2026

@stbenjam: trigger 0 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Mar 8, 2026

@stbenjam: This pull request references CCO-788 which is a valid jira issue.

Details

In response to this:

Revertomatic Revert

Field Value
Original PR #976
JIRA TRT-2568
Payload 4.22.0-0.nightly-2026-03-08-004901
Confidence 95 (HIGH)

Why

This PR is causing blocking job failures in the 4.22 nightly amd64 payload. All serial and techpreview-serial blocking jobs fail with:

[sig-instrumentation][Late] Platform Prometheus targets should not be accessible without auth [Serial]

Error: failed to ensure scraping target of pod openshift-cloud-credential-operator/cco-metrics requires auth: context deadline exceeded

The PR removed kube-rbac-proxy from the CCO metrics endpoint and reimplemented RBAC via controller-runtime. The Prometheus auth test checks that metrics endpoints require authentication, and the CCO metrics endpoint now fails this check.

Failing Jobs

  • aws-ovn-serial-1of2
  • aws-ovn-serial-2of2
  • aws-ovn-techpreview-serial-1of3
  • aws-ovn-techpreview-serial-2of3
  • aws-ovn-techpreview-serial-3of3
  • overall-analysis-all (downstream)

cc @jstuever

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@stbenjam stbenjam changed the title Revert "CCO-788: Remove kube-rbac-proxy container from metrics" TRT-2568: Revert "CCO-788: Remove kube-rbac-proxy container from metrics" Mar 8, 2026
@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Mar 8, 2026

@stbenjam: This pull request references TRT-2568 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Revertomatic Revert

Field Value
Original PR #976
JIRA TRT-2568
Payload 4.22.0-0.nightly-2026-03-08-004901
Confidence 95 (HIGH)

Why

This PR is causing blocking job failures in the 4.22 nightly amd64 payload. All serial and techpreview-serial blocking jobs fail with:

[sig-instrumentation][Late] Platform Prometheus targets should not be accessible without auth [Serial]

Error: failed to ensure scraping target of pod openshift-cloud-credential-operator/cco-metrics requires auth: context deadline exceeded

The PR removed kube-rbac-proxy from the CCO metrics endpoint and reimplemented RBAC via controller-runtime. The Prometheus auth test checks that metrics endpoints require authentication, and the CCO metrics endpoint now fails this check.

Failing Jobs

  • aws-ovn-serial-1of2
  • aws-ovn-serial-2of2
  • aws-ovn-techpreview-serial-1of3
  • aws-ovn-techpreview-serial-2of3
  • aws-ovn-techpreview-serial-3of3
  • overall-analysis-all (downstream)

cc @jstuever

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@stbenjam
Copy link
Member Author

stbenjam commented Mar 8, 2026

/payload-job periodic-ci-openshift-release-main-ci-4.22-e2e-aws-ovn-techpreview-serial-1of3

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 8, 2026

@stbenjam: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-4.22-e2e-aws-ovn-techpreview-serial-1of3

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/60ef6be0-1af1-11f1-9139-ca1df5029cef-0

@codecov
Copy link

codecov bot commented Mar 8, 2026

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 46.20%. Comparing base (6b2f075) to head (42d7c14).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
pkg/cmd/operator/cmd.go 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #982      +/-   ##
==========================================
+ Coverage   46.19%   46.20%   +0.01%     
==========================================
  Files          98       98              
  Lines       12256    12253       -3     
==========================================
  Hits         5662     5662              
+ Misses       5944     5941       -3     
  Partials      650      650              
Files with missing lines Coverage Δ
pkg/cmd/operator/cmd.go 0.00% <0.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@neisw
Copy link
Contributor

neisw commented Mar 8, 2026

/lgtm

Resolves the [sig-instrumentation][Late] Platform Prometheus targets should not be accessible without auth [Serial] [Suite:openshift/conformance/serial] failures

@neisw
Copy link
Contributor

neisw commented Mar 8, 2026

/verified by ci

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 8, 2026
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 8, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: neisw, stbenjam
Once this PR has been reviewed and has the lgtm label, please assign jstuever for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Mar 8, 2026
@openshift-ci-robot
Copy link
Contributor

@neisw: This PR has been marked as verified by ci.

Details

In response to this:

/verified by ci

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@neisw
Copy link
Contributor

neisw commented Mar 8, 2026

/label approved

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 8, 2026
@stbenjam
Copy link
Member Author

stbenjam commented Mar 8, 2026

/payload-job periodic-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-serial-1of2
/payload-job periodic-ci-openshift-release-main-ci-4.22-e2e-aws-ovn-techpreview-serial-1of3

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 8, 2026

@stbenjam: trigger 2 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-nightly-4.22-e2e-aws-ovn-serial-1of2
  • periodic-ci-openshift-release-main-ci-4.22-e2e-aws-ovn-techpreview-serial-1of3

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/784a76a0-1b22-11f1-8f57-10277777bf8d-0

@stbenjam
Copy link
Member Author

stbenjam commented Mar 8, 2026

/payload-job abort

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 8, 2026

@stbenjam: trigger 0 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

@stbenjam
Copy link
Member Author

stbenjam commented Mar 8, 2026

Hypershift is hitting another regression with CVO and AWS looks like a flake

/override ci/prow/e2e-hypershift
/override ci/prow/e2e-aws-ovn

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 8, 2026

@stbenjam: Overrode contexts on behalf of stbenjam: ci/prow/e2e-aws-ovn, ci/prow/e2e-hypershift

Details

In response to this:

Hypershift is hitting another regression with CVO and AWS looks like a flake

/override ci/prow/e2e-hypershift
/override ci/prow/e2e-aws-ovn

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-merge-bot openshift-merge-bot bot merged commit e338ba6 into openshift:master Mar 8, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants