Skip to content

Add commit status validation test#30987

Open
stbenjam wants to merge 1 commit intoopenshift:mainfrom
stbenjam:fail-commit-status
Open

Add commit status validation test#30987
stbenjam wants to merge 1 commit intoopenshift:mainfrom
stbenjam:fail-commit-status

Conversation

@stbenjam
Copy link
Copy Markdown
Member

@stbenjam stbenjam commented Apr 9, 2026

Summary

  • Adds a new [sig-arch] Ginkgo test that validates all pods are running
  • Adds .coderabbit.yaml configuration with fail_commit_status: true

Test plan

  • Verify CodeRabbit reviews the PR and sets a failing commit status

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 9, 2026

Walkthrough

Added a .coderabbit.yaml configuration file to enable inheritance and set reviews.fail_commit_status to true. Added a Ginkgo test that validates all pods across the cluster are in Running phase using in-cluster Kubernetes configuration.

Changes

Cohort / File(s) Summary
Configuration
.coderabbit.yaml
New configuration file enabling inheritance and setting reviews.fail_commit_status to true.
Test Suite
test/extended/operators/commit_status.go
New Ginkgo test suite validating pod status; loads in-cluster config, creates a context with 1-second timeout, lists all pods, and asserts all pods are in Running phase.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Apr 9, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: stbenjam

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 9, 2026
@openshift-ci openshift-ci bot requested review from deads2k and p0lyn0mial April 9, 2026 13:51
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@test/extended/operators/commit_status.go`:
- Around line 24-29: The context timeout for the cluster-wide pod listing is too
short and cancel is called immediately; change the call creating the context in
the code that calls context.WithTimeout (the ctx, cancel :=
context.WithTimeout(...) used before client.CoreV1().Pods("").List(...)) to use
a longer, realistic duration (e.g., several seconds) and replace the immediate
cancel() call with defer cancel() so the context is cancelled after the List
call completes; update the timeout constant and use ctx in Pods().List as-is to
avoid flaky deadline failures.
- Around line 11-13: Replace the direct use of rest.InClusterConfig() and manual
kubernetes.NewForConfig(...) with the test framework helper e2e.LoadClientset();
update the imports to include the e2e test loader used in this suite, remove the
InClusterConfig call and acquisition of a raw *rest.Config, and call
e2e.LoadClientset() (or the suite's equivalent loader) to obtain the kubernetes
clientset used by the tests (referencing rest.InClusterConfig,
kubernetes.NewForConfig, and the test's client variable to locate the change).
- Line 10: The pod health check currently asserts pods are strictly in Running
and fails on Succeeded pods; update the check that inspects pod.Status.Phase
(and any assertion/expectation around it) to accept both corev1.PodRunning and
corev1.PodSucceeded as healthy and only fail for other phases (e.g.,
Pending/Failed/Unknown). Locate the loop or function performing the phase check
(references to pod.Status.Phase, PodPhase, or variables named pod/pods) and
change the condition to: if phase != corev1.PodRunning && phase !=
corev1.PodSucceeded { fail } so completed job pods are treated as healthy.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: b133e84f-04ab-454f-bf2a-bbf9aa8c4051

📥 Commits

Reviewing files that changed from the base of the PR and between 7da3e1c and 29be4d8.

📒 Files selected for processing (2)
  • .coderabbit.yaml
  • test/extended/operators/commit_status.go

g "github.com/onsi/ginkgo/v2"
o "github.com/onsi/gomega"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# First, let's find and examine the file
find . -name "commit_status.go" -type f

Repository: openshift/origin

Length of output: 102


🏁 Script executed:

# Read the commit_status.go file
cat -n ./test/extended/operators/commit_status.go

Repository: openshift/origin

Length of output: 1361


🏁 Script executed:

# Let's check the cluster.go file mentioned in the scratchpad for context
find . -name "cluster.go" -path "*/test/extended/operators/*" -type f

Repository: openshift/origin

Length of output: 96


🏁 Script executed:

# Read the relevant section of cluster.go to see how pod phases are handled there
sed -n '56,85p' ./test/extended/operators/cluster.go

Repository: openshift/origin

Length of output: 951


Fix overly strict pod health assertion that fails on legitimate Succeeded pods.

Lines 31-40 require all pods to be in Running state, which will false-fail on healthy clusters with completed pods in Succeeded state (e.g., job completions). Accept both Running and Succeeded phases, and fail only on unhealthy states.

Suggested fix
 import (
 	"context"
 	"time"
 
 	g "github.com/onsi/ginkgo/v2"
 	o "github.com/onsi/gomega"
 
+	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/client-go/kubernetes"
 	"k8s.io/client-go/rest"
 )
 
 var _ = g.Describe("[sig-arch] Commit status validation", func() {
 	defer g.GinkgoRecover()
 
 	g.It("should validate all pods are running", func() {
 		cfg, err := rest.InClusterConfig()
 		o.Expect(err).NotTo(o.HaveOccurred())
 		client, err := kubernetes.NewForConfig(cfg)
 		o.Expect(err).NotTo(o.HaveOccurred())
 
 		ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
 
 		pods, err := client.CoreV1().Pods("").List(ctx, metav1.ListOptions{})
 		o.Expect(err).NotTo(o.HaveOccurred())
 
 		cancel()
 
-		totalPods := 0
-		runningPods := 0
-		for i := 0; i < len(pods.Items); i++ {
-			totalPods = totalPods + 1
-			if pods.Items[i].Status.Phase == "Running" {
-				runningPods = runningPods + 1
-			}
-		}
-
-		o.Expect(runningPods).To(o.Equal(totalPods), "all pods should be running")
+		unhealthyPods := 0
+		for i := range pods.Items {
+			switch pods.Items[i].Status.Phase {
+			case corev1.PodRunning, corev1.PodSucceeded:
+				continue
+			default:
+				unhealthyPods++
+			}
+		}
+
+		o.Expect(unhealthyPods).To(o.Equal(0), "all pods should be healthy")
 	})
 })
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/extended/operators/commit_status.go` at line 10, The pod health check
currently asserts pods are strictly in Running and fails on Succeeded pods;
update the check that inspects pod.Status.Phase (and any assertion/expectation
around it) to accept both corev1.PodRunning and corev1.PodSucceeded as healthy
and only fail for other phases (e.g., Pending/Failed/Unknown). Locate the loop
or function performing the phase check (references to pod.Status.Phase,
PodPhase, or variables named pod/pods) and change the condition to: if phase !=
corev1.PodRunning && phase != corev1.PodSucceeded { fail } so completed job pods
are treated as healthy.

Comment on lines +11 to +13
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

# Check if the file exists and examine its content
ls -la test/extended/operators/commit_status.go

Repository: openshift/origin

Length of output: 149


🏁 Script executed:

# Look at the actual content around the import lines and the code in question
head -30 test/extended/operators/commit_status.go | cat -n

Repository: openshift/origin

Length of output: 995


🏁 Script executed:

# Search for other uses of e2e.LoadClientset() in the test suite to confirm it's a common pattern
rg "e2e\.LoadClientset\(\)" test/ --type go -A 2 -B 2

Repository: openshift/origin

Length of output: 4360


🏁 Script executed:

# Check how InClusterConfig is used elsewhere in test files
rg "rest\.InClusterConfig\(\)" test/ --type go -A 2 -B 2

Repository: openshift/origin

Length of output: 443


🏁 Script executed:

# Verify the e2e framework import path exists
find . -path "*/test/e2e/framework*" -type d | head -5

Repository: openshift/origin

Length of output: 325


🏁 Script executed:

# Check the exact import statement for e2e in files that use LoadClientset
head -20 test/extended/operators/cluster.go | cat -n

Repository: openshift/origin

Length of output: 588


🏁 Script executed:

# Also check operators.go for the e2e import
head -20 test/extended/operators/operators.go | cat -n

Repository: openshift/origin

Length of output: 619


🏁 Script executed:

# Check what's imported in operators.go more completely
grep -n "^[[:space:]]*e2e" test/extended/operators/operators.go | head -5

Repository: openshift/origin

Length of output: 597


🏁 Script executed:

# Check if there are any other imports needed besides the e2e framework import
grep -n "import" test/extended/operators/operators.go -A 30 | head -40

Repository: openshift/origin

Length of output: 990


🏁 Script executed:

# Check what LoadClientset returns to ensure the suggested fix is correct
rg "func LoadClientset" vendor/k8s.io/kubernetes/test/e2e/framework/ -A 5

Repository: openshift/origin

Length of output: 612


🏁 Script executed:

# Verify the suggested diff imports are correct by looking at the actual return type
rg "LoadClientset.*\(" vendor/k8s.io/kubernetes/test/e2e/framework/ -B 2 -A 10 | head -30

Repository: openshift/origin

Length of output: 1197


Use the standard e2e client loader instead of InClusterConfig().

This setup can fail when the test runs outside a pod. In this suite, the common pattern is e2e.LoadClientset(), which is more robust for CI execution contexts.

Suggested fix
 import (
 	"context"
 	"time"
 
 	g "github.com/onsi/ginkgo/v2"
 	o "github.com/onsi/gomega"
 
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	"k8s.io/client-go/kubernetes"
-	"k8s.io/client-go/rest"
+	e2e "k8s.io/kubernetes/test/e2e/framework"
 )
@@
-		cfg, err := rest.InClusterConfig()
-		o.Expect(err).NotTo(o.HaveOccurred())
-		client, err := kubernetes.NewForConfig(cfg)
+		client, err := e2e.LoadClientset()
 		o.Expect(err).NotTo(o.HaveOccurred())

Also applies to: 19-22

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/extended/operators/commit_status.go` around lines 11 - 13, Replace the
direct use of rest.InClusterConfig() and manual kubernetes.NewForConfig(...)
with the test framework helper e2e.LoadClientset(); update the imports to
include the e2e test loader used in this suite, remove the InClusterConfig call
and acquisition of a raw *rest.Config, and call e2e.LoadClientset() (or the
suite's equivalent loader) to obtain the kubernetes clientset used by the tests
(referencing rest.InClusterConfig, kubernetes.NewForConfig, and the test's
client variable to locate the change).

Comment on lines +24 to +29
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)

pods, err := client.CoreV1().Pods("").List(ctx, metav1.ListOptions{})
o.Expect(err).NotTo(o.HaveOccurred())

cancel()
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

The 1-second timeout is too short for cluster-wide pod listing.

This is likely to introduce flaky deadline failures. Use a more realistic timeout and defer cancellation.

Suggested fix
-		ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
+		ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+		defer cancel()
@@
-		cancel()
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
pods, err := client.CoreV1().Pods("").List(ctx, metav1.ListOptions{})
o.Expect(err).NotTo(o.HaveOccurred())
cancel()
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
pods, err := client.CoreV1().Pods("").List(ctx, metav1.ListOptions{})
o.Expect(err).NotTo(o.HaveOccurred())
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/extended/operators/commit_status.go` around lines 24 - 29, The context
timeout for the cluster-wide pod listing is too short and cancel is called
immediately; change the call creating the context in the code that calls
context.WithTimeout (the ctx, cancel := context.WithTimeout(...) used before
client.CoreV1().Pods("").List(...)) to use a longer, realistic duration (e.g.,
several seconds) and replace the immediate cancel() call with defer cancel() so
the context is cancelled after the List call completes; update the timeout
constant and use ctx in Pods().List as-is to avoid flaky deadline failures.

@stbenjam
Copy link
Copy Markdown
Member Author

stbenjam commented Apr 9, 2026

@coderabbitai Why is your github status check green? Is my coderabbit config wrong?

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 9, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check the current PR status checks
gh pr checks 30987 --repo openshift/origin

# Also check the .coderabbit.yaml content
cat .coderabbit.yaml

Length of output: 3108


🛠 Debug Information Gathered


@stbenjam: Your .coderabbit.yaml config is correct and is being read properly — fail_commit_status: true is active. However, the behavior may not be what you're expecting.

According to the schema, fail_commit_status does this:

"Set the commit status to 'failure' when the PR cannot be reviewed by CodeRabbit for any reason."

This means it only sets the commit status to failure when CodeRabbit fails to complete the review (e.g., an internal error, access problem, timeout, etc.). Since the review completed successfully (Review completed), the status is reported as pass — even if CodeRabbit left review comments.

In short: fail_commit_status: true ≠ "fail if CodeRabbit found issues". It's a safety net for when the review itself errors out.


If your goal is to block merging based on review findings, you'd want to look at these settings instead:

reviews:
  request_changes_workflow: true   # CodeRabbit will "request changes" until its comments are resolved
  pre_merge_checks:
    title:
      mode: error
    description:
      mode: error
    custom_checks:
      - name: "My Check"
        mode: error
        instructions: "..."

With request_changes_workflow: true, CodeRabbit will set a "changes requested" review state, which can block merging if branch protection rules require resolved reviews.

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Apr 9, 2026

@stbenjam: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-vsphere-ovn 29be4d8 link true /test e2e-vsphere-ovn
ci/prow/e2e-metal-ipi-ovn-ipv6 29be4d8 link true /test e2e-metal-ipi-ovn-ipv6
ci/prow/e2e-gcp-ovn 29be4d8 link true /test e2e-gcp-ovn
ci/prow/e2e-aws-ovn-fips 29be4d8 link true /test e2e-aws-ovn-fips
ci/prow/e2e-vsphere-ovn-upi 29be4d8 link true /test e2e-vsphere-ovn-upi
ci/prow/e2e-aws-ovn-microshift 29be4d8 link true /test e2e-aws-ovn-microshift
ci/prow/e2e-gcp-csi 29be4d8 link true /test e2e-gcp-csi

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-trt
Copy link
Copy Markdown

openshift-trt bot commented Apr 9, 2026

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New Test Risks for sha: 29be4d8

Job Name New Test Risk
pull-ci-openshift-origin-main-e2e-aws-ovn-fips High - "[sig-arch] Commit status validation should validate all pods are running [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn-microshift High - "[sig-arch] Commit status validation should validate all pods are running [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-gcp-ovn High - "[sig-arch] Commit status validation should validate all pods are running [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6 High - "[sig-arch] Commit status validation should validate all pods are running [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-vsphere-ovn-upi High - "[sig-arch] Commit status validation should validate all pods are running [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit

New tests seen in this PR at sha: 29be4d8

  • "[sig-arch] Commit status validation should validate all pods are running [Suite:openshift/conformance/parallel]" [Total: 5, Pass: 0, Fail: 5, Flake: 0]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant