Skip to content

Add Quay 3.18, 3.16, 3.15, 3.14 upgrade testing on OCP 4.21#78885

Open
LiZhang19817 wants to merge 2 commits intoopenshift:mainfrom
LiZhang19817:add-quay318-316-315-upgrade-testing
Open

Add Quay 3.18, 3.16, 3.15, 3.14 upgrade testing on OCP 4.21#78885
LiZhang19817 wants to merge 2 commits intoopenshift:mainfrom
LiZhang19817:add-quay318-316-315-upgrade-testing

Conversation

@LiZhang19817
Copy link
Copy Markdown
Contributor

@LiZhang19817 LiZhang19817 commented May 6, 2026

Summary

  • Add Quay 3.18, 3.16, 3.15, and 3.14 upgrade test jobs on OCP 4.21
  • Rename existing upgrade test to quay317-ocp421-upgrade for consistency
  • Each job uses the corresponding FBC operator catalog index image and Quay operator channel

Test plan

  • Verify generated Prow jobs are correct via make update
  • Rehearse jobs pass CI validation

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Expanded OCP 4.21 upgrade test matrix to include Quay versions 3.13, 3.14, 3.15, 3.16, 3.17 and 3.18.
    • Replaced a generic upgrade entry with explicit per-version upgrade entries for clearer coverage.
    • Updated existing upgrade entries for accuracy.
    • Removed obsolete CSO and QBO-specific upgrade test configurations to streamline the suite.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 6, 2026

Walkthrough

Added explicit OCP 4.21 upgrade test entries for Quay versions 3.18, 3.17 (updated image SHA), 3.16, 3.15, 3.14, and 3.13; removed CSO and QBO-specific upgrade test blocks. All changes are confined to the Quay upgrade CI config file.

Changes

OCP 4.21 Quay Upgrade Test Matrix

Layer / File(s) Summary
New upgrade entries
ci-operator/config/quay/quay-tests/quay-quay-tests-master__quay-upgrade.yaml
Added quay318-ocp421-upgrade, quay316-ocp421-upgrade, quay315-ocp421-upgrade, quay314-ocp421-upgrade, and quay313-ocp421-upgrade blocks with QUAY_VERSION, QUAY_OPERATOR_CHANNEL, MULTISTAGE_PARAM_OVERRIDE_QUAY_INDEX_IMAGE, and test refs.
Updated existing entry
ci-operator/config/quay/quay-tests/quay-quay-tests-master__quay-upgrade.yaml
Updated quay317-ocp421-upgrade block with a new image SHA while preserving its upgrade configuration for Quay 3.17.
Cleanup / Removals
ci-operator/config/quay/quay-tests/quay-quay-tests-master__quay-upgrade.yaml
Removed cso-quay-ocp421-upgrade and qbo-quay-ocay-ocp421-upgrade blocks and their environment/test definitions.

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 12
✅ Passed checks (12 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and specifically describes the main change: adding Quay 3.18, 3.16, 3.15, and 3.14 upgrade testing on OCP 4.21, which aligns with the primary objective of expanding the upgrade test matrix across multiple Quay versions.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed Custom check is not applicable. PR modifies only YAML CI configuration files, not Go test code. No Ginkgo test definitions (It(), Describe(), etc.) are present or modified in this PR.
Test Structure And Quality ✅ Passed Custom check for Ginkgo test code quality is not applicable. PR modifies only YAML CI/CD configuration files, not Ginkgo test code.
Microshift Test Compatibility ✅ Passed Custom check is not applicable. This PR modifies only CI operator configuration (YAML), not Ginkgo e2e test code. The check applies only when new Ginkgo tests (It, Describe, Context, When) are added.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PR modifies only Prow/CI job configuration YAML, not Ginkgo e2e test code. The custom check applies specifically to new Ginkgo tests (It(), Describe(), etc.), which are not present in this change.
Topology-Aware Scheduling Compatibility ✅ Passed PR modifies only ci-operator test configuration, not deployment manifests or operator code. No scheduling constraints, affinity rules, or topology-unfriendly assumptions introduced.
Ote Binary Stdout Contract ✅ Passed OTE Binary Stdout Contract check is not applicable. This PR contains only YAML CI configuration changes with no Go source code modifications or OTE binary implementations.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The check applies only to new Ginkgo e2e tests (It(), Describe(), Context(), When()). This PR modifies only CI-Operator YAML job configuration files, not new Ginkgo test code.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 6, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: LiZhang19817

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 6, 2026
@openshift-ci openshift-ci Bot requested review from deepsm007 and pruan-rht May 6, 2026 05:47
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/config/quay/quay-tests/quay-quay-tests-master__quay-upgrade.yaml`:
- Line 46: The MULTISTAGE_PARAM_OVERRIDE_QUAY_INDEX_IMAGE variable is pinned to
the same sha256 across multiple versioned jobs, collapsing the version matrix;
update each occurrence of MULTISTAGE_PARAM_OVERRIDE_QUAY_INDEX_IMAGE used for
different upgrade-source jobs (the four entries currently on lines corresponding
to the four versioned jobs) so each versioned index image is pinned to its own
matching digest (or remove the `@sha256` digest if you prefer tag-based selection)
ensuring each job references a distinct catalog image.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 21db2019-6042-45f6-933b-be1ede65b853

📥 Commits

Reviewing files that changed from the base of the PR and between 0fe1b46 and 5fa118c.

⛔ Files ignored due to path filters (1)
  • ci-operator/jobs/quay/quay-tests/quay-quay-tests-master-periodics.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (1)
  • ci-operator/config/quay/quay-tests/quay-quay-tests-master__quay-upgrade.yaml

Comment thread ci-operator/config/quay/quay-tests/quay-quay-tests-master__quay-upgrade.yaml Outdated
…annel from 3.15/3.14

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@openshift-merge-bot
Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@LiZhang19817: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
periodic-ci-quay-quay-tests-master-quay-upgrade-quay318-ocp421-upgrade N/A periodic Periodic changed
periodic-ci-quay-quay-tests-master-quay-upgrade-quay316-ocp421-upgrade N/A periodic Periodic changed
periodic-ci-quay-quay-tests-master-quay-upgrade-quay313-ocp421-upgrade N/A periodic Periodic changed
periodic-ci-quay-quay-tests-master-quay-upgrade-quay314-ocp421-upgrade N/A periodic Periodic changed
periodic-ci-quay-quay-tests-master-quay-upgrade-quay315-ocp421-upgrade N/A periodic Periodic changed
periodic-ci-quay-quay-tests-master-quay-upgrade-quay317-ocp421-upgrade N/A periodic Periodic changed
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/config/quay/quay-tests/quay-quay-tests-master__quay-upgrade.yaml`:
- Around line 95-112: The quay315-ocp421-upgrade (and similarly
quay314-ocp421-upgrade) job includes the
quay-tests-resource-provisioning-storage-odf step but omits
ODF_OPERATOR_CHANNEL, causing a silent fallback to the wrong channel; fix by
adding ODF_OPERATOR_CHANNEL: stable-4.21 to the job's env block (the env under
quay315-ocp421-upgrade) so the quay-tests-resource-provisioning-storage-odf step
uses the correct OCP 4.21 channel, or alternatively remove the
quay-tests-resource-provisioning-storage-odf ref from the job if ODF
provisioning isn't needed for these Quay versions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 759f7096-6756-4044-a29a-b061eca2343f

📥 Commits

Reviewing files that changed from the base of the PR and between 5fa118c and d7274a3.

⛔ Files ignored due to path filters (1)
  • ci-operator/jobs/quay/quay-tests/quay-quay-tests-master-periodics.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (1)
  • ci-operator/config/quay/quay-tests/quay-quay-tests-master__quay-upgrade.yaml

Comment on lines +95 to +112
- as: quay315-ocp421-upgrade
cron: '@yearly'
steps:
cluster_profile: aws-quay-qe
env:
BASE_DOMAIN: quayqe.devcluster.openshift.com
COMPUTE_NODE_REPLICAS: "6"
COMPUTE_NODE_TYPE: m6a.8xlarge
MULTISTAGE_PARAM_OVERRIDE_QUAY_INDEX_IMAGE: quay.io/redhat-user-workloads/quay-eng-tenant/stable-3-15-v4-21@sha256:9229c544bd8f93345c85bdd1facd9bdec9155c883a7476e3d249d304520acc2d
QUAY_OPERATOR_CHANNEL: stable-3.15
QUAY_OPERATOR_SOURCE: fbc-operator-catalog
QUAY_UPGRADE_TESTCASE: Quay-Upgrade-High|Quay-Upgrade-Medium
QUAY_VERSION: "3.15"
test:
- ref: quay-tests-enable-quay-catalogsource
- ref: quay-tests-resource-provisioning-storage-odf
- ref: quay-tests-test-quay-upgrade
workflow: cucushift-installer-rehearse-aws-ipi
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Locate the step definition and check for ODF_OPERATOR_CHANNEL usage
fd -t f 'quay-tests-resource-provisioning-storage-odf' --exec cat {}
rg -n 'ODF_OPERATOR_CHANNEL' --type yaml -C3

Repository: openshift/release

Length of output: 50374


ODF_OPERATOR_CHANNEL missing from quay315 and quay314 jobs; will silently fall back to incorrect default.

Both quay315-ocp421-upgrade and quay314-ocp421-upgrade include the quay-tests-resource-provisioning-storage-odf step but omit ODF_OPERATOR_CHANNEL. While the step has a built-in default of stable-4.17, this is mismatched for OCP 4.21. All other jobs in the file explicitly set ODF_OPERATOR_CHANNEL: stable-4.21, and omitting it from these two jobs will cause the ODF operator to deploy an older, incompatible channel without any error or warning.

Either add ODF_OPERATOR_CHANNEL: stable-4.21 to both jobs, or remove the quay-tests-resource-provisioning-storage-odf ref if ODF is not required for these older Quay versions on OCP 4.21.

🛠️ Option A – restore the missing channel variable
 - as: quay315-ocp421-upgrade
   ...
     env:
       ...
       MULTISTAGE_PARAM_OVERRIDE_QUAY_INDEX_IMAGE: quay.io/redhat-user-workloads/quay-eng-tenant/stable-3-15-v4-21@sha256:9229c544bd8f93345c85bdd1facd9bdec9155c883a7476e3d249d304520acc2d
+      ODF_OPERATOR_CHANNEL: stable-4.21
       QUAY_OPERATOR_CHANNEL: stable-3.15

 - as: quay314-ocp421-upgrade
   ...
     env:
       ...
       MULTISTAGE_PARAM_OVERRIDE_QUAY_INDEX_IMAGE: quay.io/redhat-user-workloads/quay-eng-tenant/stable-3-14-v4-21@sha256:3002040ed207ad7b3b9b7ae862a04a082fe17c7e199a8c62163f2dfe09277122
+      ODF_OPERATOR_CHANNEL: stable-4.21
       QUAY_OPERATOR_CHANNEL: stable-3.14
🛠️ Option B – remove the ODF provisioning step if ODF is not required
 - as: quay315-ocp421-upgrade
   ...
     test:
     - ref: quay-tests-enable-quay-catalogsource
-    - ref: quay-tests-resource-provisioning-storage-odf
     - ref: quay-tests-test-quay-upgrade

 - as: quay314-ocp421-upgrade
   ...
     test:
     - ref: quay-tests-enable-quay-catalogsource
-    - ref: quay-tests-resource-provisioning-storage-odf
     - ref: quay-tests-test-quay-upgrade
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- as: quay315-ocp421-upgrade
cron: '@yearly'
steps:
cluster_profile: aws-quay-qe
env:
BASE_DOMAIN: quayqe.devcluster.openshift.com
COMPUTE_NODE_REPLICAS: "6"
COMPUTE_NODE_TYPE: m6a.8xlarge
MULTISTAGE_PARAM_OVERRIDE_QUAY_INDEX_IMAGE: quay.io/redhat-user-workloads/quay-eng-tenant/stable-3-15-v4-21@sha256:9229c544bd8f93345c85bdd1facd9bdec9155c883a7476e3d249d304520acc2d
QUAY_OPERATOR_CHANNEL: stable-3.15
QUAY_OPERATOR_SOURCE: fbc-operator-catalog
QUAY_UPGRADE_TESTCASE: Quay-Upgrade-High|Quay-Upgrade-Medium
QUAY_VERSION: "3.15"
test:
- ref: quay-tests-enable-quay-catalogsource
- ref: quay-tests-resource-provisioning-storage-odf
- ref: quay-tests-test-quay-upgrade
workflow: cucushift-installer-rehearse-aws-ipi
- as: quay315-ocp421-upgrade
cron: '@yearly'
steps:
cluster_profile: aws-quay-qe
env:
BASE_DOMAIN: quayqe.devcluster.openshift.com
COMPUTE_NODE_REPLICAS: "6"
COMPUTE_NODE_TYPE: m6a.8xlarge
MULTISTAGE_PARAM_OVERRIDE_QUAY_INDEX_IMAGE: quay.io/redhat-user-workloads/quay-eng-tenant/stable-3-15-v4-21@sha256:9229c544bd8f93345c85bdd1facd9bdec9155c883a7476e3d249d304520acc2d
ODF_OPERATOR_CHANNEL: stable-4.21
QUAY_OPERATOR_CHANNEL: stable-3.15
QUAY_OPERATOR_SOURCE: fbc-operator-catalog
QUAY_UPGRADE_TESTCASE: Quay-Upgrade-High|Quay-Upgrade-Medium
QUAY_VERSION: "3.15"
test:
- ref: quay-tests-enable-quay-catalogsource
- ref: quay-tests-resource-provisioning-storage-odf
- ref: quay-tests-test-quay-upgrade
workflow: cucushift-installer-rehearse-aws-ipi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci-operator/config/quay/quay-tests/quay-quay-tests-master__quay-upgrade.yaml`
around lines 95 - 112, The quay315-ocp421-upgrade (and similarly
quay314-ocp421-upgrade) job includes the
quay-tests-resource-provisioning-storage-odf step but omits
ODF_OPERATOR_CHANNEL, causing a silent fallback to the wrong channel; fix by
adding ODF_OPERATOR_CHANNEL: stable-4.21 to the job's env block (the env under
quay315-ocp421-upgrade) so the quay-tests-resource-provisioning-storage-odf step
uses the correct OCP 4.21 channel, or alternatively remove the
quay-tests-resource-provisioning-storage-odf ref from the job if ODF
provisioning isn't needed for these Quay versions.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 6, 2026

@LiZhang19817: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant