feat: Add Tekton Chains regression testing for OpenShift Pipelines 1.21 & 1.22 by siddardh-ra · Pull Request #78865 · openshift/release

siddardh-ra · 2026-05-05T18:27:45Z

Added comprehensive CI jobs for Tekton Chains regression testing across versions 1.21 and 1.22 with multiple load scenarios and configurations.

Jobs added (24 total):

Basic signing tests: 2500, 5000, 7500 (6 jobs)
HA-enabled tests: 2500, 5000, 7500 (6 jobs)
Performance-tuned (QBT) tests: 2500, 5000, 7500 (6 jobs)
HA + QBT combined tests: 2500, 5000, 7500 (6 jobs)

These jobs evaluate Chains Controller metrics to identify performance regressions between OpenShift Pipelines versions.

Summary by CodeRabbit

Chores
- Expanded performance testing infrastructure with new test scenarios for pipeline versions 1.21 and 1.22
- Added multiple load-testing configurations at various concurrency levels with high-availability and performance tuning parameters

…21 & 1.22 Added comprehensive CI jobs for Tekton Chains regression testing across versions 1.21 and 1.22 with multiple load scenarios and configurations. Jobs added (24 total): - Basic signing tests: 2500, 5000, 7500 (6 jobs) - HA-enabled tests: 2500, 5000, 7500 (6 jobs) - Performance-tuned (QBT) tests: 2500, 5000, 7500 (6 jobs) - HA + QBT combined tests: 2500, 5000, 7500 (6 jobs) These jobs evaluate Chains Controller metrics to identify performance regressions between OpenShift Pipelines versions. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

coderabbitai · 2026-05-05T18:28:17Z

Walkthrough

The PR adds 24 new downstream performance test jobs to the CI pipeline configuration for Tekton Bigbang scenarios across deployment versions 1.21 and 1.22, with varying concurrency scales (2500, 5000, 7500), HA controller configurations, and QBT API/threading tuning parameters.

Changes

Performance Test Job Expansion

Layer / File(s)	Summary
Base Max-Concurrency Jobs `ci-operator/config/openshift-pipelines/performance/openshift-pipelines-performance-main.yaml`	Six new `max-concurrency-downstream` jobs added for versions 1.21 and 1.22 at scales 2500, 5000, 7500 with `TEST_SCENARIOS` configuration and 8-hour timeout.
HA Variant Jobs `ci-operator/config/openshift-pipelines/performance/openshift-pipelines-performance-main.yaml`	Six new HA (High Availability) variant jobs added for versions 1.21 and 1.22 at same scales, setting `DEPLOYMENT_CHAINS_CONTROLLER_HA_REPLICAS: "10"`.
QBT Tuning Variant Jobs `ci-operator/config/openshift-pipelines/performance/openshift-pipelines-performance-main.yaml`	Six new QBT (Query-Burst-Tuning) variant jobs for versions 1.21 and 1.22 at same scales, configuring `DEPLOYMENT_CHAINS_KUBE_API_BURST`, `DEPLOYMENT_CHAINS_KUBE_API_QPS`, and `DEPLOYMENT_CHAINS_THREADS_PER_CONTROLLER`.
HA+QBT Combined Variant Jobs `ci-operator/config/openshift-pipelines/performance/openshift-pipelines-performance-main.yaml`	Six new combined HA-QBT variant jobs for versions 1.21 and 1.22 at same scales, applying both HA replicas and QBT tuning parameters together.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

🚥 Pre-merge checks | ✅ 12

✅ Passed checks (12 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately captures the main change: adding Tekton Chains regression testing for OpenShift Pipelines 1.21 & 1.22, which directly aligns with the PR's objective of adding 24 new CI jobs for this purpose.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	PR modifies CI/CD YAML config only. The 24 new job names are all stable and deterministic with no dynamic identifiers, timestamps, or generated values.
Test Structure And Quality	✅ Passed	This PR modifies only YAML CI operator configuration files, not Ginkgo test code. The custom check requires reviewing Ginkgo test quality, so it is not applicable.
Microshift Test Compatibility	✅ Passed	This PR modifies only CI configuration files (YAML), not test code. No Ginkgo e2e tests are added. The check is inapplicable as it applies only to new test code additions.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	The custom check is designed for Ginkgo e2e tests. This PR adds CI configuration (YAML) for load testing, not Ginkgo e2e tests. The check is not applicable.
Topology-Aware Scheduling Compatibility	✅ Passed	PR adds CI test job configurations only. No deployment manifests, operator code, or scheduling constraints are introduced.
Ote Binary Stdout Contract	✅ Passed	This PR only modifies a YAML CI configuration file, not executable source code. The OTE Binary Stdout Contract check applies to binary source code violations, not configuration files.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	PR modifies only a CI configuration YAML file. No Ginkgo e2e test code files (.go) are added. The check does not apply as it targets new Ginkgo tests.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

Generate code and open pull requests
Plan features and break down work
Investigate incidents and troubleshoot customer tickets together
Automate recurring tasks and respond to alerts with triggers
Summarize progress and report instantly

Built for teams:

Shared memory across your entire org—no repeating context
Per-thread sandboxes to safely plan and execute work
Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

openshift-ci · 2026-05-05T18:28:21Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: siddardh-ra

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~ci-operator/config/openshift-pipelines/performance/OWNERS~~ [siddardh-ra]
~~ci-operator/jobs/openshift-pipelines/performance/OWNERS~~ [siddardh-ra]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

coderabbitai

🧹 Nitpick comments (2)

ci-operator/config/openshift-pipelines/performance/openshift-pipelines-performance-main.yaml (2)
1466-1478: 💤 Low value

Job name suffixes (2500/5000/7500) represent derived totals, not literal TEST_SCENARIOS values.

The TEST_SCENARIOS values are 500/20, 1000/20, 1500/20 while the names say 2500, 5000, 7500. The totals are computed as TEST_SCENARIOS_first_field × TEST_NAMESPACE (e.g., 500 × 5 = 2500). This matches the pre-existing 1.14/1.15 convention but is easy to misread. A brief inline comment or a doc note explaining the naming convention would help future contributors avoid confusion.

Also applies to: 1492-1504, 1518-1530
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/openshift-pipelines/performance/openshift-pipelines-performance-main.yaml`
around lines 1466 - 1478, Add a brief inline comment near the job entries (e.g.,
next to the "as: max-concurrency-downstream-1-21-2500-sign-tekton-bigbang",
"TEST_SCENARIOS" and "TEST_NAMESPACE" keys or at the top of the
"openshift-pipelines-max-concurrency" workflow block) explaining that the job
name suffix (2500/5000/7500) represents the derived total computed as
TEST_SCENARIOS_first_field × TEST_NAMESPACE (for example, "500/20" ×
TEST_NAMESPACE=5 → 2500), and apply the same comment to the other similar blocks
referenced (the sections around the other occurrences noted in the review) so
future readers understand the naming convention.
1466-1543: ⚡ Quick win

All 24 new jobs are on-demand only — confirm no scheduled trigger is needed.

Every new job is always_run: false / optional: true with no cron field, meaning they only run when manually triggered. This matches the existing pattern for 1.14/1.15 signing jobs, so it's likely intentional. However, since the PR goal is to detect regressions between 1.21 and 1.22, it's worth confirming that periodic automated runs are not desired here. The existing scheduled jobs for 1.21/1.22 (lines 252–391) cover the math / standard scenario, but not the Chains signing scenario.

If periodic regression detection is intended, consider adding cron expressions similar to the max-concurrency-downstream-pipelines1-21-* group.

Also applies to: 1658-1741, 1832-1927, 2024-2125
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/openshift-pipelines/performance/openshift-pipelines-performance-main.yaml`
around lines 1466 - 1543, These 24 new jobs (examples:
max-concurrency-downstream-1-21-2500-sign-tekton-bigbang,
max-concurrency-downstream-1-22-7500-sign-tekton-bigbang) are currently
manual-only (always_run: false, optional: true, no cron); decide whether they
should run periodically for regression detection and if so add a cron schedule
to each job (and/or flip always_run as your CI pattern requires) using the same
cron expression and scheduling conventions used by the existing
max-concurrency-downstream-pipelines1-21-* scheduled group so the
signing-tr-tekton-bigbang scenarios run automatically.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In
`@ci-operator/config/openshift-pipelines/performance/openshift-pipelines-performance-main.yaml`:
- Around line 1466-1478: Add a brief inline comment near the job entries (e.g.,
next to the "as: max-concurrency-downstream-1-21-2500-sign-tekton-bigbang",
"TEST_SCENARIOS" and "TEST_NAMESPACE" keys or at the top of the
"openshift-pipelines-max-concurrency" workflow block) explaining that the job
name suffix (2500/5000/7500) represents the derived total computed as
TEST_SCENARIOS_first_field × TEST_NAMESPACE (for example, "500/20" ×
TEST_NAMESPACE=5 → 2500), and apply the same comment to the other similar blocks
referenced (the sections around the other occurrences noted in the review) so
future readers understand the naming convention.
- Around line 1466-1543: These 24 new jobs (examples:
max-concurrency-downstream-1-21-2500-sign-tekton-bigbang,
max-concurrency-downstream-1-22-7500-sign-tekton-bigbang) are currently
manual-only (always_run: false, optional: true, no cron); decide whether they
should run periodically for regression detection and if so add a cron schedule
to each job (and/or flip always_run as your CI pattern requires) using the same
cron expression and scheduling conventions used by the existing
max-concurrency-downstream-pipelines1-21-* scheduled group so the
signing-tr-tekton-bigbang scenarios run automatically.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 3b122790-57b2-4d34-801f-0c706a534f73

📥 Commits

Reviewing files that changed from the base of the PR and between a1eadb1 and 777bc47.

⛔ Files ignored due to path filters (1)

ci-operator/jobs/openshift-pipelines/performance/openshift-pipelines-performance-main-presubmits.yaml is excluded by !ci-operator/jobs/**

📒 Files selected for processing (1)

ci-operator/config/openshift-pipelines/performance/openshift-pipelines-performance-main.yaml

siddardh-ra · 2026-05-05T18:35:55Z

/pj-rehearse pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tekton-bigbang pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tekton-bigbang-ha pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tkn-bigbang-ha-qbt pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tkn-bigbang-qbt pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tekton-bigbang pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tekton-bigbang-ha pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tkn-bigbang-ha-qbt pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tkn-bigbang-qbt

openshift-merge-bot · 2026-05-05T18:35:58Z

@siddardh-ra: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

siddardh-ra · 2026-05-05T18:45:17Z

/pj-rehearse pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-7500-sign-tkn-bigbang-ha-qbt

openshift-merge-bot · 2026-05-05T18:45:20Z

@siddardh-ra: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

siddardh-ra · 2026-05-05T18:47:07Z

/ok-to-test

siddardh-ra · 2026-05-05T18:51:18Z

/pj-rehearse pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tekton-bigbang pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tekton-bigbang-ha pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tkn-bigbang-ha-qbt pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tkn-bigbang-qbt pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tekton-bigbang pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tekton-bigbang-ha pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tkn-bigbang-ha-qbt pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tkn-bigbang-qbt

openshift-merge-bot · 2026-05-05T18:51:20Z

@siddardh-ra: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

siddardh-ra · 2026-05-05T18:59:03Z

/test ?

openshift-ci · 2026-05-05T18:59:40Z

@siddardh-ra: The following commands are available to trigger required jobs:

/test app-ci-config-dry

/test boskos-config

/test boskos-config-generation

/test build03-dry

/test build04-dry

/test build05-dry

/test build06-dry

/test build07-dry

/test build08-dry

/test build09-dry

/test build10-dry

/test build11-dry

/test check-gh-automation

/test check-gh-automation-tide

/test check-trigger-trusted-apps

/test ci-operator-config

/test ci-operator-config-metadata

/test ci-operator-registry

/test ci-secret-bootstrap-config-validation

/test ci-testgrid-allow-list

/test clusterimageset-validate

/test config

/test core-ci-config-dry

/test core-valid

/test generated-config

/test generated-dashboards

/test hosted-mgmt-dry

/test image-mirroring-config-validation

/test jira-lifecycle-config

/test labels

/test openshift-image-mirror-mappings

/test ordered-prow-config

/test owners

/test pr-reminder-config

/test prow-config

/test prow-config-filenames

/test prow-config-semantics

/test pylint

/test release-config

/test release-controller-config

/test rover-groups-config-validation

/test secret-generator-config-valid

/test services-valid

/test stackrox-stackrox-stackrox-stackrox-check

/test step-registry-metadata

/test step-registry-shellcheck

/test sync-rover-groups

/test verified-config

/test vsphere02-dry

/test yamllint

The following commands are available to trigger optional jobs:

/test build12-dry

/test check-cluster-profiles-config

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-release-check-gh-automation

pull-ci-openshift-release-main-ci-operator-config

pull-ci-openshift-release-main-ci-operator-config-metadata

pull-ci-openshift-release-main-ci-operator-registry

pull-ci-openshift-release-main-config

pull-ci-openshift-release-main-core-valid

pull-ci-openshift-release-main-generated-config

pull-ci-openshift-release-main-ordered-prow-config

pull-ci-openshift-release-main-owners

pull-ci-openshift-release-main-prow-config-filenames

pull-ci-openshift-release-main-prow-config-semantics

pull-ci-openshift-release-main-release-controller-config

pull-ci-openshift-release-openshift-image-mirror-mappings

pull-ci-openshift-release-yamllint

Details

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

siddardh-ra · 2026-05-05T19:00:42Z

/pj-rehearse pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tekton-bigbang pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tekton-bigbang-ha pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tkn-bigbang-ha-qbt pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tkn-bigbang-qbt pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tekton-bigbang pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tekton-bigbang-ha pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tkn-bigbang-ha-qbt pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tkn-bigbang-qbt

openshift-merge-bot · 2026-05-05T19:00:45Z

@siddardh-ra: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

siddardh-ra · 2026-05-06T03:49:48Z

/pj-rehearse pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tekton-bigbang pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tekton-bigbang-ha pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tkn-bigbang-ha-qbt pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-21-2500-sign-tkn-bigbang-qbt pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tekton-bigbang pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tekton-bigbang-ha pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tkn-bigbang-ha-qbt pull-ci-openshift-pipelines-performance-main-max-concurrency-downstream-1-22-2500-sign-tkn-bigbang-qbt

openshift-merge-bot · 2026-05-06T03:49:51Z

@siddardh-ra: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

openshift-ci · 2026-05-06T06:56:59Z

@siddardh-ra: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 5, 2026

openshift-ci Bot requested review from deekshith-24 and rh-rahulshetty May 5, 2026 18:28

coderabbitai Bot reviewed May 5, 2026

View reviewed changes

openshift-ci Bot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label May 5, 2026

Conversation

siddardh-ra commented May 5, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Uh oh!

openshift-ci Bot commented May 5, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

siddardh-ra commented May 5, 2026

Uh oh!

openshift-merge-bot Bot commented May 5, 2026

Uh oh!

siddardh-ra commented May 5, 2026

Uh oh!

openshift-merge-bot Bot commented May 5, 2026

Uh oh!

siddardh-ra commented May 5, 2026

Uh oh!

siddardh-ra commented May 5, 2026

Uh oh!

openshift-merge-bot Bot commented May 5, 2026

Uh oh!

siddardh-ra commented May 5, 2026

Uh oh!

openshift-ci Bot commented May 5, 2026

Uh oh!

siddardh-ra commented May 5, 2026

Uh oh!

openshift-merge-bot Bot commented May 5, 2026

Uh oh!

siddardh-ra commented May 6, 2026

Uh oh!

openshift-merge-bot Bot commented May 6, 2026

Uh oh!

openshift-ci Bot commented May 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

siddardh-ra commented May 5, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented May 5, 2026 •

edited

Loading