Skip to content

NO-JIRA: Move synthetic release flag from BigQuery to config#3560

Open
smg247 wants to merge 1 commit into
openshift:mainfrom
smg247:synthetic-config
Open

NO-JIRA: Move synthetic release flag from BigQuery to config#3560
smg247 wants to merge 1 commit into
openshift:mainfrom
smg247:synthetic-config

Conversation

@smg247
Copy link
Copy Markdown
Member

@smg247 smg247 commented May 27, 2026

The Synthetic bool on the BigQuery Releases table was non-nullable and forced anyone running make update-variants to have GCP service account credentials. Since the customizations file already defines which releases are synthetic, the flag now lives in the ReleaseConfig struct (synthetic: true in YAML) instead of being queried from BigQuery.

  • Add Synthetic bool to ReleaseConfig and mark all 12 synthetic releases in openshift-customizations.yaml
  • Refactor BuildSyntheticReleaseJobOverrides to use config directly, removing the BigQuery releaseConfigs parameter
  • Remove Synthetic from ReleaseRow and Release BQ types
  • Remove BigQuery/GCP flags from variants snapshot command; update-variants no longer requires credentials
  • Regenerate openshift.yaml locally with changes from sippy-config-generator: add Synthetic field to ReleaseConfig ci-tools#5210 (this will be overwritten anyways the next time the job runs after both PRs are merged)

Summary by CodeRabbit

  • New Features

    • Added ability to mark releases as synthetic directly in configuration files.
  • Refactor

    • Simplified variant snapshot generation workflow.
    • Removed BigQuery dependency from variant snapshot command.
  • Chores

    • Updated job variant registry and OpenShift configurations with new synthetic release definitions.

The Synthetic bool on the BigQuery Releases table was non-nullable and
forced anyone running `make update-variants` to have GCP service account
credentials. Since the customizations file already defines which releases
are synthetic, the flag now lives in the ReleaseConfig struct
(synthetic: true in YAML) instead of being queried from BigQuery.

- Add Synthetic bool to ReleaseConfig and mark all 12 synthetic releases
  in openshift-customizations.yaml
- Refactor BuildSyntheticReleaseJobOverrides to use config directly,
  removing the BigQuery releaseConfigs parameter
- Remove Synthetic from ReleaseRow and Release BQ types
- Remove BigQuery/GCP flags from variants snapshot command; update-variants
  no longer requires credentials
- Regenerate openshift.yaml locally with changes from
  openshift/ci-tools#5210

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 27, 2026
@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci-robot
Copy link
Copy Markdown

@smg247: This pull request explicitly references no jira issue.

Details

In response to this:

The Synthetic bool on the BigQuery Releases table was non-nullable and forced anyone running make update-variants to have GCP service account credentials. Since the customizations file already defines which releases are synthetic, the flag now lives in the ReleaseConfig struct (synthetic: true in YAML) instead of being queried from BigQuery.

  • Add Synthetic bool to ReleaseConfig and mark all 12 synthetic releases in openshift-customizations.yaml
  • Refactor BuildSyntheticReleaseJobOverrides to use config directly, removing the BigQuery releaseConfigs parameter
  • Remove Synthetic from ReleaseRow and Release BQ types
  • Remove BigQuery/GCP flags from variants snapshot command; update-variants no longer requires credentials
  • Regenerate openshift.yaml locally with changes from sippy-config-generator: add Synthetic field to ReleaseConfig ci-tools#5210 (this will be overwritten anyways the next time the job runs after both PRs are merged)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 27, 2026

Walkthrough

This PR refactors synthetic release identification from a BigQuery-dependent external list to a configuration-level boolean field. The changes move the Synthetic flag from the cached Release struct into ReleaseConfig, simplify the BuildSyntheticReleaseJobOverrides function signature, and remove all BigQuery integration from the variant snapshot command. Configuration YAML files are updated to declare synthetic releases declaratively, and all tests are updated to match the new approach.

Changes

Synthetic Release Configuration Refactor

Layer / File(s) Summary
Data Model: Synthetic Field Movement
pkg/apis/config/v1/types.go, pkg/apis/sippy/v1/types.go, pkg/api/releases.go
ReleaseConfig gains a Synthetic boolean field to mark releases not tied to a specific OCP version. The Synthetic field is removed from the cached Release and ReleaseRow structs; transformRelease no longer sets the field during release transformation.
Core Synthetic Override Algorithm
pkg/variantregistry/synthetic.go
BuildSyntheticReleaseJobOverrides signature changed to accept only releases map[string]v1.ReleaseConfig instead of also requiring a separate releaseConfigs list. Synthetic detection logic moved inline: checks releaseCfg.Synthetic boolean directly instead of looking up release names from a precomputed set. Helper function syntheticReleaseNames removed.
Command Entry Points and CLI
cmd/sippy/load.go, cmd/sippy/variants_generate.go, cmd/sippy/variants_snapshot.go, Makefile
All command callers updated to invoke BuildSyntheticReleaseJobOverrides with only the releases configuration. VariantSnapshotFlags simplified by removing BigQueryFlags and GoogleCloudFlags; snapshot command RunE now loads config directly and builds overrides from cfg.Releases without BigQuery client setup or releases fetch. Make target removes --bigquery-project argument and related variable.
Configuration Declaration of Synthetic Releases
config/openshift-customizations.yaml, config/openshift.yaml, pkg/variantregistry/snapshot.yaml
Release groups marked as synthetic: true in overlays. Azure ARO HCP and RP variant entries added with standard field values. Existing hypershift cilium-guest and disaster-recovery variants removed across OpenShift 4.19–5.0. Job enablement changed across multiple release versions, including hypershift variant renames (n1minor/n2minor/n3minor to private-guest/private-mgmt/shared-vpc-guest) and longduration job splits (1of2/2of2).
Test Updates
pkg/variantregistry/synthetic_test.go, pkg/variantregistry/ocp_test.go
Test cases updated to configure synthetic releases inline in ReleaseConfig entries using Synthetic: true and optional Regexp patterns. Separate releaseConfigs slice construction removed. Function calls to BuildSyntheticReleaseJobOverrides simplified to pass only releases argument.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (1 error, 2 warnings)

Check name Status Explanation Resolution
Sql Injection Prevention ❌ Error SQL injection vulnerability in new pkg/api/releases.go: PayloadForJobRun() concatenates unsanitized user input (jobRunID) into BigQuery SQL using fmt.Sprintf without parameterization. Use BigQuery parameterized queries with placeholders instead of fmt.Sprintf for user-supplied jobRunID parameter.
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Go Error Handling ⚠️ Warning Three error handling violations: synthetic.go:25 returns unwrapped error from AddExact; variants_snapshot.go:48,58 return unwrapped errors without fmt.Errorf context wrapping. Wrap all returned errors with fmt.Errorf using %w format. Line 25 of synthetic.go and lines 48,58 of variants_snapshot.go should add context messages.
✅ Passed checks (17 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: moving the synthetic release flag from BigQuery to configuration, which is the primary objective reflected across all file modifications.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Excessive Css In React Should Use Styles ✅ Passed PR contains only backend code (Go, YAML, Makefile); no React/JSX components present. The check for excessive inline CSS in React is not applicable.
Test Coverage For New Features ✅ Passed Pure function BuildSyntheticReleaseJobOverrides has 15 unit test cases. New ReleaseConfig.Synthetic field tested via integration tests. Command changes tested indirectly.
Single Responsibility And Clear Naming ✅ Passed PR improves SRP: VariantSnapshotFlags simplified, BuildSyntheticReleaseJobOverrides reduced from 2 to 1 parameter, Synthetic moved to ReleaseConfig. All field counts and naming clear.
Stable And Deterministic Test Names ✅ Passed All test names added/modified in this PR are stable and deterministic; they use static strings without dynamic content. The t.Run(job, ...) issue in TestVariantsSnapshot pre-existed in main.
Test Structure And Quality ✅ Passed The custom check requests review of Ginkgo test code, but this repository uses standard Go testing with testify, not Ginkgo. The check is not applicable to this PR.
Microshift Test Compatibility ✅ Passed PR adds no new Ginkgo e2e tests; it modifies a Go backend tool (sippy) using standard testing.T, not Ginkgo framework. No Ginkgo patterns found in codebase.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PR modifies the sippy tool configuration and refactors release handling; it adds no new Ginkgo e2e tests. The check is not applicable to this repository.
Topology-Aware Scheduling Compatibility ✅ Passed PR contains no deployment manifests, operator code, or Kubernetes scheduling constraints. This is a CI tooling refactor moving synthetic release config from BigQuery to local YAML configuration.
Ote Binary Stdout Contract ✅ Passed No new stdout writes in process-level code. Modified cmd files use logrus (writes to stderr) and no fmt.Print/os.Stdout calls found. PrintVersion function in unmodified version.go is pre-existing.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed This PR adds no new Ginkgo e2e tests. It only refactors configuration, build system, and unit tests. The check for IPv6/disconnected network compatibility does not apply.
No-Weak-Crypto ✅ Passed PR refactors synthetic release flag from BigQuery to config. No weak crypto (MD5, SHA1, DES, RC4, 3DES, Blowfish, ECB), custom implementations, or insecure comparisons introduced.
Container-Privileges ✅ Passed PR contains no Kubernetes manifests or container configs with privilege settings. Changes are to Go source code and CI configuration files only, not K8s pod/deployment specs.
No-Sensitive-Data-In-Logs ✅ Passed No sensitive data logging introduced. The PR removes BigQuery credential code and adds safe logging statements only: error messages with non-sensitive data and generic status messages.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from petr-muller and stbenjam May 27, 2026 17:47
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 27, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: smg247

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 27, 2026
@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 27, 2026

@smg247: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants