Skip to content

Irreconcilable configs test suite#6085

Open
proietfb wants to merge 4 commits into
openshift:mainfrom
proietfb:irreconcilable_configs_tests
Open

Irreconcilable configs test suite#6085
proietfb wants to merge 4 commits into
openshift:mainfrom
proietfb:irreconcilable_configs_tests

Conversation

@proietfb
Copy link
Copy Markdown
Member

@proietfb proietfb commented May 26, 2026

[MCO-1990] Test plan for irreconcilable changes feature

Implemented:

  • MCO-1754: scale up irreconcilable changes only
  • MCO-1745: Scale up mixed scenario
  • MCO-1755: Feature disabling reporting consistency
  • MCO-1899: Scale up and revert scenario

Summary by CodeRabbit

  • Tests
    • Added a new end-to-end test suite covering irreconcilable configuration behavior across node scaling, enabling/disabling validation overrides, RAID/storage and filesystem scenarios, and render/degrade transitions.
    • Added helper to skip SNO-specific tests and other test housekeeping/cleanups.
  • Test Fixtures
    • Added a template fixture to exercise extra-disk, RAID, and file creation storage configurations.

proietfb added 3 commits May 26, 2026 15:03
- MCO-1745: Scale up mixed scenario
- MCO-1755: Feature disabling reporting consistency
- MCO-1899: Scale up and revert scenario
@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 26, 2026

Walkthrough

This PR adds a comprehensive test suite for irreconcilable validation overrides in machine configuration, including helper utilities, YAML test fixtures, four end-to-end test cases validating feature behavior across platform configurations and node lifecycle transitions, and cleanup of existing storage tests.

Changes

Irreconcilable Validation Overrides Testing

Layer / File(s) Summary
Test helper methods
test/extended-priv/machineconfiguration.go, test/extended-priv/util.go
RemoveIrreconcilableValidationOverrides() method added to MachineConfiguration for cleanup; skipTestIfSNOCluster() helper skips tests on single-node clusters; RotateMCSCertificates() now defers cleanup of temporary remote kubeconfig.
Test fixture and platform helpers
test/extended-priv/testdata/files/extra-disks-with-files.yaml, test/extended-priv/irreconcilableChanges.go
Parameterized YAML template configures MachineConfig with RAID disk setup and files; platform-aware helpers handle AWS blockDevices vs non-AWS disks patching and device name mapping.
Irreconcilable changes test cases
test/extended-priv/irreconcilableChanges.go
Four disruptive tests: (1) enable override and verify new nodes boot clean with extra disks, (2) apply irreconcilable MC template and verify new node clean state, (3) verify changes persist after disabling feature then reach RenderDegraded, (4) verify clearing via MC reversion and new node later reporting irreconcilable again.
Storage test cleanup and refactoring
test/extended-priv/mco_storage.go, test/extended-priv/util.go
Removed old PolarionID:84219 disruptive test; refactored oc variable declaration; cleaned up matcher formatting in AWS assertions; applied whitespace normalization across test boundaries and utility helper functions.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested labels

lgtm, approved

Suggested reviewers

  • pablintino
  • cheesesashimi
🚥 Pre-merge checks | ✅ 8 | ❌ 4

❌ Failed checks (4 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Test Structure And Quality ⚠️ Warning Assertions lack meaningful failure messages; GCP device names incorrect (/dev/sda1-4 vs /dev/sdb-e); SNO-detection errors silently skip instead of failing. Add failure messages to assertions; fix GCP devices to /dev/sdb-e; change skipTestIfSNOCluster to fail on SNO-detection errors, not silent skip.
Microshift Test Compatibility ⚠️ Warning Test uses machine.openshift.io APIs (MachineSet/MachineConfig) unavailable on MicroShift without protective mechanisms like [apigroup:] tag, [Skipped:MicroShift] label, or IsMicroShiftCluster() check. Add [apigroup:machine.openshift.io] tag to line 45 test description or add exutil.IsMicroShiftCluster() check with g.Skip() in JustBeforeEach.
Single Node Openshift (Sno) Test Compatibility ⚠️ Warning The new multi-node tests have SNO protection via skipTestIfSNOCluster(), but the function incorrectly treats any IsSNOSafe() error as SNO, suppressing real failures instead of reporting them. Fix skipTestIfSNOCluster() to expect no error via o.ExpectWithOffset(1, err).NotTo(o.HaveOccurred()) and only skip if sno==true, matching the pattern in SkipTestIfWorkersCannotBeScaled().
✅ Passed checks (8 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly summarizes the main change: a new test suite for irreconcilable configs, which aligns with the primary additions in the PR (new test file, test suite, and supporting utilities).
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed All Ginkgo test names in irreconcilableChanges.go and mco_storage.go are static, deterministic strings with no dynamic content like UUIDs, timestamps, pod/node names, or computed variables.
Topology-Aware Scheduling Compatibility ✅ Passed PR adds test code only (test/extended-priv/). No production deployment manifests or pod scheduling constraints introduced. Tests skip on SNO via skipTestIfSNOCluster().
Ote Binary Stdout Contract ✅ Passed No violations of OTE Binary Stdout Contract found; all logging uses ginkgo.GinkgoWriter and no direct stdout writes in process-level code.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed Test suite contains no IPv4 assumptions, hardcoded IPv4 addresses, IPv4-specific patterns, or external connectivity requirements. Uses cluster-internal APIs and platform-specific device names.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from RishabhSaini and yuqi-zhang May 26, 2026 13:29
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 26, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: proietfb
Once this PR has been reviewed and has the lgtm label, please assign djoshy for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/extended-priv/irreconcilableChanges.go`:
- Around line 49-54: Add an explicit MicroShift skip guard to the suite setup:
inside the g.JustBeforeEach block (alongside PreChecks, skipIfNoTechPreview,
skipTestIfSupportedPlatformNotMatched, skipTestIfSNOCluster) call the existing
helper that skips MicroShift clusters (e.g., skipIfMicroShift(oc) or the
project’s equivalent), or if none exists add a short guard that detects
MicroShift and calls GinkgoT().Skip/By to skip the test; ensure the new call
runs before tests that rely on MachineSet scaling.
- Around line 339-344: The test is using AWS-only NVMe paths in mc2.Create which
will fail on GCE; replace the hardcoded "/dev/nvme3n1" and "/dev/nvme4n1"
parameters with platform-aware device paths by computing device1 and device2
based on the test environment (e.g., via an existing helper or function that
returns device paths for the current provider) and pass those variables into
mc2.Create along with newMcName and mcpool; update references around mc2.Create
to use the computed device variables so the test runs on both AWS and GCE.
- Around line 351-354: Remove the duplicate deletion by deleting the deferred
call and keeping the explicit deletion/wait: remove the defer
mc2.DeleteWithWait() line and leave the immediate mc2.DeleteWithWait() followed
by mcp.WaitForNotDegradedStatus(); this ensures the resource is deleted once
while preserving the existing wait logic (refer to mc2.DeleteWithWait() and
mcp.WaitForNotDegradedStatus()).

In `@test/extended-priv/util.go`:
- Around line 662-663: The call to IsSNOSafe(oc) currently discards the error
(sno, _ := IsSNOSafe(oc)) which can cause SNO checks to be bypassed; change the
call to capture the error, check it, and handle it appropriately (e.g., fail the
test, return the error, or skip the disruptive test) instead of proceeding when
detection fails — locate the IsSNOSafe call and the subsequent if sno { block in
test/extended-priv/util.go and replace the blank error discard with error
handling that logs or fails the test using the existing test framework/context
(use the same oc and surrounding helper functions to report the failure or
skip).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b0f1fe58-1928-43aa-9512-3f0acc39684f

📥 Commits

Reviewing files that changed from the base of the PR and between 935d692 and 65cf04b.

📒 Files selected for processing (5)
  • test/extended-priv/irreconcilableChanges.go
  • test/extended-priv/machineconfiguration.go
  • test/extended-priv/mco_storage.go
  • test/extended-priv/testdata/files/extra-disks-with-files.yaml
  • test/extended-priv/util.go

Comment on lines +49 to +54
g.JustBeforeEach(func() {
PreChecks(oc)
skipIfNoTechPreview(oc)
skipTestIfSupportedPlatformNotMatched(oc, AWSPlatform, GCPPlatform)
skipTestIfSNOCluster(oc)
})
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Add an explicit MicroShift skip guard in suite setup.

These new Ginkgo tests depend on MachineSet scaling and other features that are not available on MicroShift, but the suite setup only guards SNO/platform/feature gate.

Suggested guard
 g.JustBeforeEach(func() {
 	PreChecks(oc)
 	skipIfNoTechPreview(oc)
 	skipTestIfSupportedPlatformNotMatched(oc, AWSPlatform, GCPPlatform)
+	if exutil.IsMicroShiftCluster(oc) {
+		g.Skip("skip test because current cluster is MicroShift")
+	}
 	skipTestIfSNOCluster(oc)
 })
As per coding guidelines: “When new Ginkgo e2e tests are added ... check whether they use any APIs or features that are NOT available on MicroShift...”.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
g.JustBeforeEach(func() {
PreChecks(oc)
skipIfNoTechPreview(oc)
skipTestIfSupportedPlatformNotMatched(oc, AWSPlatform, GCPPlatform)
skipTestIfSNOCluster(oc)
})
g.JustBeforeEach(func() {
PreChecks(oc)
skipIfNoTechPreview(oc)
skipTestIfSupportedPlatformNotMatched(oc, AWSPlatform, GCPPlatform)
if exutil.IsMicroShiftCluster(oc) {
g.Skip("skip test because current cluster is MicroShift")
}
skipTestIfSNOCluster(oc)
})
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended-priv/irreconcilableChanges.go` around lines 49 - 54, Add an
explicit MicroShift skip guard to the suite setup: inside the g.JustBeforeEach
block (alongside PreChecks, skipIfNoTechPreview,
skipTestIfSupportedPlatformNotMatched, skipTestIfSNOCluster) call the existing
helper that skips MicroShift clusters (e.g., skipIfMicroShift(oc) or the
project’s equivalent), or if none exists add a short guard that detects
MicroShift and calls GinkgoT().Skip/By to skip the test; ensure the new call
runs before tests that rely on MachineSet scaling.

Comment thread test/extended-priv/irreconcilableChanges.go
Comment thread test/extended-priv/irreconcilableChanges.go Outdated
Comment thread test/extended-priv/util.go Outdated
- Removed hardcoded devices names
- Enhanced security on skipIfSNOCluster api
- Removed redundant check
- Aligned tests api structure
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/extended-priv/irreconcilableChanges.go`:
- Around line 34-40: The GCP case in platformBasedDisksNames returns root
partition paths (/dev/sda1–sda4) which can target the boot disk; change the GCP
mapping in the GCPPlatform branch (case GCPPlatform) to use non-root GCE device
names (e.g. /dev/sdb, /dev/sdc, /dev/sdd, /dev/sde) so tests reference
additional disks rather than the boot partitions.

In `@test/extended-priv/util.go`:
- Around line 662-664: The current check treats any IsSNOSafe error as a SNO and
silently skips tests; instead, call out the error when IsSNOSafe(oc) returns an
error and only skip when sno is true. Replace the combined condition using sno
and err so that after calling IsSNOSafe(oc) you first handle err (e.g., fail the
test or g.Fatalf/log the error with the error value from err) and only call
g.Skip("skip test because current cluster is a SNO cluster") when sno == true;
references: IsSNOSafe, sno, err, and g.Skip.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 314db668-ecaa-4a77-b3bf-4d64778c39aa

📥 Commits

Reviewing files that changed from the base of the PR and between 65cf04b and d75fbf5.

📒 Files selected for processing (2)
  • test/extended-priv/irreconcilableChanges.go
  • test/extended-priv/util.go

Comment on lines +34 to +40
case GCPPlatform:
return []string{
"/dev/sda1",
"/dev/sda2",
"/dev/sda3",
"/dev/sda4",
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Use non-root GCP device names in platformBasedDisksNames.

At Line 36-39, the GCP mapping points to /dev/sda* partitions, but these tests add extra non-boot disks. This can target the wrong devices and make GCE runs fail or behave unpredictably (downstream at Line 90 and Line 339-340).

Suggested fix
 case GCPPlatform:
 	return []string{
-		"/dev/sda1",
-		"/dev/sda2",
-		"/dev/sda3",
-		"/dev/sda4",
+		"/dev/sdb",
+		"/dev/sdc",
+		"/dev/sdd",
+		"/dev/sde",
 	}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
case GCPPlatform:
return []string{
"/dev/sda1",
"/dev/sda2",
"/dev/sda3",
"/dev/sda4",
}
case GCPPlatform:
return []string{
"/dev/sdb",
"/dev/sdc",
"/dev/sdd",
"/dev/sde",
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended-priv/irreconcilableChanges.go` around lines 34 - 40, The GCP
case in platformBasedDisksNames returns root partition paths (/dev/sda1–sda4)
which can target the boot disk; change the GCP mapping in the GCPPlatform branch
(case GCPPlatform) to use non-root GCE device names (e.g. /dev/sdb, /dev/sdc,
/dev/sdd, /dev/sde) so tests reference additional disks rather than the boot
partitions.

Comment on lines +662 to +664
sno, err := IsSNOSafe(oc)
if sno || err != nil {
g.Skip("skip test because current cluster is a SNO cluster")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Don’t convert SNO-detection failures into silent skips.

At Line 663-664, err != nil is treated as “SNO” and skipped with a misleading reason. This can hide real failures and suppress test coverage.

Suggested fix
 func skipTestIfSNOCluster(oc *exutil.CLI) {
 	sno, err := IsSNOSafe(oc)
-	if sno || err != nil {
+	o.ExpectWithOffset(1, err).NotTo(o.HaveOccurred(), "Could not determine whether cluster is SNO")
+	if sno {
 		g.Skip("skip test because current cluster is a SNO cluster")
 	}
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
sno, err := IsSNOSafe(oc)
if sno || err != nil {
g.Skip("skip test because current cluster is a SNO cluster")
func skipTestIfSNOCluster(oc *exutil.CLI) {
sno, err := IsSNOSafe(oc)
o.ExpectWithOffset(1, err).NotTo(o.HaveOccurred(), "Could not determine whether cluster is SNO")
if sno {
g.Skip("skip test because current cluster is a SNO cluster")
}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended-priv/util.go` around lines 662 - 664, The current check treats
any IsSNOSafe error as a SNO and silently skips tests; instead, call out the
error when IsSNOSafe(oc) returns an error and only skip when sno is true.
Replace the combined condition using sno and err so that after calling
IsSNOSafe(oc) you first handle err (e.g., fail the test or g.Fatalf/log the
error with the error value from err) and only call g.Skip("skip test because
current cluster is a SNO cluster") when sno == true; references: IsSNOSafe, sno,
err, and g.Skip.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 26, 2026

@proietfb: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/unit d75fbf5 link true /test unit

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant