Skip to content

[rally] Add Rally role#3996

Open
lkuchlan wants to merge 2 commits into
openstack-k8s-operators:mainfrom
lkuchlan:add-rally-role
Open

[rally] Add Rally role#3996
lkuchlan wants to merge 2 commits into
openstack-k8s-operators:mainfrom
lkuchlan:add-rally-role

Conversation

@lkuchlan

@lkuchlan lkuchlan commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Add a new `rally` Ansible role that runs OpenStack Rally benchmarks inside a podman
container (`quay.io/airshipit/xrally-openstack:3.0.0`). The role auto-discovers OpenStack
credentials from the cluster's KeystoneAPI resource and appends the deployment CA to the
trust bundle so SSL verification works without any manual configuration.

The role supports two execution modes:

  • Single run (default): uses top-level `cifmw_rally_*` variables; artifacts stored
    under `cifmw_rally_artifacts_basedir/`.
  • Multi-run: set `cifmw_rally_runs` to a list of dicts, each with a `name` key and
    optional `cifmw_rally_*` overrides. Artifacts are namespaced under
    `cifmw_rally_artifacts_basedir//`.

Jobs invoke Rally via the ci-framework hooks system using `hooks/playbooks/rally_run.yaml`:

post_tests_90_rally_run:
  type: playbook
  source: rally_run.yaml

Key design decisions:

  • Podman container (no Kubernetes CRD exists for Rally)
  • Shell script templated to disk to avoid YAML/JSON quoting issues
  • `--fromenv` deployment creation — no JSON config file needed
  • `rootca-public` CA from the `openstack` namespace appended to the CA bundle so the
    container can verify the Keystone TLS endpoint

Assisted-By: Claude Sonnet 4.6 noreply@anthropic.com

@openshift-ci

openshift-ci Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign michburk for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/bf4c2f1d592c429fbc5f647b5e6d0291

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 16m 50s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 31m 41s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 32m 19s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 04m 01s
cifmw-multinode-tempest FAILURE in 1h 48m 46s
cifmw-pod-zuul-files FAILURE in 5m 23s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 16s
cifmw-pod-pre-commit FAILURE in 7m 15s
✔️ cifmw-molecule-test_operator SUCCESS in 4m 14s

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/662f66b62e854e55b7a590b4ec9e18a6

openstack-k8s-operators-content-provider FAILURE in 7m 00s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal-minor-update SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-multinode-tempest SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
cifmw-pod-zuul-files FAILURE in 5m 41s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 14s
cifmw-pod-pre-commit FAILURE in 7m 29s
✔️ cifmw-molecule-test_operator SUCCESS in 4m 18s

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/eaffeaa267354aed85c70b494ff76c00

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 39m 19s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 33m 46s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 39m 09s
cifmw-crc-podified-edpm-baremetal-minor-update FAILURE in 2h 15m 21s
✔️ cifmw-multinode-tempest SUCCESS in 1h 56m 24s
cifmw-pod-zuul-files FAILURE in 5m 11s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 10m 02s
cifmw-pod-pre-commit FAILURE in 8m 59s
✔️ cifmw-molecule-test_operator SUCCESS in 4m 05s

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/431e13de80224aaa9d839282787ec241

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 23m 47s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 27m 54s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 45m 04s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 09m 30s
✔️ cifmw-multinode-tempest SUCCESS in 1h 50m 53s
cifmw-pod-zuul-files FAILURE in 5m 27s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 57s
cifmw-pod-pre-commit FAILURE in 8m 13s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 55s

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/13459fdd143e435db35c4ca775a62b17

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 11m 55s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 24m 27s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 34m 33s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 1h 57m 28s
✔️ cifmw-multinode-tempest SUCCESS in 1h 46m 49s
cifmw-pod-zuul-files FAILURE in 5m 45s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 10m 19s
cifmw-pod-pre-commit FAILURE in 9m 59s
✔️ cifmw-molecule-test_operator SUCCESS in 4m 08s

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/d659f648237b4abfb44ac4397974dfbd

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 20m 46s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 24m 27s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 42m 24s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 07m 25s
✔️ cifmw-multinode-tempest SUCCESS in 1h 48m 57s
cifmw-pod-zuul-files FAILURE in 4m 49s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 08s
cifmw-pod-pre-commit FAILURE in 8m 44s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 55s

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/1970968512b442e9beac4bb8344b1676

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 15m 22s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 23m 43s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 39m 23s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 02m 11s
✔️ cifmw-multinode-tempest SUCCESS in 1h 49m 55s
cifmw-pod-zuul-files FAILURE in 4m 58s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 36s
cifmw-pod-pre-commit FAILURE in 9m 39s
✔️ cifmw-molecule-test_operator SUCCESS in 4m 00s

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/2cd9d504a7b4483b8f2bf56e4832d1b3

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 22m 37s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 30m 09s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 36m 14s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 07m 02s
✔️ cifmw-multinode-tempest SUCCESS in 1h 44m 44s
cifmw-pod-zuul-files FAILURE in 5m 13s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 55s
cifmw-pod-pre-commit FAILURE in 8m 52s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 55s

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/730a61807ee541ea8236bceedf18b81b

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 21m 42s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 24m 31s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 30m 43s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 07m 57s
✔️ cifmw-multinode-tempest SUCCESS in 1h 44m 57s
cifmw-pod-zuul-files FAILURE in 5m 07s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 49s
cifmw-pod-pre-commit FAILURE in 8m 45s
✔️ cifmw-molecule-test_operator SUCCESS in 4m 13s

@lkuchlan

lkuchlan commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

Testproject MR

A testproject MR has been added to validate this implementation end-to-end

Test Results

This POC confirms that the implementation is working correctly.

Cinder workload: One test failure observed, but it appears to be a Rally bug, not an issue with the integration itself.

Manila workload: The failures seen are configuration-related only and are not caused by the Rally role or its wiring into the test_operator framework.

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/590479b51e2347af83ea54d9e458cc73

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 31m 07s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 27m 57s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 42m 50s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 16m 05s
✔️ cifmw-multinode-tempest SUCCESS in 1h 57m 11s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 46s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 10m 29s
✔️ cifmw-pod-pre-commit SUCCESS in 10m 29s
cifmw-molecule-rally FAILURE in 5m 28s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 51s

@lkuchlan
lkuchlan force-pushed the add-rally-role branch 2 times, most recently from 43000e6 to 9ba4734 Compare June 22, 2026 11:58
@centosinfra-prod-github-app

Copy link
Copy Markdown

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/openstack-k8s-operators/ci-framework for 3996,9ba473493032fddaa9d4b4066af30c520a05dd15

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/2ca2994a4b5d47b39690b002e63ddba2

openstack-k8s-operators-content-provider FAILURE in 4m 24s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal-minor-update SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ cifmw-pod-zuul-files SUCCESS in 4m 36s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 55s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 43s
cifmw-molecule-rally FAILURE in 4m 12s

@@ -0,0 +1,15 @@
---

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couldn't hooks/playbooks/manila_create_default_resources.yml be used instead, after overriding some variables?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we can use it.
rally only requires the share type, the same as tempest, so we can reuse this playbook with the required variable overrides.
Thanks for the suggestion!

@michburk michburk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of comments, the check jobs are failing because of some ongoing OpenSSL issues, sorry about that!

Comment thread roles/rally/tasks/main.yml Outdated
Comment thread roles/rally/tasks/run_once.yml Outdated
Comment thread .ansible-lint Outdated
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/2ed1e09f353a4151b37e848aaa91b897

✔️ openstack-k8s-operators-content-provider SUCCESS in 52m 58s
podified-multinode-edpm-deployment-crc NODE_FAILURE Node(set) request 099-0000137815 failed in 0s
cifmw-crc-podified-edpm-baremetal FAILURE in 37m 14s
cifmw-crc-podified-edpm-baremetal-minor-update NODE_FAILURE Node(set) request 099-0000137817 failed in 0s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 56s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 48s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 41s
✔️ cifmw-molecule-rally SUCCESS in 7m 30s

@michburk

michburk commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

recheck

Comment thread hooks/playbooks/manila_create_default_resources.yml
@lkuchlan

Copy link
Copy Markdown
Contributor Author

/retest

Comment thread roles/rally/templates/rally-run.sh.j2 Outdated
rally task report --out {{ _cifmw_rally_container_reports_dir }}/rally-report.html || true
rally task report --json \
--out {{ _cifmw_rally_container_reports_dir }}/rally-report.json || true
exit ${RALLY_TASK_RC:-0}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(non-blocking) suggestion: I'd better initialize this variable at the start. And the move this to be exit RALLY_TASK_RC

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread roles/rally/tasks/main.yml Outdated
mode: "0755"

- name: Create rally deployment configuration
ansible.builtin.include_tasks: create-deployment.yml

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(non-blocking) Suggestion: Should this be called better prepare-deployment.yml? or prepare-rally-credentials.yaml?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense
I changed it to prepare-rally-credentials.yaml

Comment thread roles/rally/tasks/prepare-rally-credentials.yaml
Comment thread hooks/playbooks/rally_cinder_prerequisites.yaml
--ram 512 --disk 1 --vcpus 1 --public

- name: Create lvmdriver-1 volume type if missing
ansible.builtin.shell: |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(blocking) suggestion: same as above

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

openstack volume type create lvmdriver-1 --public

- name: Check if cirros image already exists
ansible.builtin.shell: |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(blocking) suggestion: I'd move here to k8s_exec.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread hooks/playbooks/rally_cinder_prerequisites.yaml
Comment thread roles/rally/molecule/default/converge.yml
lkuchlan and others added 2 commits July 16, 2026 11:05
Add a new `rally` Ansible role that runs OpenStack Rally benchmarks
inside a podman container (quay.io/airshipit/xrally-openstack:3.0.0).
The role auto-discovers OpenStack credentials from the cluster's
KeystoneAPI resource and appends the deployment CA to the trust bundle
so SSL verification works without any manual configuration.

The role supports two execution modes:

* Single run (default): `cifmw_rally_runs: []` — uses the top-level
  `cifmw_rally_*` variables, artifacts stored in
  `cifmw_rally_artifacts_basedir/`.

* Multi-run: set `cifmw_rally_runs` to a list of dicts, each with a
  `name` key and optional `cifmw_rally_*` overrides. Artifacts for each
  run are namespaced under `cifmw_rally_artifacts_basedir/<name>/`.
  This is modelled after `cifmw_test_operator_stages` and replaces the
  former test_operator-based wiring (Rally is not a test-operator CRD).

A `hooks/playbooks/rally_run.yaml` entry point is provided so jobs can
invoke Rally via the ci-framework hooks system, e.g.:

  post_tests_90_rally_run:
    type: playbook
    source: rally_run.yaml

Assisted-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: lkuchlan <lkuchlan@redhat.com>
…ling

When extra_specs is passed via extra_vars on the command line
(e.g. -e 'extra_specs={}'), Ansible receives it as a string instead
of a dict. The original task-level vars: block evaluated
extra_specs.items() before the when: condition was checked, causing
the play to fail with 'str object has no attribute items'.

Split into a set_fact task guarded by 'extra_specs is mapping' and a
separate command task guarded by 'extra_spec_opt is defined' to handle
both dict and string inputs safely.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: lkuchlan <lkuchlan@redhat.com>

@evallesp evallesp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/66902d6c6e2e4204a5821f06fe180b34

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 01m 12s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 27m 17s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 46m 37s
cifmw-crc-podified-edpm-baremetal-minor-update FAILURE in 1h 01m 18s
✔️ cifmw-pod-zuul-files SUCCESS in 7m 12s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 54m 40s
cifmw-crc-podified-edpm-baremetal-bootc NODE_FAILURE Node(set) request 099-0000144386 failed in 0s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 11m 30s
✔️ cifmw-pod-pre-commit SUCCESS in 10m 42s
✔️ cifmw-molecule-rally SUCCESS in 6m 55s

@michburk

Copy link
Copy Markdown
Contributor

recheck

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants