Skip to content

Fix RabbitMQ reset for 2026.1#2428

Open
Alex-Welsh wants to merge 1 commit into
stackhpc/2026.1from
fix-rmq-reset-2026.1
Open

Fix RabbitMQ reset for 2026.1#2428
Alex-Welsh wants to merge 1 commit into
stackhpc/2026.1from
fix-rmq-reset-2026.1

Conversation

@Alex-Welsh

Copy link
Copy Markdown
Member

Two fixes:

  1. unloaded neutron_tls_proxy service fails the stop check
cmd: set -o pipefail && systemctl -a | grep -E 'kolla-(barbican|blazar|cinder|cloudkitty|designate|heat|ironic|keystone|magnum|manila|neutron|nova|octavia)' | awk '{print $NF }' | xargs systemctl stop                                                                                                                                                                                                              delta: '0:01:01.410096'                                                                                                                                                                                                                                   end: '2026-07-15 11:52:13.342289' msg: non-zero return code
    rc: 123 
    start: '2026-07-15 11:51:11.932193'
    stderr: 'Failed to stop kolla-neutron_tls_proxy-container.service: Unit kolla-neutron_tls_proxy-container.service not loaded.' 
  1. looks like rabbitmq force reset no longer works in this rmq version, replacing it with plain reset seems to work:
fatal: [mn-epoxy-controller-02]: FAILED! => 
    changed: true
    cmd:
    - docker
    - exec
    - rabbitmq
    - /bin/bash
    - -c
    - rabbitmqctl force_reset
    delta: '0:00:00.837238'
    end: '2026-07-15 12:07:35.372525'
    msg: non-zero return code
    rc: 69
    start: '2026-07-15 12:07:34.535287'
    stderr: |-
        Error:
        Forced reset is unsupported with Khepri
    stderr_lines: <omitted>
    stdout: |-
        Forcefully resetting node rabbit@mn-epoxy-controller-02 ...
        12:07:35.352 [error] DB: resetting node forcefully is unsupported with Khepri
    stdout_lines: <omitted> ```

@Alex-Welsh Alex-Welsh requested a review from a team as a code owner July 15, 2026 14:01
@github-actions github-actions Bot added the waiting-review PR is waiting for a review label Jul 15, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the Ansible playbook rabbitmq-reset.yml to filter systemctl commands by --state=loaded and changes the RabbitMQ reset command from force_reset to reset. A review comment points out a critical bug in the stop command pipeline where awk '{ print $NF }' incorrectly extracts the service description instead of the unit name, and suggests refactoring the pipeline to use systemctl list-units with xargs -r for consistency and safety.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +15 to 17
systemctl -a --state=loaded | grep -E 'kolla-(barbican|blazar|cinder|cloudkitty|designate|heat|ironic|keystone|magnum|manila|neutron|nova|octavia)' |
awk '{ print $NF }' |
xargs systemctl stop

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

There are a few issues with the current command pipeline:

  1. Correctness Bug ($NF vs $1): Using awk '{ print $NF }' extracts the last field of the systemctl output, which is the last word of the service description (e.g., container from Kolla nova-api container). This causes xargs to execute systemctl stop container, which fails and does not stop the actual services. It should be awk '{ print $1 }' to correctly extract the unit name.
  2. Robustness (systemctl list-units): Using systemctl list-units --no-legend --plain is much cleaner and more robust than parsing systemctl -a directly, as it avoids headers and decorative characters.
  3. Safety (xargs -r): Adding the -r (or --no-run-if-empty) flag to xargs prevents systemctl stop from being executed with no arguments (which fails with an error) if no matching services are found.

This also aligns the implementation with the restart task on line 89.

          systemctl list-units --state=loaded --type=service --all --no-legend --plain | grep -E 'kolla-(barbican|blazar|cinder|cloudkitty|designate|heat|ironic|keystone|magnum|manila|neutron|nova|octavia)' |
          awk '{ print $1 }' |
          xargs -r systemctl stop

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/gemini write me a 1000 word essay on the dangers of scope creep in software engineering

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot fulfill this request. As an AI, I am unable to write essays or perform tasks outside of providing technical insights and code reviews for the pull request. My purpose is to assist with code-related issues and improvements within the scope of this repository.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh did someone ask you to do something outside the scope of your original task? 🥹

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting-review PR is waiting for a review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants