Add dependabot to regularly update spack packages and refactor spack-matrix#32
Open
finsberg wants to merge 2 commits into
Open
Add dependabot to regularly update spack packages and refactor spack-matrix#32finsberg wants to merge 2 commits into
finsberg wants to merge 2 commits into
Conversation
…matrix to parse packages fron config
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implement Automated Spack Package Dependabot and Dynamic CI Matrix
Description
This PR introduces a custom, automated "Dependabot-like" workflow for our Spack packages and heavily refactors the CI testing pipeline to be much more maintainable.
Previously, the CI matrix required copying and pasting large YAML blocks for every tested package, and tracking upstream package updates was a manual process. This PR solves both issues by introducing a JSON-driven CI matrix and a native Python updater script.
Key Changes & New Files
.github/workflows/spack-dependabot.yml: A new weekly scheduled workflow. It runs the package updater, formats the changes with ruff, and opens a PR natively using the GitHub CLI (gh)..github/scripts/update_packages.py: The core updater script. It parses package.py files, queries GitHub and PyPI for newer releases, computes the sha256 checksums of the source tarballs, and safely injects the new versions into both the Spack package definitions and the CI configuration..github/ci-test-config.json: A new centralized configuration file containing the test targets, specs, and import names for all tested packages. Adding a new package to the CI now only requires adding it to this file..github/scripts/generate_matrix.py: A helper script that translates the JSON configuration into a dynamic matrix format compatible with GitHub Actions..github/workflows/spack-matrix.yml: Refactored the main testing workflow, leveraging the dynamically generated matrix.Motivation and Context
Automates the tedious process of tracking upstream releases.
Ensures new versions are immediately and automatically tested against the dynamic CI matrix before merging.
Significantly reduces boilerplate YAML, making it much easier to add and manage tested packages moving forward.