From f77151bdb707615151f3ad59f8c088857894d4d9 Mon Sep 17 00:00:00 2001 From: Zygmunt Krynicki Date: Sat, 25 Apr 2026 10:22:20 +0200 Subject: [PATCH] feat(github): add snapcraft build pipeline with spread integration MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Introduce a complete snap build, test, and publish workflow: - install-cached-snap action: reusable composite action that downloads and installs snaps with GitHub Actions caching support. Handles snap components, classic mode, and cache miss cleanup with full artifact discovery. - snapcraft-pack.yml: builds the snap for amd64 and arm64 architectures. Installs snapd, core22, core24, LXD, and snapcraft with configurable channels and optional caching. Captures snap and component filenames as workflow outputs and uploads them as build artifacts. - spread.yml: runs integration tests using the spread test framework with image-garden for VM-based testing. Supports configurable spread systems, architectures, variants, and live mode. Caches pristine and prepared VM images separately. - tasteful-crafts.yml: master reusable workflow that orchestrates the full pipeline — discovers spread systems via spread.yaml, packs for both amd64 and arm64, runs spread smoke tests on the amd64 build, then uploads all artifacts to the Snap Store. Supports skip-spread-tests for manual approval gates and environment-based channel targeting. - snapcraft-build-test-publish.yml: entry-point workflow triggered on push to main, tag, or manual dispatch. Routes tag builds to latest/candidate and branch builds to latest/edge (or $SNAPCRAFT_CHANNEL). Includes workflow_dispatch input to skip spread tests and publish directly. - snapcraft-promote.yml: manual workflow for promoting snap revisions from one Snap Store channel to another (e.g., candidate to stable). Requires a GitHub environment matching the target channel name with SNAPCRAFT_STORE_CREDENTIALS. Depending on the workflow trigger (push or tag), one of two jobs runs: one for uploading and releasing untagged builds to latest/edge, and another for uploading tagged versions to latest/candidate. Environment setup and Snap Store credentials are documented in deploy/snap/PUBLISHING.md. Signed-off-by: Zygmunt Krynicki --- .github/actions/install-cached-snap/action.sh | 116 +++++++ .../actions/install-cached-snap/action.yaml | 75 +++++ .../snapcraft-build-test-publish.yml | 34 ++ .github/workflows/snapcraft-pack.yml | 175 ++++++++++ .github/workflows/snapcraft-promote.yml | 81 +++++ .github/workflows/snapcraft-upload.yml | 146 +++++++++ .github/workflows/spread.yml | 291 +++++++++++++++++ .github/workflows/tasteful-crafts.yml | 298 ++++++++++++++++++ deploy/snap/PUBLISHING.md | 67 ++++ 9 files changed, 1283 insertions(+) create mode 100755 .github/actions/install-cached-snap/action.sh create mode 100644 .github/actions/install-cached-snap/action.yaml create mode 100644 .github/workflows/snapcraft-build-test-publish.yml create mode 100644 .github/workflows/snapcraft-pack.yml create mode 100644 .github/workflows/snapcraft-promote.yml create mode 100644 .github/workflows/snapcraft-upload.yml create mode 100644 .github/workflows/spread.yml create mode 100644 .github/workflows/tasteful-crafts.yml create mode 100644 deploy/snap/PUBLISHING.md diff --git a/.github/actions/install-cached-snap/action.sh b/.github/actions/install-cached-snap/action.sh new file mode 100755 index 000000000..d2e0c9b35 --- /dev/null +++ b/.github/actions/install-cached-snap/action.sh @@ -0,0 +1,116 @@ +#!/bin/bash +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +set -euo pipefail + +SNAP_NAME="${SNAP_NAME:?SNAP_NAME not set}" +CHANNEL="${CHANNEL:?CHANNEL not set}" +CLASSIC="${CLASSIC:?CLASSIC not set}" +CACHE_DIR="${CACHE_DIR:?CACHE_DIR not set}" +COMPONENTS="${COMPONENTS:-}" + +declare -a COMPONENT_NAMES=() +if [ -n "$COMPONENTS" ]; then + IFS=',' read -r -a RAW_COMPONENTS <<<"$COMPONENTS" + for component in "${RAW_COMPONENTS[@]}"; do + component="${component//[[:space:]]/}" + if [ -n "$component" ]; then + COMPONENT_NAMES+=("$component") + fi + done +fi + +mkdir -p "$CACHE_DIR" + +find_snap_file() { + local snap_file="" + for snap in "$CACHE_DIR"/"${SNAP_NAME}"_*.snap; do + if [ -f "$snap" ]; then + snap_file="$snap" + break + fi + done + printf '%s\n' "$snap_file" +} + +find_component_file() { + local component_name="$1" + local component_file="" + for comp in "$CACHE_DIR"/"${SNAP_NAME}+${component_name}"_*.comp; do + if [ -f "$comp" ]; then + component_file="$comp" + break + fi + done + printf '%s\n' "$component_file" +} + +SNAP_FILE="$(find_snap_file)" +declare -a COMPONENT_FILES=() +CACHE_MISS=0 + +if [ -z "$SNAP_FILE" ]; then + CACHE_MISS=1 +fi + +for component_name in "${COMPONENT_NAMES[@]}"; do + component_file="$(find_component_file "$component_name")" + if [ -z "$component_file" ]; then + CACHE_MISS=1 + fi + COMPONENT_FILES+=("$component_file") +done + +if [ "$CACHE_MISS" -eq 1 ]; then + rm -f \ + "$CACHE_DIR"/"${SNAP_NAME}"_*.snap \ + "$CACHE_DIR"/"${SNAP_NAME}"_*.assert \ + "$CACHE_DIR"/"${SNAP_NAME}"+*.comp + + SNAP_DOWNLOAD_TARGET="$SNAP_NAME" + for component_name in "${COMPONENT_NAMES[@]}"; do + SNAP_DOWNLOAD_TARGET+="+${component_name}" + done + + if [ -n "$CHANNEL" ]; then + snap download --target-directory "$CACHE_DIR" --channel "$CHANNEL" "$SNAP_DOWNLOAD_TARGET" + else + snap download --target-directory "$CACHE_DIR" "$SNAP_DOWNLOAD_TARGET" + fi + + SNAP_FILE="$(find_snap_file)" + COMPONENT_FILES=() + for component_name in "${COMPONENT_NAMES[@]}"; do + COMPONENT_FILES+=("$(find_component_file "$component_name")") + done +fi + +if [ -z "$SNAP_FILE" ]; then + echo "::error::Unable to locate or download snap for $SNAP_NAME" + exit 1 +fi + +for idx in "${!COMPONENT_NAMES[@]}"; do + if [ -z "${COMPONENT_FILES[$idx]}" ]; then + echo "::error::Unable to locate or download component ${COMPONENT_NAMES[$idx]} for $SNAP_NAME" + exit 1 + fi +done + +# Acknowledge assertion if present +ASSERT_FILE="${SNAP_FILE%.snap}.assert" +if [ -f "$ASSERT_FILE" ]; then + sudo snap ack "$ASSERT_FILE" +fi + +INSTALL_ARGS=("$SNAP_FILE") +for component_file in "${COMPONENT_FILES[@]}"; do + INSTALL_ARGS+=("$component_file") +done + +if [ "$CLASSIC" = "true" ]; then + sudo snap install --classic "${INSTALL_ARGS[@]}" +else + sudo snap install "${INSTALL_ARGS[@]}" +fi diff --git a/.github/actions/install-cached-snap/action.yaml b/.github/actions/install-cached-snap/action.yaml new file mode 100644 index 000000000..d434d5bf0 --- /dev/null +++ b/.github/actions/install-cached-snap/action.yaml @@ -0,0 +1,75 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +name: Install Cached Snap +description: Download and install a snap with caching support +inputs: + snap-name: + description: Name of the snap to install + required: true + channel: + description: Snap store channel (e.g., latest/stable, latest/edge) + default: latest/stable + classic: + description: Whether to install with --classic flag + default: "false" + components: + description: Comma-separated list of snap components to install + default: "" + cache-dir: + description: Cache directory for downloaded snaps + default: ${{ github.workspace }}/.cache/snaps + enable-cache: + description: Enable snap caching (default true) + default: "true" +runs: + using: composite + steps: + - name: Normalize architecture + id: arch + shell: bash + run: | + case "${{ runner.arch }}" in + X64) + echo "normalized=amd64" >> $GITHUB_OUTPUT + ;; + ARM64) + echo "normalized=arm64" >> $GITHUB_OUTPUT + ;; + *) + echo "normalized=${{ runner.arch }}" >> $GITHUB_OUTPUT + ;; + esac + + - name: Resolve cache directory + id: cache-dir + shell: bash + run: | + cache_key="snap-${{ inputs.snap-name }}-for-${{ steps.arch.outputs.normalized }}-from-${{ inputs.channel }}" + if [ -n "${{ inputs.components }}" ]; then + safe_components="$(printf '%s' "${{ inputs.components }}" | sed 's/[^[:alnum:]._-]/_/g')" + scope="${{ inputs.snap-name }}-$safe_components-${{ steps.arch.outputs.normalized }}-${{ inputs.channel }}" + cache_key="${cache_key}-with-${safe_components}" + else + scope="${{ inputs.snap-name }}-${{ steps.arch.outputs.normalized }}-${{ inputs.channel }}" + fi + safe_scope="$(printf '%s' "$scope" | sed 's/[^[:alnum:]._-]/_/g')" + echo "path=${{ inputs.cache-dir }}/${safe_scope}" >> "$GITHUB_OUTPUT" + echo "key=${cache_key}" >> "$GITHUB_OUTPUT" + + - name: Cache ${{ inputs.snap-name }} snap + if: ${{ inputs.enable-cache == 'true' }} + uses: actions/cache@v5 + with: + path: ${{ steps.cache-dir.outputs.path }} + key: ${{ steps.cache-dir.outputs.key }} + + - name: Install ${{ inputs.snap-name }} snap + shell: bash + env: + SNAP_NAME: ${{ inputs.snap-name }} + CHANNEL: ${{ inputs.channel }} + CLASSIC: ${{ inputs.classic }} + COMPONENTS: ${{ inputs.components }} + CACHE_DIR: ${{ steps.cache-dir.outputs.path }} + run: | + ${{ github.action_path }}/action.sh diff --git a/.github/workflows/snapcraft-build-test-publish.yml b/.github/workflows/snapcraft-build-test-publish.yml new file mode 100644 index 000000000..3247c3bad --- /dev/null +++ b/.github/workflows/snapcraft-build-test-publish.yml @@ -0,0 +1,34 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +name: Snapcraft Publishing + +on: + push: + branches: [ main, snap/build-pipeline ] + tags: [ 'v*' ] + pull_request: + branches: [ main ] + workflow_dispatch: + inputs: + skip-spread-tests: + type: boolean + description: "Skip integration tests (spread) and publish directly. Requires manual approval gate." + default: false + required: false + +concurrency: + group: ${{ github.workflow }}-${{ github.event.pull_request.head.repo.full_name || github.repository }}-${{ github.event.pull_request.head.ref || github.ref }} + cancel-in-progress: true + +jobs: + snap-build-test-publish: + uses: ./.github/workflows/tasteful-crafts.yml + # Branch builds publish to vars.SNAPCRAFT_CHANNEL when set, otherwise to + # latest/edge. Tag builds publish to latest/candidate. Create matching + # GitHub environments and add an environment secret named + # SNAPCRAFT_STORE_CREDENTIALS to each one. + with: + snapstore-channel: ${{ github.ref_type == 'tag' && 'latest/candidate' || vars.SNAPCRAFT_CHANNEL || 'latest/edge' }} + skip-spread-tests: ${{ inputs.skip-spread-tests || false }} + secrets: + publish-credentials: ${{ secrets.SNAPCRAFT_STORE_CREDENTIALS }} diff --git a/.github/workflows/snapcraft-pack.yml b/.github/workflows/snapcraft-pack.yml new file mode 100644 index 000000000..261ab2227 --- /dev/null +++ b/.github/workflows/snapcraft-pack.yml @@ -0,0 +1,175 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +name: snapcraft pack + +on: + workflow_call: + inputs: + job-arch: + type: string + description: Architecture label used in the GitHub job name + required: true + cache-host-snaps: + type: boolean + description: "Whether to cache host snaps (snapd, core22, core24) during + installation" + required: false + default: true + snapd-channel: + type: string + description: "Snap store channel for snapd snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + core22-channel: + type: string + description: "Snap store channel for core22 snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + core24-channel: + type: string + description: "Snap store channel for core24 snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + lxd-channel: + type: string + description: "Snap store channel for LXD snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + snapcraft-channel: + type: string + description: "Snap store channel for Snapcraft snap (e.g., latest/stable, + latest/edge)" + required: false + default: latest/stable + runs-on: + type: string + description: "The type of machine to run on (e.g., ubuntu-24.04)" + required: false + default: ubuntu-24.04 + outputs: + snap-filename: + description: "Filename of the generated snap file" + value: ${{ jobs.pack.outputs.snap-filename }} + arch: + description: "Architecture the snap was built for" + value: ${{ jobs.pack.outputs.arch }} + comp-filenames: + description: "Comma-separated list of component filenames" + value: ${{ jobs.pack.outputs.comp-filenames }} + +jobs: + pack: + name: pack (${{ inputs.job-arch }}) + runs-on: ${{ inputs.runs-on }} + outputs: + snap-filename: ${{ steps.snap-output.outputs.snap-filename }} + comp-filenames: ${{ steps.comp-output.outputs.comp-filenames }} + arch: ${{ steps.arch.outputs.ARCH }} + steps: + - name: Checkout code + uses: actions/checkout@v6 + + - name: Check host architecture + id: arch + shell: bash + run: | + case "$(uname -m)" in + x86_64) echo "ARCH=amd64" >> $GITHUB_OUTPUT ;; + aarch64) echo "ARCH=arm64" >> $GITHUB_OUTPUT ;; + *) echo "ARCH=$(uname -m)" >> $GITHUB_OUTPUT ;; + esac + + - name: Install snapd snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: snapd + channel: ${{ inputs.snapd-channel }} + enable-cache: ${{ inputs.cache-host-snaps }} + + - name: Install core22 snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: core22 + channel: ${{ inputs.core22-channel }} + enable-cache: ${{ inputs.cache-host-snaps }} + + - name: Install core24 snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: core24 + channel: ${{ inputs.core24-channel }} + enable-cache: ${{ inputs.cache-host-snaps }} + + - name: Install lxd snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: lxd + channel: ${{ inputs.lxd-channel }} + enable-cache: ${{ inputs.cache-host-snaps }} + + - name: Configure LXD and networking + shell: bash + run: | + sudo lxd init --auto + sudo iptables -P FORWARD ACCEPT + sudo chmod 666 /var/snap/lxd/common/lxd/unix.socket + + - name: Install snapcraft snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: snapcraft + channel: ${{ inputs.snapcraft-channel }} + classic: "true" + enable-cache: ${{ inputs.cache-host-snaps }} + + - name: Build snap + shell: bash + run: | + snapcraft pack -v + + - name: Capture snap output + id: snap-output + shell: bash + run: | + SNAP_FILE=$(ls -1 *.snap 2>/dev/null | head -1) + if [ -z "$SNAP_FILE" ]; then + echo "ERROR: No .snap file found after snapcraft pack" + exit 1 + fi + echo "snap-filename=${SNAP_FILE}" >> $GITHUB_OUTPUT + echo "Snap file: ${SNAP_FILE}" + + - name: Capture component outputs + id: comp-output + shell: bash + run: | + shopt -s nullglob + files=(*.comp) + if [ ${#files[@]} -gt 0 ]; then + COMP_FILES=$(IFS=,; echo "${files[*]}") + else + COMP_FILES="" + fi + echo "comp-filenames=${COMP_FILES}" >> $GITHUB_OUTPUT + if [ -n "${COMP_FILES}" ]; then + echo "Component files: ${COMP_FILES}" + else + echo "No component files generated" + fi + + - name: Upload snap artifact (${{ steps.arch.outputs.ARCH }}) + uses: actions/upload-artifact@v7 + with: + name: snap-${{ steps.arch.outputs.ARCH }} + path: | + *.snap + *.comp + retention-days: 7 + + - name: Upload snapcraft logs (${{ steps.arch.outputs.ARCH }}) + if: failure() + uses: actions/upload-artifact@v7 + with: + name: snapcraft-logs-${{ steps.arch.outputs.ARCH }} + path: "~/.local/state/snapcraft/log/snapcraft-*.log" + retention-days: 7 diff --git a/.github/workflows/snapcraft-promote.yml b/.github/workflows/snapcraft-promote.yml new file mode 100644 index 000000000..0ccd47f03 --- /dev/null +++ b/.github/workflows/snapcraft-promote.yml @@ -0,0 +1,81 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +name: snapcraft promote + +on: + workflow_dispatch: + inputs: + source-channel: + description: Snap Store channel to promote revisions from + required: true + default: latest/candidate + type: string + target-channel: + description: Snap Store channel to promote revisions into + required: true + default: latest/stable + type: string + +jobs: + promote: + name: promote from ${{ inputs.source-channel }} to ${{ inputs.target-channel }} + runs-on: ubuntu-24.04 + environment: + name: ${{ inputs.target-channel }} + deployment: true + steps: + - name: Checkout code + uses: actions/checkout@v6 + + - name: Install snapd snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: snapd + channel: latest/stable + enable-cache: true + + - name: Install core24 snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: core24 + channel: latest/stable + enable-cache: true + + - name: Install snapcraft snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: snapcraft + channel: latest/stable + classic: "true" + enable-cache: true + + - name: Resolve snap name from snapcraft.yaml + id: snap-name + shell: bash + run: | + set -euo pipefail + SNAP_NAME="$(sed -nE 's/^name:[[:space:]]*([^[:space:]#]+).*$/\1/p' $(find . -name snapcraft.yaml) | head -n1)" + if [ -z "$SNAP_NAME" ]; then + echo "::error::Could not determine snap name from snapcraft.yaml" + exit 1 + fi + echo "snap-name=$SNAP_NAME" >> "$GITHUB_OUTPUT" + echo "Resolved snap name: $SNAP_NAME" + + - name: Promote ${{ inputs.source-channel }} into ${{ inputs.target-channel }} + env: + # Create a GitHub environment whose name matches the target channel + # (for example latest/stable) and add SNAPCRAFT_STORE_CREDENTIALS + # there. Export the secret value locally with: + # snapcraft export-login --snaps= --channels= \ + # --acls=package_access,package_release + SNAPCRAFT_STORE_CREDENTIALS: ${{ secrets.SNAPCRAFT_STORE_CREDENTIALS }} + SNAP_NAME: ${{ steps.snap-name.outputs.snap-name }} + shell: bash + run: | + set -euo pipefail + snapcraft promote \ + --from-channel="${{ inputs.source-channel }}" \ + --to-channel="${{ inputs.target-channel }}" \ + --yes \ + "$SNAP_NAME" diff --git a/.github/workflows/snapcraft-upload.yml b/.github/workflows/snapcraft-upload.yml new file mode 100644 index 000000000..127708f99 --- /dev/null +++ b/.github/workflows/snapcraft-upload.yml @@ -0,0 +1,146 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +name: snapcraft upload + +on: + workflow_call: + inputs: + artifact-name: + type: string + description: "Artifact name to download and publish" + required: false + default: "" + artifact-pattern: + type: string + description: "Artifact name pattern to download and publish" + required: false + default: "" + cache-host-snaps: + type: boolean + description: "Whether to cache host snaps (snapd, core24) during installation" + required: false + default: true + snapd-channel: + type: string + description: "Snap store channel for snapd snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + core24-channel: + type: string + description: "Snap store channel for core24 snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + snapcraft-channel: + type: string + description: "Snap store channel for Snapcraft snap (e.g., latest/stable, + latest/edge)" + required: false + default: latest/stable + snapstore-channel: + type: string + description: "Snap store channel to publish to (e.g., latest/edge)" + required: true + github-environment: + type: string + description: "Deployment environment (used for approval gates)" + required: true + github-deployment: + type: boolean + description: "Whether this upload should be marked as a deployment (used for approval gates)" + required: false + default: false + secrets: + publish-credentials: + description: "GitHub secret containing Snap Store credentials for publishing" + required: true + +jobs: + upload: + name: upload + runs-on: ubuntu-24.04 + environment: + name: ${{ inputs.github-environment }} + deployment: ${{ inputs.github-deployment }} + steps: + - name: Checkout code + uses: actions/checkout@v6 + + - name: Install snapd snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: snapd + channel: ${{ inputs.snapd-channel }} + enable-cache: ${{ inputs.cache-host-snaps }} + + - name: Install core24 snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: core24 + channel: ${{ inputs.core24-channel }} + enable-cache: ${{ inputs.cache-host-snaps }} + + - name: Install snapcraft snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: snapcraft + channel: ${{ inputs.snapcraft-channel }} + classic: "true" + enable-cache: ${{ inputs.cache-host-snaps }} + + - name: Download snap artifact + if: ${{ inputs.artifact-pattern == '' }} + uses: actions/download-artifact@v4 + with: + name: ${{ inputs.artifact-name }} + path: ./snaps/ + + - name: Download snap artifacts + if: ${{ inputs.artifact-pattern != '' }} + uses: actions/download-artifact@v4 + with: + pattern: ${{ inputs.artifact-pattern }} + path: ./snaps/ + merge-multiple: true + + - name: Release to ${{ inputs.snapstore-channel }} + env: + SNAPCRAFT_STORE_CREDENTIALS: ${{ secrets.publish-credentials }} + SNAP_CHANNEL: ${{ inputs.snapstore-channel }} + run: | + shopt -s nullglob globstar + + for SNAP_FILE in ./snaps/**/*.snap; do + [ -f "$SNAP_FILE" ] || continue + + echo "Publishing $SNAP_FILE to $SNAP_CHANNEL" + + SNAP_NAME="$(basename "$SNAP_FILE")" + SNAP_NAME="${SNAP_NAME%.snap}" + SNAP_NAME="${SNAP_NAME%%_*}" + SNAP_DIR="$(dirname "$SNAP_FILE")" + + COMPONENT_ARGS=() + for comp in "$SNAP_DIR"/"$SNAP_NAME"+*.comp; do + [ -f "$comp" ] || continue + comp_file="$(basename "$comp")" + comp_stem="${comp_file%.comp}" + comp_tail="${comp_stem#*+}" + comp_name="${comp_tail%%_*}" + + if [ "$comp_tail" = "$comp_stem" ] || [ -z "$comp_name" ]; then + echo "::warning::Could not infer component name from $comp_file, skipping" + continue + fi + + echo "Adding component: $comp_name from $comp_file" + COMPONENT_ARGS+=(--component "$comp_name=$comp") + done + + if [ ${#COMPONENT_ARGS[@]} -gt 0 ]; then + echo "Publishing with components: ${COMPONENT_ARGS[*]}" + snapcraft upload --verbosity=brief "$SNAP_FILE" "${COMPONENT_ARGS[@]}" --release "$SNAP_CHANNEL" + else + echo "Publishing without components" + snapcraft upload --verbosity=brief "$SNAP_FILE" --release "$SNAP_CHANNEL" + fi + done diff --git a/.github/workflows/spread.yml b/.github/workflows/spread.yml new file mode 100644 index 000000000..a54c00cc4 --- /dev/null +++ b/.github/workflows/spread.yml @@ -0,0 +1,291 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +name: spread + +on: + workflow_call: + inputs: + job-arch: + type: string + description: Architecture label used in the GitHub job name + required: true + cache-host-snaps: + type: boolean + description: "Whether to cache host snaps (snapd, core22, core24) during + installation" + required: false + default: true + cache-pristine-images: + type: boolean + description: Use GitHub cache to store pristine images (recommended) + required: false + default: true + cache-prepared-images: + type: boolean + description: Use GitHub cache to store project-specific images (cache-intensive, + scales poorly with number of systems) + required: false + default: false + snapd-channel: + type: string + description: "Snap store channel for snapd snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + core24-channel: + type: string + description: "Snap store channel for core24 snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + image-garden-channel: + type: string + description: "Snap store channel for image-garden snap (e.g., latest/stable, + latest/edge)" + required: false + default: latest/stable + image-garden-components: + type: string + description: "Comma-separated list of image-garden components to install; + derived from spread-arch and spread-variant when empty" + required: false + default: "" + runs-on: + type: string + description: "The type of machine to run on (e.g., ubuntu-24.04)" + required: false + default: ubuntu-24.04 + spread-system: + type: string + description: "The name of the spread system to use (e.g., ubuntu-cloud-24.04)" + required: true + spread-arch: + type: string + description: "The name of the CPU architecture to use (e.g., x86_64)" + required: true + spread-tasks: + type: string + description: "The name of the spread tasks to run (e.g., tests/smoke)" + required: false + spread-variant: + type: string + description: "Variant of spread to use (empty string or plus); also selects + the matching image-garden component" + default: "" + required: false + spread-live: + type: boolean + description: "Enable live mode for spread (only available when spread-variant is + plus)" + required: false + default: false + spread-artifacts: + type: string + description: Path where spread saves task artifacts + required: false + default: "" + spread-artifacts-suffix: + type: string + description: Suffix appended to GitHub artifact archive with spread artifacts + required: false + default: "" + snap-artifact-name: + type: string + description: Name of the artifact containing the snap file (e.g., snap-x86_64) + required: false + default: "" + snap-filename: + type: string + description: Filename of the snap to download and test + required: false + default: "" + +jobs: + spread: + name: spread (${{ inputs.job-arch }}) - ${{ inputs.spread-system }} + runs-on: ${{ inputs.runs-on }} + steps: + - name: Checkout code + uses: actions/checkout@v6 + + - name: Download snap artifact + if: ${{ inputs.snap-artifact-name != '' }} + uses: actions/download-artifact@v7 + with: + name: ${{ inputs.snap-artifact-name }} + path: . + + - name: Work around a bug in snapd suspend logic + run: | + echo "::group::Work around a bug in snapd suspend logic" + sudo mkdir -p /etc/systemd/system/snapd.service.d + ( + echo "[Service]" + echo "Environment=SNAPD_STANDBY_WAIT=15m" + ) | sudo tee /etc/systemd/system/snapd.service.d/standby.conf + sudo systemctl daemon-reload + sudo systemctl restart snapd.service + echo "::endgroup::" + + - name: Install snapd snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: snapd + channel: ${{ inputs.snapd-channel }} + enable-cache: ${{ inputs.cache-host-snaps }} + + - name: Install core24 snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: core24 + channel: ${{ inputs.core24-channel }} + enable-cache: ${{ inputs.cache-host-snaps }} + + - name: Resolve spread configuration + id: spread-config + shell: bash + run: | + set -euo pipefail + spread_arch="${{ inputs.spread-arch }}" + spread_variant="${{ inputs.spread-variant }}" + spread_live="${{ fromJSON(inputs.spread-live) }}" + image_garden_components="${{ inputs.image-garden-components }}" + + case "$spread_variant" in + "") + spread_component="spread" + ;; + plus) + spread_component="spread-plus" + ;; + *) + echo "::error::Unsupported spread-variant: $spread_variant" + exit 1 + ;; + esac + + if [ "$spread_live" = "true" ] && [ "$spread_variant" != "plus" ]; then + echo "::error::spread-live requires spread-variant=plus" + exit 1 + fi + + if [ -z "$image_garden_components" ]; then + image_garden_components="qemu-${spread_arch//_/-},${spread_component}" + fi + + echo "image-garden-components=$image_garden_components" >> "$GITHUB_OUTPUT" + + - name: Install image-garden snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: image-garden + channel: ${{ inputs.image-garden-channel }} + components: ${{ steps.spread-config.outputs.image-garden-components }} + enable-cache: ${{ inputs.cache-host-snaps }} + + - name: Cache pristine virtual machine images + if: ${{ fromJSON(inputs.cache-pristine-images) }} + uses: actions/cache@v5 + with: + path: ~/snap/image-garden/common/cache/dl + key: image-garden-dl-${{ inputs.spread-system }}-${{ inputs.spread-arch }} + + - name: Cache prepared virtual machine images + uses: actions/cache@v5 + if: ${{ fromJSON(inputs.cache-prepared-images) }} + with: + path: | + .image-garden + !.image-garden/*.log + key: image-garden-img-${{ inputs.spread-system }}-${{ inputs.spread-arch }} + + - name: Make permissions on /dev/kvm more lax + run: | + if [ -c /dev/kvm ]; then + sudo chmod -v 666 /dev/kvm + fi + + - name: Use spread from image-garden snap + run: sudo snap alias image-garden.spread spread + + - name: Configure spread variant + if: ${{ inputs.spread-variant != '' }} + run: sudo snap set image-garden spread-variant=${{ inputs.spread-variant }} + + - name: Make the virtual machine image (dry run) + run: | + echo "::group::Make the virtual machine image (dry run)" + mkdir -p ~/snap/image-garden/common/cache/dl + image-garden make --debug --dry-run \ + ${{ inputs.spread-system }}.${{ inputs.spread-arch }}.qcow2 + echo "::endgroup::" + + - name: Make the virtual machine image + id: make-image + run: | + echo "::group::Make the virtual machine image" + IMAGE_EXISTS=false + if [ -f "${{ inputs.spread-system }}.${{ inputs.spread-arch }}.qcow2" ]; then + IMAGE_EXISTS=true + fi + image-garden make \ + ${{ inputs.spread-system }}.${{ inputs.spread-arch }}.qcow2 \ + ${{ inputs.spread-system }}.${{ inputs.spread-arch }}.run \ + ${{ inputs.spread-system }}.${{ inputs.spread-arch }}.user-data \ + ${{ inputs.spread-system }}.${{ inputs.spread-arch }}.meta-data \ + ${{ inputs.spread-system }}.${{ inputs.spread-arch }}.seed.iso + echo "::endgroup::" + # The image existed before this step => came from cache => needs rebase. + # The image did NOT exist => freshly built => no rebase needed. + echo "image-from-cache=$IMAGE_EXISTS" >> $GITHUB_OUTPUT + + - name: Rebase the virtual machine image + if: ${{ fromJSON(inputs.cache-prepared-images) && steps.make-image.outputs.image-from-cache == 'true' }} + run: | + image-garden rebase ${{ inputs.spread-system }}.${{ inputs.spread-arch }}.qcow2 + + - name: Run spread tests (${{ inputs.spread-system }}) + env: + SPREAD_ARCH: ${{ inputs.spread-arch }} + run: | + set +e + SPREAD_ARGS=() + if [ -n "${{ inputs.spread-artifacts }}" ]; then + SPREAD_ARGS+=(-artifacts "${{ inputs.spread-artifacts }}") + fi + if [ "${{ fromJSON(inputs.spread-live) }}" = "true" ]; then + SPREAD_ARGS+=(-live) + fi + spread "${SPREAD_ARGS[@]}" -v ${{ inputs.spread-system }}:${{ inputs.spread-tasks }} \ + 2>&1 | tee spread.log + SPREAD_EXIT_CODE=${PIPESTATUS[0]} + set -e + exit $SPREAD_EXIT_CODE + + - name: Upload spread log + if: always() + uses: actions/upload-artifact@v7 + with: + name: spread-log-${{ inputs.spread-artifacts-suffix || inputs.spread-system }} + path: spread.log + if-no-files-found: ignore + + - name: Rename artifacts for GitHub compatibility + if: ${{ inputs.spread-artifacts }} + shell: bash + run: | + find "${{ inputs.spread-artifacts }}" -depth -name '*:*' -execdir bash -c 'mv -v "$1" "${1//:/_}"' bash "{}" \; + + - name: Upload spread artifacts + if: ${{ inputs.spread-artifacts }} + uses: actions/upload-artifact@v7 + with: + name: spread-artifacts-${{ inputs.spread-artifacts-suffix || + inputs.spread-system }} + path: ${{ inputs.spread-artifacts }} + + - name: Upload image-garden boot logs (${{ inputs.spread-system }}) + if: ${{ failure() }} + uses: actions/upload-artifact@v7 + with: + name: boot-logs-${{ inputs.spread-system }} + path: .image-garden/*.log + if-no-files-found: ignore diff --git a/.github/workflows/tasteful-crafts.yml b/.github/workflows/tasteful-crafts.yml new file mode 100644 index 000000000..0a2eef794 --- /dev/null +++ b/.github/workflows/tasteful-crafts.yml @@ -0,0 +1,298 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +name: tasteful crafts + +on: + workflow_call: + inputs: + cache-host-snaps: + type: boolean + description: "Whether to cache host snaps (snapd, core22, core24) during installation" + required: false + default: true + cache-pristine-images: + type: boolean + description: Use GitHub cache to store pristine images (recommended) + required: false + default: true + cache-prepared-images: + type: boolean + description: Use GitHub cache to store project-specific images + required: false + default: false + snapd-channel: + type: string + description: "Snap store channel for snapd snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + core22-channel: + type: string + description: "Snap store channel for core22 snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + core24-channel: + type: string + description: "Snap store channel for core24 snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + lxd-channel: + type: string + description: "Snap store channel for LXD snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + snapcraft-channel: + type: string + description: "Snap store channel for Snapcraft snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + image-garden-channel: + type: string + description: "Snap store channel for image-garden snap (e.g., latest/stable, latest/edge)" + required: false + default: latest/stable + image-garden-components: + type: string + description: "Comma-separated list of image-garden components to install" + required: false + default: "" + spread-systems: + type: string + description: "JSON array of spread systems to test, or empty to auto-discover via spread -list" + required: false + default: "" + spread-tasks: + type: string + description: "The name of the spread tasks to run (e.g., tests/smoke)" + required: false + default: "" + spread-variant: + type: string + description: "Variant of spread to use (empty string or plus)" + required: false + default: "" + spread-live: + type: boolean + description: "Enable live mode for spread (only available when spread-variant is plus)" + required: false + default: false + pack-runs-on-amd64: + type: string + description: Runner label for the amd64 snap build + required: false + default: ubuntu-24.04 + # TODO: Switch to linux-amd64-cpu8 once those runners land upstream. + # Other Rust/build CI jobs (rust-native-build, docker-build, branch-checks, + # release-dev, release-tag, ci-image) already use linux-amd64-cpu8 and + # linux-arm64-cpu8, which provide significantly faster build times. + pack-runs-on-arm64: + type: string + description: Runner label for the arm64 snap build + required: false + default: ubuntu-24.04-arm + # TODO: Switch to linux-arm64-cpu8 once those runners land upstream. + spread-runs-on: + type: string + description: Runner label for amd64 spread jobs + required: false + default: ubuntu-24.04 + spread-runs-on-arm64: + type: string + description: Runner label for arm64 spread jobs (unused — no nested virtualization on GitHub hosted arm64 runners) + required: false + default: ubuntu-24.04-arm + snapstore-channel: + type: string + description: "Snap store channel to publish to (e.g., latest/edge)" + required: true + github-environment: + type: string + description: "Deployment environment (defaults to latest/candidate for tags, latest/edge otherwise)" + required: false + default: "" + github-deployment: + type: boolean + description: "Whether upload jobs should be marked as deployments" + required: false + default: true + skip-spread-tests: + type: boolean + description: "Skip integration tests (spread) and proceed directly to publishing. Requires manual approval." + required: false + default: false + secrets: + publish-credentials: + description: "GitHub secret containing Snap Store credentials for publishing" + required: true + +jobs: + # TODO: When spread.yaml is added to the repo, this gate enables + # integration testing of the snap before it can be published. The + # snapcraft-upload job checks this condition so only a valid spread + # test tree will allow promotion. + discover-spread-systems: + name: discover spread systems + runs-on: ${{ inputs.spread-runs-on }} + outputs: + spread-systems: ${{ steps.prepare.outputs.spread-systems }} + steps: + - name: Checkout code + uses: actions/checkout@v6 + + - name: Check for spread.yaml + id: check + shell: bash + run: | + if [ -f spread.yaml ]; then + echo "found=true" >> "$GITHUB_OUTPUT" + echo "spread.yaml found — integration tests enabled" + else + echo "found=false" >> "$GITHUB_OUTPUT" + echo "spread.yaml not found — skipping integration tests" + fi + + - name: Install snapd snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: snapd + channel: ${{ inputs.snapd-channel }} + enable-cache: ${{ inputs.cache-host-snaps }} + if: ${{ steps.check.outputs.found == 'true' }} + + - name: Install core24 snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: core24 + channel: ${{ inputs.core24-channel }} + enable-cache: ${{ inputs.cache-host-snaps }} + if: ${{ steps.check.outputs.found == 'true' }} + + - name: Install image-garden snap + uses: ./.github/actions/install-cached-snap + with: + snap-name: image-garden + channel: ${{ inputs.image-garden-channel }} + components: ${{ inputs.image-garden-components }} + enable-cache: ${{ inputs.cache-host-snaps }} + if: ${{ steps.check.outputs.found == 'true' }} + + - name: Use spread from image-garden snap + run: sudo snap alias image-garden.spread spread + if: ${{ steps.check.outputs.found == 'true' }} + + - name: Discover spread systems + id: discover + shell: bash + env: + INPUT_SPREAD_SYSTEMS: ${{ inputs.spread-systems }} + if: ${{ steps.check.outputs.found == 'true' }} + run: | + set -euo pipefail + if [ -n "$INPUT_SPREAD_SYSTEMS" ]; then + spread_systems="$INPUT_SPREAD_SYSTEMS" + else + spread_systems="$( + spread -list | cut -d : -f 2 | sort -u | + python3 -c 'import json, sys; print(json.dumps([line.strip() for line in sys.stdin if line.strip()]))' + )" + fi + + if [ "$spread_systems" = "[]" ]; then + echo "::error::No spread systems found in spread -list output" + exit 1 + fi + + echo "spread-systems=$spread_systems" >> "$GITHUB_OUTPUT" + echo "Spread systems: $spread_systems" + + - name: Set spread-systems output (skip or discover) + id: prepare + run: | + if [ "${{ steps.check.outputs.found }}" != "true" ]; then + # Use a sentinel value instead of "[]". An empty JSON array in a + # matrix produces zero entries, and GitHub Actions evaluates `if` + # conditions before matrix expansion — the job gets stuck instead + # of being skipped. "__skip__" guarantees exactly one matrix + # entry so the `if` can correctly suppress the job. + echo 'spread-systems=["__skip__"]' >> "$GITHUB_OUTPUT" + elif [ -n "${{ steps.discover.outputs.spread-systems }}" ]; then + echo "spread-systems=${{ steps.discover.outputs.spread-systems }}" >> "$GITHUB_OUTPUT" + else + echo 'spread-systems=["__skip__"]' >> "$GITHUB_OUTPUT" + fi + + snapcraft-pack-amd64: + name: snapcraft (amd64) + uses: ./.github/workflows/snapcraft-pack.yml + with: + job-arch: amd64 + cache-host-snaps: ${{ inputs.cache-host-snaps }} + snapd-channel: ${{ inputs.snapd-channel }} + core22-channel: ${{ inputs.core22-channel }} + core24-channel: ${{ inputs.core24-channel }} + lxd-channel: ${{ inputs.lxd-channel }} + snapcraft-channel: ${{ inputs.snapcraft-channel }} + runs-on: ${{ inputs.pack-runs-on-amd64 }} + + snapcraft-pack-arm64: + name: snapcraft (arm64) + uses: ./.github/workflows/snapcraft-pack.yml + with: + job-arch: arm64 + cache-host-snaps: ${{ inputs.cache-host-snaps }} + snapd-channel: ${{ inputs.snapd-channel }} + core22-channel: ${{ inputs.core22-channel }} + core24-channel: ${{ inputs.core24-channel }} + lxd-channel: ${{ inputs.lxd-channel }} + snapcraft-channel: ${{ inputs.snapcraft-channel }} + runs-on: ${{ inputs.pack-runs-on-arm64 }} + + spread-amd64: + name: spread (amd64) - ${{ matrix.spread-system }} + if: ${{ !inputs.skip-spread-tests && !contains(needs.discover-spread-systems.outputs.spread-systems, '__skip__') }} + needs: [discover-spread-systems, snapcraft-pack-amd64] + strategy: + matrix: + spread-system: ${{ fromJSON(needs.discover-spread-systems.outputs.spread-systems) }} + uses: ./.github/workflows/spread.yml + with: + job-arch: amd64 + cache-host-snaps: ${{ inputs.cache-host-snaps }} + cache-pristine-images: ${{ inputs.cache-pristine-images }} + cache-prepared-images: ${{ inputs.cache-prepared-images }} + snapd-channel: ${{ inputs.snapd-channel }} + core24-channel: ${{ inputs.core24-channel }} + image-garden-channel: ${{ inputs.image-garden-channel }} + image-garden-components: ${{ inputs.image-garden-components }} + runs-on: ${{ inputs.spread-runs-on }} + spread-system: ${{ matrix.spread-system }} + spread-arch: x86_64 + spread-tasks: ${{ inputs.spread-tasks }} + spread-variant: ${{ inputs.spread-variant }} + spread-live: ${{ fromJSON(inputs.spread-live) }} + snap-artifact-name: snap-${{ needs.snapcraft-pack-amd64.outputs.arch }} + snap-filename: ${{ needs.snapcraft-pack-amd64.outputs.snap-filename }} + + # Create GitHub environments that match the deployment target names used + # here (latest/edge for branch builds, latest/candidate for tag builds by + # default), then add an environment secret named + # SNAPCRAFT_STORE_CREDENTIALS. Export the secret value locally with: + # snapcraft export-login --snaps= --channels= \ + # --acls=package_upload,package_release + snapcraft-upload: + name: snapcraft upload + # Upload built snaps only after smoke testing the amd64 build first. + # When spread tests are skipped, proceed directly after packing is complete. + if: ${{ always() && (inputs.skip-spread-tests || needs.spread-amd64.result == 'success' || needs.spread-amd64.result == 'skipped' || needs.spread-amd64.result == 'cancelled') }} + needs: [snapcraft-pack-amd64, snapcraft-pack-arm64, spread-amd64] + uses: ./.github/workflows/snapcraft-upload.yml + with: + artifact-pattern: snap-* + cache-host-snaps: ${{ inputs.cache-host-snaps }} + snapd-channel: ${{ inputs.snapd-channel }} + core24-channel: ${{ inputs.core24-channel }} + snapcraft-channel: ${{ inputs.snapcraft-channel }} + snapstore-channel: ${{ inputs.github-environment || (github.ref_type == 'tag' && 'latest/candidate' || 'latest/edge') }} + github-environment: ${{ inputs.github-environment || (github.ref_type == 'tag' && 'latest/candidate' || 'latest/edge') }} + github-deployment: ${{ inputs.github-deployment }} + secrets: + publish-credentials: ${{ secrets.publish-credentials }} diff --git a/deploy/snap/PUBLISHING.md b/deploy/snap/PUBLISHING.md new file mode 100644 index 000000000..c1bea8110 --- /dev/null +++ b/deploy/snap/PUBLISHING.md @@ -0,0 +1,67 @@ +# Snap Store Publishing + +This document describes how to set up the GitHub environments and secrets +required for publishing the `openshell` snap to the Snap Store. + +## Overview + +The pipeline uses three GitHub environments, each with a +`SNAPCRAFT_STORE_CREDENTIALS` secret. Each secret holds a macaroon +generated by `snapcraft export-login`. + +## Environment Setup + +On a system with snapcraft signed into the Snap Store, create three +macaroon files: + +### 1. `latest/edge` — nightly/development builds + +Allows publishing and releasing to the edge channel. + +```sh +snapcraft export-login --snaps=openshell --channels=latest/edge \ + --acls=package_push,package_release snapcraft-edge-macaroon +``` + +### 2. `latest/candidate` — pre-release builds + +Identical to edge but targets candidate. + +```sh +snapcraft export-login --snaps=openshell --channels=latest/candidate \ + --acls=package_push,package_release snapcraft-candidate-macaroon +``` + +### 3. `latest/stable` — production releases + +Promotion from candidate to stable requires different ACLs — access to +an existing build plus release rights. + +```sh +snapcraft export-login --snaps=openshell --channels=latest/stable \ + --acls=package_access,package_release snapcraft-stable-macaroon +``` + +## GitHub Configuration + +The GitHub repository needs three environments created. Note that +environments are first-class objects related to deployments, not plain +environment variables. + +For each environment, create an environment secret called +`SNAPCRAFT_STORE_CREDENTIALS` and set it to the contents of the +corresponding macaroon file created above. + +| Environment | Secret | Target Channel | +|-------------|--------|----------------| +| edge | `SNAPCRAFT_STORE_CREDENTIALS` (edge macaroon) | latest/edge | +| candidate | `SNAPCRAFT_STORE_CREDENTIALS` (candidate macaroon) | latest/candidate | +| stable | `SNAPCRAFT_STORE_CREDENTIALS` (stable macaroon) | latest/stable | + +## Channel Routing + +| Trigger | Job | Target Channel | +|---------|-----|----------------| +| Push to branch | Upload & release untagged builds | latest/edge | +| Push tag | Upload & release tagged builds | latest/candidate | +| Manual (promote) | Promote between channels | as specified |