Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 13 additions & 4 deletions .github/workflows/qubes-dom0-packagev2.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,11 @@ on:
Forced version of a package.
required: false
type: string
qubes-component-branch:
description: >
Forced repository branch to build component from
required: false
type: string

jobs:
build-and-package:
Expand Down Expand Up @@ -79,6 +84,7 @@ jobs:
PKG_DIR: ${{ inputs.qubes-pkg-src-dir }}
PKG_REV: ${{ inputs.qubes-pkg-revision }}
PKG_VER: ${{ inputs.qubes-pkg-version }}
BUILD_BRANCH: ${{ inputs.qubes-component-branch }}
# Following 2 variables are used in double expansion '${${{ github.ref_type }}}',
# do not change these names even though they don't follow the convention.
branch: ${{ github.head_ref }}
Expand All @@ -88,11 +94,14 @@ jobs:
# Switch from Qubes to Docker executor
sed -i "/^executor:$/,+4d; /^#executor:$/,+3s/#//" builder.yml

branch_name=${${{ github.ref_type }}}
branch_name="${BUILD_BRANCH}"
if [ -z "$branch_name" ]; then
# github.head_ref is set only for pull requests, this should
# handle pushes
branch_name=$(basename "$GITHUB_REF")
branch_name=${${{ github.ref_type }}}
if [ -z "$branch_name" ]; then
# github.head_ref is set only for pull requests, this should
# handle pushes
branch_name=$(basename "$GITHUB_REF")
fi
fi

if [ -n "$PKG_DIR" ]; then
Expand Down
85 changes: 85 additions & 0 deletions .github/workflows/rebase.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
---
name: Try rebasing on updated upstream, report in case of conflicts

on:
workflow_call:
secrets:
first-remote-token:
description: >
Personal access token for performing the following operations on the
downstream-repo: fetch the repository, create a branch, delete a
branch, create commits on a branch, push to a branch, open a PR, close
a PR, get list of PRs.
required: true
inputs:
downstream-repo:
description: >
<first_repo> parameter for the rebase.sh script.
required: true
type: string
downstream-branch:
description: >
<first_repo_branch> parameter for the rebase.sh script.
required: true
type: string
upstream-repo:
description: >
<second_repo> parameter for the rebase.sh script.
required: true
type: string
upstream-branch:
description: >
<second_repo_branch> parameter for the rebase.sh script.
required: true
type: string
commit-user-name:
description: >
NAME parameter for the --upstream-branch option of the rebase.sh
script.
required: true
type: string
commit-user-email:
description: >
EMAIL parameter for the --commit-user-email option of the rebase.sh
script.
required: true
type: string
cicd-trigger-resume:
description: >
MESSAGE parameter for the --cicd-trigger-resume option of the
rebase.sh script.
required: true
type: string

jobs:
build-and-package:
runs-on: ubuntu-latest
name: Try rebasing on updated upstream, report in case of conflicts
permissions:
# For creation/deletion/pushing to branches and creating PRs
contents: write
steps:
- uses: actions/checkout@v6
with:
repository: TrenchBoot/.github
path: shared
ref: master
- name: Run script for rebasing
env:
FIRST_REMOTE_TOKEN: ${{ secrets.first-remote-token }}
DOWNSTREAM_REPO: ${{ inputs.downstream-repo }}
DOWNSTREAM_BRANCH: ${{ inputs.downstream-branch }}
UPSTREAM_REPO: ${{ inputs.upstream-repo }}
UPSTREAM_BRANCH: ${{ inputs.upstream-branch }}
NAME: ${{ inputs.commit-user-name }}
EMAIL: ${{ inputs.commit-user-email }}
MESSAGE: ${{ inputs.cicd-trigger-resume }}
run: |
shared/scripts/rebase.sh --first-remote-token "$FIRST_REMOTE_TOKEN" \
--commit-user-name "$NAME" \
--commit-user-email "$EMAIL" \
--cicd-trigger-resume "$MESSAGE" \
"$DOWNSTREAM_REPO" \
"$DOWNSTREAM_BRANCH" \
"$UPSTREAM_REPO" \
"$UPSTREAM_BRANCH"
73 changes: 73 additions & 0 deletions .github/workflows/trigger-woodpecker-pipeline.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name: Trigger a Woodpecker CI/CD pipeline

on:
workflow_call:
inputs:
api-url:
description: >
Base URL of the Woodpecker instance, e.g. https://ci.example.com.
--api-url parameter for the woodpecker-trigger.sh script.
required: true
type: string
owner:
description: >
Repository owner (user or organization).
--owner parameter for the woodpecker-trigger.sh script.
required: true
type: string
repo:
description: >
Repository name.
--repo parameter for the woodpecker-trigger.sh script.
required: true
type: string
ref:
description: >
Branch to trigger the pipeline on.
--ref parameter for the woodpecker-trigger.sh script.
required: false
type: string
default: 'main'
inputs:
description: >
Additional --input flags to pass to woodpecker-trigger.sh, e.g.
"--input KEY=VALUE --input KEY2=VALUE2". Keys must be valid shell
variable names (no hyphens).
required: false
type: string
default: ''
secrets:
woodpecker-token:
description: >
Woodpecker API token for triggering the pipeline.
--token parameter for the woodpecker-trigger.sh script.
required: true

jobs:
trigger-woodpecker:
runs-on: ubuntu-latest
name: Trigger a Woodpecker CI/CD pipeline
permissions:
contents: read
steps:
- uses: actions/checkout@v4
with:
repository: TrenchBoot/.github
path: shared
ref: master
- name: Trigger Woodpecker CI/CD pipeline
env:
WOODPECKER_TOKEN: ${{ secrets.woodpecker-token }}
WOODPECKER_API_URL: ${{ inputs.api-url }}
WOODPECKER_OWNER: ${{ inputs.owner }}
WOODPECKER_REPO: ${{ inputs.repo }}
WOODPECKER_REF: ${{ inputs.ref }}
WOODPECKER_INPUTS: ${{ inputs.inputs }}
run: |
shared/scripts/woodpecker-trigger.sh \
--token "$WOODPECKER_TOKEN" \
--api-url "$WOODPECKER_API_URL" \
--owner "$WOODPECKER_OWNER" \
--repo "$WOODPECKER_REPO" \
--ref "$WOODPECKER_REF" \
$WOODPECKER_INPUTS
154 changes: 141 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,21 +68,52 @@ package, hence significantly reduced set of parameters.
There is also no need to use `qubes-builder-docker/` in this case because
builder's repository contains its own Docker image.

| Parameter | Type | Req. | Def. | Description
| --------- | ---- | ---- | ---- | -----------
| `qubes-component` | string | Yes | - | Name of QubesOS component as recognized by its build system.
| `qubes-pkg-src-dir` | string | No | - | Relative path to directory containing Qubes OS package.
| `qubes-pkg-version` | string | No | auto | Version for RPM packages
| `qubes-pkg-revision` | string | No | `1` | Revision for RPM packages

Used by [TrenchBoot/qubes-antievilmaid][aem] and
[TrenchBoot/secure-kernel-loader][skl]. The latter makes use of
`qubes-pkg-src-dir` as Qubes OS package is stored within the repository itself.
| Parameter | Type | Req. | Def. | Description
| --------- | ---- | ---- | ---- | -----------
| `qubes-component` | string | Yes | - | Name of QubesOS component as recognized by its build system.
| `qubes-pkg-src-dir` | string | No | - | Relative path to directory containing Qubes OS package.
| `qubes-pkg-version` | string | No | auto | Version for RPM packages
| `qubes-pkg-revision` | string | No | `1` | Revision for RPM packages
| `qubes-component-branch` | string | No | - | Forced repository branch to build component from

[qubes-builder-v2]: https://github.com/QubesOS/qubes-builderv2
[aem]: https://github.com/TrenchBoot/qubes-antievilmaid/blob/2b6b796e31789fca599986c9cfb0a3ceced5967d/.github/workflows/build.yml
[skl]: https://github.com/TrenchBoot/secure-kernel-loader

### rebase

This workflow automates rebasing a downstream repository branch on top of an
upstream branch. On success, it pushes the rebased branch. If conflicts arise
it, opens a pull request against the downstream repository to ask for
resolution.

| Parameter | Type | Req. | Def. | Description
| --------- | ---- | ---- | ---- | -----------
| `downstream-repo` | string | Yes | - | URL of the repository to rebase (`<first_repo>` argument of `rebase.sh`).
| `downstream-branch` | string | Yes | - | Branch in the downstream repository to rebase (`<first_repo_branch>` argument of `rebase.sh`).
| `upstream-repo` | string | Yes | - | URL of the repository that provides the new base (`<second_repo>` argument of `rebase.sh`).
| `upstream-branch` | string | Yes | - | Branch in the upstream repository to rebase onto (`<second_repo_branch>` argument of `rebase.sh`).
| `commit-user-name` | string | Yes | - | Git author name used for rebase commits (`--commit-user-name` option of `rebase.sh`).
| `commit-user-email` | string | Yes | - | Git author e-mail used for rebase commits (`--commit-user-email` option of `rebase.sh`).
| `cicd-trigger-resume` | string | Yes | - | Human-readable message appended to the conflict PR describing how to resume the pipeline (`--cicd-trigger-resume` option of `rebase.sh`).
| `first-remote-token` | string | Yes | - | Personal access token with permissions to fetch, branch, commit, push, and open/close PRs on `downstream-repo`. Passed as a GitHub Actions secret.

### trigger-woodpecker-pipeline

This workflow is a generic wrapper for the woodpecker-trigger.sh script for
triggering Woodpecker CI/CD pipelines on some remote Woodpecker instance. As for
now it is used only for triggering the pipelines for signing RPM packages built
by the `qubes-dom0-package` and `qubes-dom0-packagev2` workflows.

| Parameter | Type | Req. | Def. | Description
| --------- | ---- | ---- | ---- | -----------
| `api-url` | string | Yes | - | Base URL of the Woodpecker instance, e.g. `https://ci.example.com`.
| `owner` | string | Yes | - | Repository owner (user or organization).
| `repo` | string | Yes | - | Repository name.
| `ref` | string | No | `main` | Branch to trigger the pipeline on.
| `inputs` | string | No | - | Additional `--input KEY=VALUE` flags passed to `woodpecker-trigger.sh`. Keys must be valid shell variable names (no hyphens).
| `woodpecker-token` | string | Yes | - | Woodpecker API token for authentication. Passed as a GitHub Actions secret.

## Usage

Full details can be found in [GitHub's documentation][workflow-docs] on
Expand All @@ -91,20 +122,22 @@ modifications to workflows are necessary.

[workflow-docs]: https://docs.github.com/en/actions/using-workflows/reusing-workflows

### qubes-dom0-package or qubes-dom0-packagev2

Create a workflow file like `.github/workflows/build.yml` inside of your
repository. It will have 3 parts: name, triggering conditions and invocation
of one of the workflows defined here. Let's use [TrenchBoot/grub][grub] as an
example.

### Name
#### Name

```yaml
name: Test build and package QubesOS RPMs
```

Specify workflow title used for identification in UI.

### Triggering conditions
#### Triggering conditions

```yaml
on:
Expand All @@ -118,7 +151,7 @@ on:
Activate this workflow on push of any tag or a branch which starts with
`intel-txt-aem` (including this branch, i.e. `*` can expand to an empty string).

### Workflow invocation
#### Workflow invocation

```yaml
jobs:
Expand All @@ -134,6 +167,101 @@ jobs:
Invoke v1 workflow from `master` branch of this repository with the set of
parameters as described in a section above.

### rebase

`rebase` is typically one job in a larger workflow that first prepares the
upstream branch to rebase onto, then calls this workflow, and finally cleans up
any temporary branches.

#### Triggering conditions

There is no specific trigger condition that can be used to trigger pipelines
that contain this reusable workflow. So the developer is free to decide. But
there is one case: if the workflow that uses this reusable workflow has a
condition on push event, then the token provided via `first-remote-token` should
not have permissions to trigger CI/CDs. This is because the script used inside
this reusable workflow pushes to the remote repository several times.

#### Workflow invocation

```yaml
name: Rebase on top of QubesOS main

on:
workflow_dispatch:
schedule:
- cron: '0 0 * * 6'

jobs:
try-rebase:
uses: TrenchBoot/.github/.github/workflows/rebase.yml@master
secrets:
first-remote-token: ${{secrets.TRENCHBOOT_REBASE_TOKEN}}
permissions:
# For creation/deletion/pushing to branches and creating PRs
contents: write
with:
downstream-repo: 'https://github.com/DaniilKl/qubes-antievilmaid.git'
downstream-branch: 'main'
upstream-repo: 'https://github.com/QubesOS/qubes-antievilmaid.git'
upstream-branch: 'main'
commit-user-name: 'github-actions[bot]'
commit-user-email: 'github-actions[bot]@users.noreply.github.com'
cicd-trigger-resume: '7. Rerun the workflow https://github.com/DaniilKl/qubes-antievilmaid/actions/runs/${{ github.run_id }} to resume automated rebase.'
```

### trigger-woodpecker-pipeline

`trigger-woodpecker-pipeline` is meant to be added as an additional job to an
existing workflow, chained after a `qubes-dom0-package` or `qubes-dom0-packagev2`
job.

#### Workflow invocation

An example invocation:

```yaml
jobs:
qubes-dom0-package:
needs: get-version
uses: TrenchBoot/.github/.github/workflows/qubes-dom0-packagev2.yml@master
with:
qubes-component: 'vmm-xen'
qubes-component-branch: 'aem-next-rebased'
qubes-pkg-src-dir: '.'
qubes-pkg-version: '4.19.4'
trigger-woodpecker-cicd:
needs: qubes-dom0-package
uses: TrenchBoot/.github/.github/workflows/trigger-woodpecker-pipeline.yml@master
secrets:
woodpecker-token: ${{ secrets.WOODPECKER_TOKEN }}
with:
api-url: 'https://ci.3mdeb.com'
owner: 'zarhus'
repo: 'trenchboot-release-cicd-pipeline'
ref: 'master'
inputs: >-
--input GITHUB_REPO=xen
--input GITHUB_SHA=${{ github.sha }}
--input GITHUB_RUN_ID=${{ github.run_id }}
--input QUBES_COMPONENT=vmm-xen
--input WORKFLOW=sign-and-publish-test-rpms
```

Invokes the workflow from `master` branch of this repository after the
`qubes-dom0-package` job completes. Pass the Woodpecker API token from the
repository's GitHub secrets, point it at the target Woodpecker instance
and repository, and supply any pipeline-specific key/value pairs via repeated
`--input` flags.

Note, that all the inputs to the `trigger-woodpecker-pipeline.yml` except from
the `inputs` serve for the purpose of connection to the desired Woodpecker
instance on which a pipeline for signing is running. But the data provided via
`inputs` input and `--input` flag is consumed by the signing pipeline itself.
One must specify the name of the signing pipeline via `--input WORKFLOW=` and
all the input data the specified pipeline requires. The above example presents
the required inputs for the `sign-and-publish-test-rpms` pipeline.

## Funding

This project was partially funded through the
Expand Down
Loading