Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
237 changes: 237 additions & 0 deletions .github/workflows/docker-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,237 @@
name: Reusable - Docker Publish (chunked)

# Builds a container image and pushes it to the OneLiteFeather Harbor registry
# using chunked blob uploads via regctl. This avoids the "request entity too
# large" / 504 gateway timeout failures that occur when a single image layer
# exceeds the request-body limit of a proxy in front of the registry (e.g. the
# 100 MB Cloudflare limit): blobs larger than `blob-chunk` are split into
# multiple smaller PATCH requests instead of one monolithic PUT.
# The pushed image is optionally signed keyless with cosign via GitHub OIDC.

on:
workflow_call:
inputs:
image-name:
description: "Image repository path inside the registry, e.g. 'otis/otis' (the registry host comes from the HARBOR_REGISTRY secret)"
required: true
type: string
version:
description: "Version/tag of the image, e.g. '1.13.1' (without a leading 'v')"
required: true
type: string
context:
description: "Docker build context directory"
required: false
type: string
default: "."
dockerfile:
description: "Path to the Dockerfile (defaults to <context>/Dockerfile)"
required: false
type: string
default: ""
build-command:
description: "Optional shell command run before the docker build to produce the context (e.g. a Gradle task). The version is available as $VERSION."
required: false
type: string
default: ""
setup-java:
description: "Set up JDK + Gradle before running build-command (for Gradle-produced contexts)"
required: false
type: boolean
default: false
java-version:
description: "JDK version to use when setup-java is true"
required: false
type: string
default: "25"
java-distribution:
description: "JDK distribution (temurin, zulu, ...)"
required: false
type: string
default: "temurin"
extra-tags:
description: "Additional docker/metadata-action tag lines appended to the default semver + sha tags"
required: false
type: string
default: ""
blob-chunk:
description: "Chunk size in bytes for blob uploads above which chunked push is used (default 50 MiB, safely below a 100 MB proxy limit). Larger chunks = fewer round-trips and fewer storage-backend parts, but stay under the proxy cap."
required: false
type: string
default: "52428800"
req-concurrent:
description: "How many layers (blobs) regctl uploads concurrently. Chunks within a single layer stay sequential. Lower this if the registry's storage backend returns 5xx under load."
required: false
type: string
default: "3"
sign:
description: "Keyless-sign the pushed image with cosign via the GitHub OIDC identity. The calling job must grant `id-token: write`."
required: false
type: boolean
default: true
regctl-version:
description: "regctl release tag to install"
required: false
type: string
default: "v0.11.5"
runs-on:
description: "Runner image"
required: false
type: string
default: "ubuntu-latest"
secrets:
HARBOR_REGISTRY:
required: true
HARBOR_USERNAME:
required: true
HARBOR_PASSWORD:
required: true
outputs:
image:
description: "Fully qualified image reference (registry/image-name)"
value: ${{ jobs.docker.outputs.image }}
digest:
description: "Manifest digest of the pushed image"
value: ${{ jobs.docker.outputs.digest }}

concurrency:
group: docker-publish-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false

permissions:
contents: read
id-token: write # keyless cosign signing (caller must also grant id-token: write)

jobs:
docker:
name: Build & push (chunked)
runs-on: ${{ inputs.runs-on }}
outputs:
image: ${{ steps.vars.outputs.image }}
digest: ${{ steps.push.outputs.digest }}
env:
VERSION: ${{ inputs.version }}
BLOB_CHUNK: ${{ inputs.blob-chunk }}
REQ_CONCURRENT: ${{ inputs.req-concurrent }}
steps:
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0

- name: Resolve image reference
id: vars
shell: bash
env:
HARBOR_REGISTRY: ${{ secrets.HARBOR_REGISTRY }}
IMAGE_NAME: ${{ inputs.image-name }}
run: |
echo "image=${HARBOR_REGISTRY}/${IMAGE_NAME}" >> "$GITHUB_OUTPUT"

- name: Validate Gradle wrapper
if: ${{ inputs.setup-java }}
uses: gradle/actions/wrapper-validation@v6

- name: Set up JDK ${{ inputs.java-version }}
if: ${{ inputs.setup-java }}
uses: actions/setup-java@v5
with:
distribution: ${{ inputs.java-distribution }}
java-version: ${{ inputs.java-version }}

- name: Setup Gradle
if: ${{ inputs.setup-java }}
uses: gradle/actions/setup-gradle@v6

- name: Build context
if: ${{ inputs.build-command != '' }}
shell: bash
run: ${{ inputs.build-command }}

- name: Docker meta
id: meta
uses: docker/metadata-action@v6
with:
images: |
${{ steps.vars.outputs.image }}
tags: |
type=semver,pattern={{version}},value=${{ inputs.version }}
type=semver,pattern={{major}}.{{minor}},value=${{ inputs.version }}
type=semver,pattern={{major}},value=${{ inputs.version }}
type=sha
${{ inputs.extra-tags }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4

- name: Build image to OCI archive
uses: docker/build-push-action@v7
with:
context: ${{ inputs.context }}
file: ${{ inputs.dockerfile != '' && inputs.dockerfile || format('{0}/Dockerfile', inputs.context) }}
push: false
provenance: false
sbom: false
labels: ${{ steps.meta.outputs.labels }}
tags: ${{ steps.vars.outputs.image }}:${{ inputs.version }}
outputs: type=oci,dest=${{ runner.temp }}/image.tar

- name: Install regctl
shell: bash
run: |
curl -fsSL "https://github.com/regclient/regclient/releases/download/${{ inputs.regctl-version }}/regctl-linux-amd64" \
-o "${RUNNER_TEMP}/regctl"
chmod +x "${RUNNER_TEMP}/regctl"
echo "${RUNNER_TEMP}" >> "$GITHUB_PATH"

- name: Configure chunked uploads and login
shell: bash
env:
HARBOR_REGISTRY: ${{ secrets.HARBOR_REGISTRY }}
HARBOR_USERNAME: ${{ secrets.HARBOR_USERNAME }}
HARBOR_PASSWORD: ${{ secrets.HARBOR_PASSWORD }}
run: |
# regctl is the registry client (it chunks blob uploads, unlike docker push).
# --skip-check skips its anonymous connectivity ping, which a private
# registry answers with 401; credentials are exercised during the push.
regctl registry set --skip-check --blob-chunk "${BLOB_CHUNK}" --blob-max "${BLOB_CHUNK}" --req-concurrent "${REQ_CONCURRENT}" "${HARBOR_REGISTRY}"
printf '%s' "${HARBOR_PASSWORD}" | \
regctl registry login "${HARBOR_REGISTRY}" -u "${HARBOR_USERNAME}" --pass-stdin --skip-check

- name: Push image (chunked)
id: push
shell: bash
env:
IMAGE: ${{ steps.vars.outputs.image }}
TAGS: ${{ steps.meta.outputs.tags }}
run: |
# Push the primary tag with chunked blob uploads, then add the
# remaining tags via in-registry copy (cross-repo blob mount = tiny requests).
regctl image import "${IMAGE}:${VERSION}" "${RUNNER_TEMP}/image.tar"
while IFS= read -r tag; do
[ -z "$tag" ] && continue
[ "$tag" = "${IMAGE}:${VERSION}" ] && continue
regctl image copy "${IMAGE}:${VERSION}" "$tag"
done <<< "${TAGS}"
DIGEST="$(regctl image digest "${IMAGE}:${VERSION}")"
echo "digest=${DIGEST}" >> "$GITHUB_OUTPUT"

- name: Install Cosign
if: ${{ inputs.sign }}
uses: sigstore/cosign-installer@v4.1.2

- name: Sign image (keyless)
if: ${{ inputs.sign }}
shell: bash
env:
IMAGE: ${{ steps.vars.outputs.image }}
DIGEST: ${{ steps.push.outputs.digest }}
COSIGN_USERNAME: ${{ secrets.HARBOR_USERNAME }}
COSIGN_PASSWORD_REG: ${{ secrets.HARBOR_PASSWORD }}
run: |
# Keyless signing via the GitHub Actions OIDC identity (no key/password).
# All tags share one manifest digest, so a single signature covers them all.
cosign sign --yes \
--registry-username="${COSIGN_USERNAME}" \
--registry-password="${COSIGN_PASSWORD_REG}" \
"${IMAGE}@${DIGEST}"
66 changes: 66 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ repositories by referencing a tagged release of this repo.
| --- | --- |
| `.github/workflows/gradle-build-pr.yml` | Build & test a Gradle project on pull requests across a runner matrix. Skips when no Gradle-relevant files changed; aggregates JUnit results across the matrix; auto-enables verbose logging on debug re-runs. |
| `.github/workflows/gradle-publish.yml` | Build & publish a Gradle project to the OneLiteFeather Maven repository on tag pushes. |
| `.github/workflows/docker-publish.yml` | Build a container image and push it to the OneLiteFeather Harbor registry using **chunked blob uploads** (via [`regctl`](https://regclient.org/)) so no single request exceeds the proxy body limit; optionally **keyless-signs** the image with cosign (GitHub OIDC). |
| `.github/workflows/release-please.yml` | Run [release-please](https://github.com/googleapis/release-please) for a repository. |
| `.github/workflows/close-invalid-prs.yml` | Close PRs opened from a fork's default branch with a configurable message. |
| `.github/workflows/markdown-lint.yml` | Lint Markdown files with [`markdownlint-cli2`](https://github.com/DavidAnson/markdownlint-cli2-action) and check links with [`lychee`](https://github.com/lycheeverse/lychee-action). |
Expand Down Expand Up @@ -106,6 +107,62 @@ jobs:
secrets: inherit
```

### Publish a Docker image (chunked upload)

Pushes the image one blob at a time in chunks below the proxy body limit, so
large layers no longer fail with `413 Request Entity Too Large` / `504 Gateway
Timeout` behind a proxy such as Cloudflare (100 MB limit). Plain `docker push`
/ `buildx` cannot chunk a blob; this workflow builds the image to an OCI archive
and pushes it with [`regctl`](https://regclient.org/) (`--blob-chunk` /
`--blob-max`) instead.

Typical use is from a release job, gated on `release_created`:

Typical use is from a release job, gated on `release_created`. Grant
`id-token: write` on the calling job so cosign can sign keyless:

```yaml
jobs:
docker:
needs: release-please
if: needs.release-please.outputs.release_created == 'true'
permissions:
contents: read
id-token: write # required for keyless cosign signing
uses: OneLiteFeatherNET/workflows/.github/workflows/docker-publish.yml@v2
with:
image-name: "otis/otis" # registry host comes from HARBOR_REGISTRY
version: ${{ needs.release-please.outputs.version }}
# Build the container context with Gradle first ($VERSION is exported):
setup-java: true
build-command: "./gradlew jar optimizedBuildLayers optimizedDockerfile -Pversion=$VERSION"
context: "./backend/build/docker/optimized"
secrets: inherit
```

For a project with a plain `Dockerfile` checked into the repo, drop the Gradle
inputs (and the `permissions` block if you set `sign: false`):

```yaml
jobs:
docker:
uses: OneLiteFeatherNET/workflows/.github/workflows/docker-publish.yml@v2
with:
image-name: "myteam/myapp"
version: "1.2.3"
context: "."
sign: false # skip signing (no id-token needed)
secrets: inherit
```

Default tags are `{{version}}`, `{{major}}.{{minor}}`, `{{major}}` and a
`sha-` tag; add more via `extra-tags`. Tune chunking with `blob-chunk` (bytes,
default 50 MiB) and parallel layer uploads with `req-concurrent`. Signing is
keyless via GitHub OIDC — no signing key/secret to manage; verify with the
workflow identity (`--certificate-identity-regexp` + `--certificate-oidc-issuer
https://token.actions.githubusercontent.com`). The pushed manifest `digest` and
full `image` reference are exposed as workflow outputs.

### release-please

```yaml
Expand Down Expand Up @@ -168,6 +225,15 @@ these secrets to be available in the caller repository (and forwarded via
- `ONELITEFEATHER_MAVEN_USERNAME`
- `ONELITEFEATHER_MAVEN_PASSWORD`

`docker-publish` pushes to the Harbor registry, so it expects:

- `HARBOR_REGISTRY` — registry host (no scheme), e.g. `harbor.onelitefeather.dev`
- `HARBOR_USERNAME`
- `HARBOR_PASSWORD`

Signing is keyless (cosign + GitHub OIDC) — no signing secrets. The calling job
just needs `permissions: id-token: write` when `sign: true` (the default).

## Test results

For `gradle-build-pr`, JUnit XML from every matrix job is uploaded as
Expand Down
Loading