diff --git a/.github/workflows/release_candidate.yaml b/.github/workflows/release_candidate.yaml index 28a4b0931..859b25780 100644 --- a/.github/workflows/release_candidate.yaml +++ b/.github/workflows/release_candidate.yaml @@ -78,9 +78,13 @@ jobs: --output-dir release/ci - name: Audit source archive + env: + # The creator must ignore caller-provided gzip defaults. + GZIP: "-9" run: | scripts/releasing/verify_release_candidate.sh \ --allow-unsigned \ + --git-ref HEAD \ --skip-build \ "release/ci/apache-paimon-cpp-${RELEASE_VERSION}-src.tgz" diff --git a/scripts/releasing/README.md b/scripts/releasing/README.md index ee558c1e3..a251bed86 100644 --- a/scripts/releasing/README.md +++ b/scripts/releasing/README.md @@ -34,9 +34,9 @@ Before starting a release: - obtain an ASF code-signing key, publish it through the ASF account system, and make sure it is present in [Paimon KEYS](https://downloads.apache.org/paimon/KEYS); -- install `git`, `gpg`, `svn`, `gh`, `python3`, `curl` or `wget`, Java, CMake, - Ninja, and the toolchain needed by `ci/scripts/build_paimon.sh` (Java is - required by Apache RAT); +- install `git`, GNU `gzip`, `gpg`, `svn`, `gh`, `python3`, `curl` or `wget`, + Java, CMake, Ninja, and the toolchain needed by + `ci/scripts/build_paimon.sh` (Java is required by Apache RAT); - authenticate `gh` with access to read GitHub Actions runs in `apache/paimon-cpp`; - make sure the Apache Git remote points directly to @@ -44,6 +44,15 @@ Before starting a release: - prepare and merge a release-preparation PR that updates the release notes and all version metadata, and passes the normal and release-candidate workflows. +The source archive uses GNU gzip with fixed options so that macOS and Linux +produce the same bytes. On macOS, install Homebrew gzip and either put it first +on `PATH` or select it explicitly: + +```bash +brew install gzip +export PAIMON_GZIP="$(brew --prefix gzip)/bin/gzip" +``` + For example, update all version locations and review the diff: ```bash @@ -80,10 +89,13 @@ The release scripts use `vVERSION-rcRC` for release-candidate tags and Start from the exact clean commit approved for the candidate. Before publishing, the wrapper fetches the release branch and requires `HEAD` to be contained in -its current history. It then creates and verifies a signed RC tag, creates the -source archive and its checksum/signature, performs the full source-release -verification, pushes the tag, waits for the tag-triggered release-candidate -workflow to succeed, and imports the artifacts into ASF `dist/dev`: +its current history. It then creates and verifies a signed RC tag, pushes the +tag, and waits for the tag-triggered release-candidate workflow. That workflow +creates the canonical source archive and checksum, builds and tests the same +archive with GCC and Clang, and uploads it as a workflow artifact. The wrapper +downloads those exact bytes, confirms that they are reproducible from the tag, +signs the archive locally, performs the full source-release verification, and +imports the three files into ASF `dist/dev`: ```bash scripts/releasing/release_rc.sh \ @@ -96,12 +108,13 @@ scripts/releasing/release_rc.sh \ The release branch defaults to `main`; use `--release-branch NAME` for a maintenance release from another Apache branch. -Use `--prepare-only` to create and verify artifacts without pushing the tag or -uploading to ASF infrastructure. This local-only mode does not require `HEAD` -to match the remote release branch. Use `--dry-run` to print identifiers -without making changes. A resumed run reuses an existing local tag or complete -artifact set only after validating it. A prepare-only run does not print a vote -email and must not be used to start a vote. +Use `--prepare-only` to create and verify preview artifacts without pushing the +tag or uploading to ASF infrastructure. This local-only mode does not require +`HEAD` to match the remote release branch. A preview is not authoritative: a +published run downloads the workflow artifact and rejects an existing local +archive if its bytes differ. Use `--dry-run` to print identifiers without +making changes. A prepare-only run does not print a vote email and must not be +used to start a vote. The candidate directory contains: @@ -164,13 +177,16 @@ The verifier checks: - installation plus compilation and execution of an external CMake consumer. Pass `--git-ref v0.3.0-rc1` when the Git repository is available to regenerate -the archive from the signed tag and compare it byte-for-byte. +the archive from the signed tag with GNU gzip and compare it byte-for-byte. +This check requires GNU gzip on every platform; it intentionally rejects the +macOS system gzip instead of treating different compressed bytes as equivalent. `--allow-unsigned`, `--skip-rat`, `--skip-build`, and `--skip-install` exist for CI or local development of the release process. They are not a substitute for the corresponding checks when voting. The release-candidate workflow creates -an unsigned archive for deterministic CI validation; official artifacts must -always be signed by the release manager. +the unsigned canonical archive for deterministic CI validation. The release +manager downloads and signs that exact archive; the private signing key remains +only on the release manager's machine. ## Publish an approved release @@ -207,8 +223,9 @@ than ASF's general one-hour minimum. - `bump_version.py`: consistently check or update CMake and documentation version metadata. -- `create_source_release.sh`: deterministically create an archive, SHA-512 - checksum, and optional detached signature from an immutable Git ref. +- `create_source_release.sh`: deterministically create an archive with GNU + gzip, a SHA-512 checksum, and an optional detached signature from an immutable + Git ref. - `validate_source_archive.py`: reject unsafe or non-portable tar members and compiled files. - `verify_release_candidate.sh`: perform voter-facing integrity, license, diff --git a/scripts/releasing/create_source_release.sh b/scripts/releasing/create_source_release.sh index 125ab5799..67e4d0932 100755 --- a/scripts/releasing/create_source_release.sh +++ b/scripts/releasing/create_source_release.sh @@ -49,6 +49,9 @@ The script creates: apache-paimon-cpp-VERSION-src.tgz.asc (when --signing-key is provided) Existing artifacts are never overwritten. + +GNU gzip is required so macOS and Linux produce the same compressed bytes. +Set PAIMON_GZIP to an explicit GNU gzip executable when it is not on PATH. EOF } @@ -68,6 +71,36 @@ calculate_sha512() { fi } +find_gnu_gzip() { + local candidate + local resolved + local version_line + local -a candidates + + if [[ -n "${PAIMON_GZIP:-}" ]]; then + candidates=("${PAIMON_GZIP}") + else + candidates=(gzip ggzip) + fi + + for candidate in "${candidates[@]}"; do + if [[ -x "${candidate}" ]]; then + resolved=${candidate} + elif resolved=$(command -v "${candidate}" 2>/dev/null); then + : + else + continue + fi + version_line=$("${resolved}" --version 2>/dev/null | sed -n '1p' || true) + if [[ "${version_line}" =~ ^gzip[[:space:]][0-9] ]]; then + printf '%s\n' "${resolved}" + return 0 + fi + done + + fail "GNU gzip is required for reproducible source archives; on macOS run 'brew install gzip' and set PAIMON_GZIP to the Homebrew gzip executable" +} + while [[ $# -gt 0 ]]; do case "$1" in --version) @@ -128,6 +161,8 @@ DOCS_VERSION=$( [[ "${DOCS_VERSION}" == "${RELEASE_VERSION}" ]] || fail "documentation version ${DOCS_VERSION:-} does not match ${RELEASE_VERSION}" +GZIP_BIN=$(find_gnu_gzip) + ARTIFACT_NAME="apache-paimon-cpp-${RELEASE_VERSION}-src.tgz" ARCHIVE_ROOT="paimon-cpp-${RELEASE_VERSION}" @@ -147,7 +182,10 @@ git -C "${SOURCE_ROOT}" -c tar.umask=0022 archive \ --format=tar \ --prefix="${ARCHIVE_ROOT}/" \ "${GIT_REF}" | - gzip -n >"${TEMP_DIR}/${ARTIFACT_NAME}" + ( + unset GZIP + "${GZIP_BIN}" --no-name --stdout -6 + ) >"${TEMP_DIR}/${ARTIFACT_NAME}" SHA512=$(calculate_sha512 "${TEMP_DIR}/${ARTIFACT_NAME}") printf '%s %s\n' "${SHA512}" "${ARTIFACT_NAME}" \ diff --git a/scripts/releasing/release_rc.sh b/scripts/releasing/release_rc.sh index f943491ed..345486384 100755 --- a/scripts/releasing/release_rc.sh +++ b/scripts/releasing/release_rc.sh @@ -31,6 +31,8 @@ DIST_DEV_BASE_URL="https://dist.apache.org/repos/dist/dev/paimon" PREPARE_ONLY=false DRY_RUN=false WORKFLOW_DISCOVERY_TIMEOUT_SECONDS=600 +WORKFLOW_RUN_ID="" +TEMP_DIR="" usage() { cat <<'EOF' @@ -53,8 +55,9 @@ Options: --dry-run Print the planned release identifiers and exit -h, --help Show this help -The script is resumable when the local signed tag or artifacts already exist, -provided that they match HEAD and pass all verification checks. +For a published RC, GitHub Actions creates and tests the canonical source +archive. This script downloads those exact bytes, signs them locally, and +uploads them to ASF dist/dev. Prepare-only mode creates a local preview. EOF } @@ -117,6 +120,74 @@ wait_for_release_candidate_workflow() { --exit-status \ --interval 30 || fail "Release Candidate workflow run ${run_id} failed" + WORKFLOW_RUN_ID=${run_id} +} + +validate_workflow_artifact_directory() { + local directory=$1 + local -a entries + local entry + local name + + shopt -s dotglob nullglob + entries=("${directory}"/*) + shopt -u dotglob nullglob + [[ ${#entries[@]} -eq 2 ]] || + fail "workflow artifact must contain exactly the archive and checksum" + for entry in "${entries[@]}"; do + [[ -f "${entry}" ]] || + fail "workflow artifact contains a non-file entry: ${entry}" + name=$(basename "${entry}") + case "${name}" in + "${ARTIFACT_NAME}" | "${ARTIFACT_NAME}.sha512") + ;; + *) + fail "workflow artifact contains an unexpected file: ${name}" + ;; + esac + done +} + +download_and_sign_workflow_artifact() { + local workflow_dir="${TEMP_DIR}/source-archive" + local source + local target + local suffix + + [[ -n "${WORKFLOW_RUN_ID}" ]] || fail "release workflow run ID is missing" + mkdir -p "${workflow_dir}" + gh run download "${WORKFLOW_RUN_ID}" \ + --repo apache/paimon-cpp \ + --name source-archive \ + --dir "${workflow_dir}" + validate_workflow_artifact_directory "${workflow_dir}" + + mkdir -p "${OUTPUT_DIR}" + for suffix in "" ".sha512"; do + source="${workflow_dir}/${ARTIFACT_NAME}${suffix}" + target="${OUTPUT_DIR}/${ARTIFACT_NAME}${suffix}" + if [[ -e "${target}" ]]; then + [[ -f "${target}" ]] || fail "artifact path is not a file: ${target}" + cmp "${source}" "${target}" >/dev/null || + fail "existing ${target} differs from workflow run ${WORKFLOW_RUN_ID}" + else + cp -p "${source}" "${target}" + fi + done + + ARTIFACT="${OUTPUT_DIR}/${ARTIFACT_NAME}" + if [[ -e "${ARTIFACT}.asc" ]]; then + [[ -f "${ARTIFACT}.asc" ]] || + fail "artifact signature path is not a file: ${ARTIFACT}.asc" + echo "Reusing existing source artifact signature." + else + echo "Signing workflow source artifact with ${SIGNING_KEY}." + gpg --armor \ + --local-user "${SIGNING_KEY}" \ + --detach-sign \ + --output "${ARTIFACT}.asc" \ + "${ARTIFACT}" + fi } validate_artifact_directory() { @@ -224,6 +295,9 @@ EOF exit 0 fi +TEMP_DIR=$(mktemp -d) +trap 'rm -rf "${TEMP_DIR}"' EXIT + for command in git gpg python3; do require_command "${command}" done @@ -269,33 +343,33 @@ else fi ARTIFACT="${OUTPUT_DIR}/${ARTIFACT_NAME}" -if [[ -e "${ARTIFACT}" || -e "${ARTIFACT}.asc" || -e "${ARTIFACT}.sha512" ]]; then - [[ -f "${ARTIFACT}" && -f "${ARTIFACT}.asc" && -f "${ARTIFACT}.sha512" ]] || - fail "artifact directory contains an incomplete release candidate" - echo "Reusing existing artifacts in ${OUTPUT_DIR}." -else - "${SCRIPT_DIR}/create_source_release.sh" \ - --version "${VERSION}" \ +if [[ "${PREPARE_ONLY}" == true ]]; then + if [[ -e "${ARTIFACT}" || -e "${ARTIFACT}.asc" || -e "${ARTIFACT}.sha512" ]]; then + [[ -f "${ARTIFACT}" && -f "${ARTIFACT}.asc" && -f "${ARTIFACT}.sha512" ]] || + fail "artifact directory contains an incomplete release candidate" + echo "Reusing existing preview artifacts in ${OUTPUT_DIR}." + else + "${SCRIPT_DIR}/create_source_release.sh" \ + --version "${VERSION}" \ + --git-ref "${RC_TAG}" \ + --output-dir "${OUTPUT_DIR}" \ + --signing-key "${SIGNING_KEY}" + fi + + "${SCRIPT_DIR}/verify_release_candidate.sh" \ --git-ref "${RC_TAG}" \ - --output-dir "${OUTPUT_DIR}" \ - --signing-key "${SIGNING_KEY}" -fi - -"${SCRIPT_DIR}/verify_release_candidate.sh" \ - --git-ref "${RC_TAG}" \ - --keys-url "https://downloads.apache.org/paimon/KEYS" \ - "${ARTIFACT}" + --keys-url "https://downloads.apache.org/paimon/KEYS" \ + "${ARTIFACT}" + validate_artifact_directory -validate_artifact_directory - -if [[ "${PREPARE_ONLY}" == true ]]; then cat </dev/null 2>&1; then fi git push "${REMOTE}" "${RC_TAG}" wait_for_release_candidate_workflow +download_and_sign_workflow_artifact + +"${SCRIPT_DIR}/verify_release_candidate.sh" \ + --git-ref "${RC_TAG}" \ + --keys-url "https://downloads.apache.org/paimon/KEYS" \ + "${ARTIFACT}" +validate_artifact_directory + svn import "${OUTPUT_DIR}" "${RC_URL}" \ -m "Add Apache Paimon C++ ${VERSION} RC${RC}" diff --git a/scripts/releasing/tests/test_release_tools.py b/scripts/releasing/tests/test_release_tools.py index d1b4eeaf7..827ddfeac 100644 --- a/scripts/releasing/tests/test_release_tools.py +++ b/scripts/releasing/tests/test_release_tools.py @@ -20,6 +20,8 @@ import io import json import os +import re +import shutil import subprocess import sys import tarfile @@ -33,9 +35,26 @@ ARCHIVE_VALIDATOR = RELEASING_DIR / "validate_source_archive.py" VERSION_TOOL = RELEASING_DIR / "bump_version.py" RELEASE_VERIFIER = RELEASING_DIR / "verify_release_candidate.sh" +SOURCE_RELEASE_CREATOR = RELEASING_DIR / "create_source_release.sh" +SOURCE_ROOT = RELEASING_DIR.parents[1] class ReleaseToolTest(unittest.TestCase): + def head_release_version(self) -> str: + result = subprocess.run( + ["git", "-C", str(SOURCE_ROOT), "show", "HEAD:CMakeLists.txt"], + universal_newlines=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + check=False, + ) + self.assertEqual(result.returncode, 0, msg=result.stderr) + match = re.search( + r"^\s*VERSION\s+(\d+\.\d+\.\d+)\s*$", result.stdout, re.MULTILINE + ) + self.assertIsNotNone(match) + return match.group(1) + def run_tool( self, tool: Path, *args: str, expected_returncode: int = 0 ) -> subprocess.CompletedProcess: @@ -314,6 +333,269 @@ def test_verifier_rejects_unknown_licenses(self) -> None: result.stderr, ) + @unittest.skipUnless( + shutil.which("gpg") and shutil.which("gzip"), "gpg and gzip are required" + ) + def test_verifier_uses_keys_file_for_unsigned_artifact_tag(self) -> None: + with tempfile.TemporaryDirectory() as temp: + directory = Path(temp) + source_root = directory / "source" + releasing_dir = source_root / "scripts/releasing" + releasing_dir.mkdir(parents=True) + for script in ( + "bump_version.py", + "create_source_release.sh", + "validate_source_archive.py", + "verify_release_candidate.sh", + ): + shutil.copy2(RELEASING_DIR / script, releasing_dir / script) + + files = { + "CMakeLists.txt": "project(paimon\n VERSION 1.2.3\n)\n", + "LICENSE": "Apache License\n", + "NOTICE": "Apache Paimon\n", + "docs/source/conf.py": 'version = "1.2.3"\n', + "docs/source/_static/versions.json": ( + '[{"name": "1.2.3", "version": "1.2.3", ' + '"url": "https://paimon.apache.org/docs/cpp/"}]\n' + ), + ".github/.rat-excludes": "", + } + for name, content in files.items(): + path = source_root / name + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(content, encoding="utf-8") + + empty_git_config = directory / "empty-gitconfig" + empty_git_config.touch() + git_env = os.environ.copy() + for name in list(git_env): + if name in ("GIT_CONFIG_COUNT", "GIT_CONFIG_PARAMETERS") or re.fullmatch( + r"GIT_CONFIG_(KEY|VALUE)_\d+", name + ): + del git_env[name] + git_env["GIT_CONFIG_GLOBAL"] = str(empty_git_config) + git_env["GIT_CONFIG_SYSTEM"] = str(empty_git_config) + + subprocess.run( + ["git", "init", "-q", str(source_root)], env=git_env, check=True + ) + subprocess.run( + ["git", "-C", str(source_root), "config", "user.name", "Release Test"], + env=git_env, + check=True, + ) + subprocess.run( + [ + "git", + "-C", + str(source_root), + "config", + "user.email", + "release-test@example.com", + ], + env=git_env, + check=True, + ) + subprocess.run( + ["git", "-C", str(source_root), "add", "."], + env=git_env, + check=True, + ) + subprocess.run( + ["git", "-C", str(source_root), "commit", "-q", "-m", "test"], + env=git_env, + check=True, + ) + + signing_home = directory / "signing-home" + signing_home.mkdir(mode=0o700) + signing_env = git_env.copy() + signing_env["GNUPGHOME"] = str(signing_home) + subprocess.run( + [ + "gpg", + "--batch", + "--passphrase", + "", + "--quick-generate-key", + "Release Test ", + "ed25519", + "sign", + "0", + ], + env=signing_env, + check=True, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + ) + key_listing = subprocess.run( + ["gpg", "--batch", "--with-colons", "--list-secret-keys"], + env=signing_env, + universal_newlines=True, + stdout=subprocess.PIPE, + check=True, + ) + fingerprint = next( + line.split(":")[9] + for line in key_listing.stdout.splitlines() + if line.startswith("fpr:") # codespell:ignore fpr + ) + subprocess.run( + [ + "git", + "-C", + str(source_root), + "tag", + "-s", + "-u", + fingerprint, + "-m", + "test tag", + "v1.2.3-rc1", + ], + env=signing_env, + check=True, + ) + + keys_file = directory / "KEYS" + with keys_file.open("w", encoding="utf-8") as output: + subprocess.run( + ["gpg", "--batch", "--armor", "--export", fingerprint], + env=signing_env, + universal_newlines=True, + stdout=output, + check=True, + ) + + real_gzip = shutil.which("gzip") + self.assertIsNotNone(real_gzip) + fake_gzip = directory / "gzip" + fake_gzip.write_text( + "#!/usr/bin/env bash\n" + "set -euo pipefail\n" + "if [[ ${1:-} == --version ]]; then\n" + " echo 'gzip 1.99'\n" + " exit 0\n" + "fi\n" + f'exec "{real_gzip}" -n -c -6\n', + encoding="utf-8", + ) + fake_gzip.chmod(0o755) + release_env = git_env.copy() + release_env["PAIMON_GZIP"] = str(fake_gzip) + + artifact_dir = directory / "release" + subprocess.run( + [ + "bash", + str(releasing_dir / "create_source_release.sh"), + "--version", + "1.2.3", + "--git-ref", + "v1.2.3-rc1", + "--output-dir", + str(artifact_dir), + ], + env=release_env, + check=True, + stdout=subprocess.DEVNULL, + ) + artifact = artifact_dir / "apache-paimon-cpp-1.2.3-src.tgz" + result = subprocess.run( + [ + "bash", + str(releasing_dir / "verify_release_candidate.sh"), + "--allow-unsigned", + "--keys-file", + str(keys_file), + "--git-ref", + "v1.2.3-rc1", + "--skip-rat", + "--skip-build", + str(artifact), + ], + universal_newlines=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + env=release_env, + check=False, + ) + self.assertEqual( + result.returncode, 0, msg=result.stdout + result.stderr + ) + self.assertIn("Git ref reproducibility: valid", result.stdout) + + def test_source_creator_rejects_non_gnu_gzip(self) -> None: + with tempfile.TemporaryDirectory() as temp: + directory = Path(temp) + fake_gzip = directory / "gzip" + fake_gzip.write_text( + "#!/usr/bin/env bash\n" + "echo 'Apple gzip 999.0'\n", + encoding="utf-8", + ) + fake_gzip.chmod(0o755) + env = os.environ.copy() + env["PAIMON_GZIP"] = str(fake_gzip) + result = subprocess.run( + [ + "bash", + str(SOURCE_RELEASE_CREATOR), + "--version", + self.head_release_version(), + "--git-ref", + "HEAD", + "--output-dir", + str(directory / "release"), + ], + universal_newlines=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + env=env, + check=False, + ) + self.assertEqual(result.returncode, 1, msg=result.stdout + result.stderr) + self.assertIn("GNU gzip is required", result.stderr) + + def test_source_creator_clears_gzip_environment_options(self) -> None: + with tempfile.TemporaryDirectory() as temp: + directory = Path(temp) + fake_gzip = directory / "gzip" + fake_gzip.write_text( + "#!/usr/bin/env bash\n" + "set -euo pipefail\n" + "if [[ ${1:-} == --version ]]; then\n" + " echo 'gzip 1.99'\n" + " exit 0\n" + "fi\n" + "[[ -z ${GZIP+x} ]] || { echo 'GZIP was not cleared' >&2; exit 1; }\n" + "dd of=/dev/null 2>/dev/null\n", + encoding="utf-8", + ) + fake_gzip.chmod(0o755) + env = os.environ.copy() + env["GZIP"] = "-9" + env["PAIMON_GZIP"] = str(fake_gzip) + result = subprocess.run( + [ + "bash", + str(SOURCE_RELEASE_CREATOR), + "--version", + self.head_release_version(), + "--git-ref", + "HEAD", + "--output-dir", + str(directory / "release"), + ], + universal_newlines=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + env=env, + check=False, + ) + self.assertEqual(result.returncode, 0, msg=result.stdout + result.stderr) + def create_version_tree(self, root: Path) -> None: (root / "docs/source/_static").mkdir(parents=True) (root / "CMakeLists.txt").write_text( diff --git a/scripts/releasing/verify_release_candidate.sh b/scripts/releasing/verify_release_candidate.sh index 99e7df0a9..0f3e76d05 100755 --- a/scripts/releasing/verify_release_candidate.sh +++ b/scripts/releasing/verify_release_candidate.sh @@ -28,6 +28,8 @@ DIST_DEV_BASE_URL="https://dist.apache.org/repos/dist/dev/paimon" KEYS_URL="" KEYS_FILE="" GIT_REF="" +VERIFY_GNUPG_HOME="" +VERIFY_KEYS_FILE="" RAT_JAR=${RAT_JAR:-} RAT_VERSION="0.16.1" ALLOW_UNSIGNED=false @@ -52,7 +54,7 @@ Download options: Trust and reproducibility: --keys-url URL Download KEYS and verify in an isolated GPG home --keys-file FILE Import this KEYS file into an isolated GPG home - --git-ref REF Regenerate the archive from REF and compare bytes + --git-ref REF Regenerate with GNU gzip and compare archive bytes Verification options: --rat-jar FILE Apache RAT executable jar (or set RAT_JAR) @@ -99,6 +101,24 @@ download_file() { fi } +prepare_verification_keyring() { + [[ -z "${VERIFY_GNUPG_HOME}" ]] || return 0 + + command -v gpg >/dev/null 2>&1 || fail "gpg is required for signature verification" + VERIFY_GNUPG_HOME="${TEMP_DIR}/gnupg" + mkdir -m 700 "${VERIFY_GNUPG_HOME}" + if [[ -n "${KEYS_URL}" ]]; then + VERIFY_KEYS_FILE="${TEMP_DIR}/KEYS" + download_file "${KEYS_URL}" "${VERIFY_KEYS_FILE}" + else + VERIFY_KEYS_FILE=$(cd "$(dirname "${KEYS_FILE}")" && pwd)/$(basename "${KEYS_FILE}") + fi + [[ -f "${VERIFY_KEYS_FILE}" ]] || + fail "KEYS file does not exist: ${VERIFY_KEYS_FILE}" + gpg --batch --homedir "${VERIFY_GNUPG_HOME}" \ + --import "${VERIFY_KEYS_FILE}" >/dev/null +} + while [[ $# -gt 0 ]]; do case "$1" in --version) @@ -249,19 +269,10 @@ echo "SHA-512 checksum: valid" if [[ -f "${SIGNATURE_FILE}" ]]; then command -v gpg >/dev/null 2>&1 || fail "gpg is required to verify the signature" if [[ -n "${KEYS_URL}" || -n "${KEYS_FILE}" ]]; then - GNUPG_HOME="${TEMP_DIR}/gnupg" - mkdir -m 700 "${GNUPG_HOME}" - if [[ -n "${KEYS_URL}" ]]; then - KEYS_FILE="${TEMP_DIR}/KEYS" - download_file "${KEYS_URL}" "${KEYS_FILE}" - else - KEYS_FILE=$(cd "$(dirname "${KEYS_FILE}")" && pwd)/$(basename "${KEYS_FILE}") - fi - [[ -f "${KEYS_FILE}" ]] || fail "KEYS file does not exist: ${KEYS_FILE}" - gpg --batch --homedir "${GNUPG_HOME}" --import "${KEYS_FILE}" >/dev/null - gpg --batch --homedir "${GNUPG_HOME}" \ + prepare_verification_keyring + gpg --batch --homedir "${VERIFY_GNUPG_HOME}" \ --verify "${SIGNATURE_FILE}" "${ARTIFACT}" - echo "OpenPGP signature: valid against ${KEYS_FILE}" + echo "OpenPGP signature: valid against ${VERIFY_KEYS_FILE}" else gpg --verify "${SIGNATURE_FILE}" "${ARTIFACT}" echo "OpenPGP signature: valid against the default GPG keyring" @@ -313,7 +324,13 @@ if [[ -n "${GIT_REF}" ]]; then fail "Git ref does not resolve to a commit: ${GIT_REF}" if git -C "${SOURCE_ROOT}" rev-parse --verify "${GIT_REF}^{tag}" \ >/dev/null 2>&1; then - git -C "${SOURCE_ROOT}" verify-tag "${GIT_REF}" + if [[ -n "${KEYS_URL}" || -n "${KEYS_FILE}" ]]; then + prepare_verification_keyring + GNUPGHOME=${VERIFY_GNUPG_HOME} \ + git -C "${SOURCE_ROOT}" verify-tag "${GIT_REF}" + else + git -C "${SOURCE_ROOT}" verify-tag "${GIT_REF}" + fi fi REPRO_DIR="${TEMP_DIR}/reproduced" "${SCRIPT_DIR}/create_source_release.sh" \