Skip to content

[INFRA] Bump GHA workflows to latest action versions (branch-1.10 backport)#7566

Closed
pan3793 wants to merge 3 commits into
apache:branch-1.10from
pan3793:gha-backport-1.10
Closed

[INFRA] Bump GHA workflows to latest action versions (branch-1.10 backport)#7566
pan3793 wants to merge 3 commits into
apache:branch-1.10from
pan3793:gha-backport-1.10

Conversation

@pan3793

@pan3793 pan3793 commented Jul 14, 2026

Copy link
Copy Markdown
Member

Logical backport of the GHA modernization to branch-1.10, delivered as three commits mirroring master's split (#7561#7564#7562).

Commits

  1. 2a278f2db[KYUUBI #7561] Cache CI engine archives per URL with a reusable action (cherry-pick of 445a906c2). Replaces the single-blob cache-engine-archives action with per-URL download-archive caching, adds resolve-engine-archive-urls.sh / expose-engine-homes.sh, rewires every master.yml job to resolve → download → expose only the engines it needs, adds the spark-home-from-archive profile to the Spark-3.3/3.4/3.5 extension poms, and makes the Flink test helpers honor FLINK_HOME.

  2. 62d4b2b23[INFRA] Bump GHA workflows to latest action versions (logical backport of 99bed3885 / [INFRA] Bump GHA workflows to latest action versions #7564). Master's [INFRA] Bump GHA workflows to latest action versions #7564 delta is anchored on versions branch-1.10 never carried, so it is reimplemented by bumping branch-1.10's baseline directly — the conflict-free equivalent. The resulting uses: set now matches master exactly (latest major tags for trusted actions; latest allowlist hashes + # vX.Y.Z comments for the Docker actions).

  3. 69b77b1cd[KYUUBI #7562] [INFRA] Remove gluten integration test module (cherry-pick of 00ba4e65d). This is needed because [INFRA] Cache CI engine archives per URL with a reusable action #7561 deletes cache-engine-archives but, unlike master, branch-1.10 still had gluten.yml referencing it (on master the gluten workflow had been removed earlier by [INFRA] Remove gluten integration test module #7562, so [INFRA] Cache CI engine archives per URL with a reusable action #7561 never converted it). Removing the gluten IT module + workflow here eliminates that dangling reference cleanly. The 6 modify/delete conflicts (gluten.yml, gluten.md, GlutenSuiteMixin.scala, and the diverged gluten-IT files) are resolved to deletion; integration-tests/pom.xml and docs/.../index.rst auto-merged.

How was this patch tested?

  • After resolving, the tree is gluten-free (git grep gluten → none) and has no cache-engine-archives references; gluten.yml is gone.
  • The uses: set on branch-1.10 matches master exactly; no old action-version tokens remain.
  • ./dev/reformat was run; it only failed on the Spotless black step (a local pyenv-shim issue, exit 127) and modified no files — YAML is not Spotless-managed, and the [INFRA] Cache CI engine archives per URL with a reusable action #7561 source changes are authored against master and Spotless-clean.
  • CI on this PR exercises the new per-engine download flow across the master.yml matrix.

Was this patch authored or co-authored using generative AI tooling?

Assisted-by: codemaker with glm-5.2

pan3793 added 2 commits July 14, 2026 14:55
…reusable action

The old `cache-engine-archives` cache is one blob under a constant key, so bumping any
engine invalidates all of them. This provisions engines explicitly instead:

- New reusable action `.github/actions/download-archive` downloads a tarball and caches it
  with `actions/cache` **keyed by the URL** (atomic download), so each engine caches
  independently and stable versions restore instantly.
- `.github/scripts/resolve-engine-archive-urls.sh` derives each URL from the Maven
  properties (honoring the build's profiles/`-D` overrides); `expose-engine-homes.sh`
  publishes `SPARK_HOME`/`HIVE_HOME`/`FLINK_HOME` from `/opt`.
- Every `master.yml` job now resolves → downloads → exposes only the engines it needs, and
  the build runs with `-P{spark,flink,hive}-provided`. The now-orphaned
  `cache-engine-archives` action is removed.
- Flink test helpers now honor `FLINK_HOME` first (fallback to the download dir for local
  runs), matching Spark/Hive.
- Spark extension modules gain a `spark-home-from-archive` profile, auto-activated only when
  `SPARK_HOME` is unset, so the Standalone-cluster tests use the downloaded archive locally
  but a provided `SPARK_HOME` is no longer overridden.
- The `hive-it` avatica/CALCITE-1224 hack is now a single `zip -d`.

Run by this PR's CI.

Locally checked: URL resolution across all matrix cases (Scala 2.13, Spark/Flink/Hive
version overrides, CDH6 empty-query), `*_HOME` resolution, the `closer.lua` redirect for
`curl -fSL`, the `zip -d` patch on a mock jar, and the extension `SPARK_HOME` profile via
the effective POM (inherits the env when set, falls back to the archive otherwise).

Assisted-by: Claude Opus 4.8

Closes apache#7561 from pan3793/gha-dl.

Closes apache#7561

1f9ad04 [Cheng Pan] [INFRA] Restore SPARK_SCALA_VERSION for Spark extension local-cluster tests
20a94b2 [Cheng Pan] [INFRA] Bump actions/cache to v6 in download-archive action
a5c4208 [Cheng Pan] fix spark extension standalone tests
0010f4e [Cheng Pan] [INFRA] Remove orphaned cache-engine-archives action
da90a49 [Cheng Pan] nit
13948d3 [Cheng Pan] [INFRA] Cache CI engine archives per URL with a reusable action

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
Backport of apache#7564 to branch-1.10. Combined with the parent backport of
apache#7561 (2a278f2), this brings the GHA modernization (apache#7561 + apache#7564) to
the 1.10 maintenance branch.

Bring all GitHub Actions references on branch-1.10 to the latest,
ASF-allowlist-compliant versions (matching master). The version
normalization is reimplemented rather than cherry-picked, because
master's apache#7564 delta (v7.0.0 -> v7, etc.) is anchored on versions
branch-1.10 never carried; bumping branch-1.10's baseline directly is
the conflict-free equivalent.

Trusted actions (-> latest major tag):
  actions/checkout                 v4 -> v7
  actions/setup-java               v4 -> v5
  actions/setup-python             v5 -> v6
  actions/setup-node               v4 -> v6
  actions/stale                    v9 -> v10
  actions/upload-artifact          v4 -> v7
  actions/dependency-review-action v4  -> v5
  actions/first-interaction        v1 -> v3
  actions/labeler                  v5 -> v6  (.github/labeler.yml already v6-schema)
  codecov/codecov-action           v3 -> v7
  actions/cache                    v4 -> v6  (in .github/actions/setup-maven; @v6 elsewhere unchanged)

Docker actions (-> latest allowlist hash + tag comment):
  docker/build-push-action   -> 53b7df96c91f9c12dcc8a07bcb9ccacbed38856a  # v7.3.0
  docker/setup-buildx-action -> bb05f3f5519dd87d3ba754cc423b652a5edd6d2c  # v4.2.0
  docker/setup-qemu-action   -> 96fe6ef7f33517b61c61be40b68a1882f3264fb8  # v4.2.0
  docker/login-action        -> af1e73f918a031802d376d3c8bbc3fe56130a9b0  # v4.4.0

github/super-linter/slim@v7 was already latest; unchanged.
@github-actions github-actions Bot added kind:infra license, community building, project builds, asf infra related, etc. module:spark module:flink kind:build module:integration-tests module:extensions labels Jul 14, 2026
This removes the kyuubi-gluten-it module, the gluten-it Maven profile, the CI workflow, and the GlutenSuiteMixin test utility.

Gluten IT is not maintained well, let's just remove it.

Pass GHA.

Assisted-by: GLM 5.2

Closes apache#7562 from pan3793/remove-gluten-it.

Closes apache#7562

45918d4 [Cheng Pan] [INFRA] Remove gluten integration test module

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
@github-actions github-actions Bot added kind:documentation Documentation is a feature! module:common labels Jul 14, 2026
@pan3793 pan3793 closed this Jul 14, 2026
@pan3793
pan3793 deleted the gha-backport-1.10 branch July 14, 2026 07:24
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 0.00%. Comparing base (47a0553) to head (69b77b1).
⚠️ Report is 3 commits behind head on branch-1.10.

Additional details and impacted files
@@             Coverage Diff             @@
##           branch-1.10   #7566   +/-   ##
===========================================
  Coverage         0.00%   0.00%           
===========================================
  Files              692     692           
  Lines            43028   43028           
  Branches          5849    5849           
===========================================
  Misses           43028   43028           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind:build kind:documentation Documentation is a feature! kind:infra license, community building, project builds, asf infra related, etc. module:common module:extensions module:flink module:integration-tests module:spark

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants