Skip to content

[INFRA] Use hadoop 2.10.2 for uniffle test#12111

Open
wecharyu wants to merge 3 commits into
apache:mainfrom
wecharyu:uniffle_test
Open

[INFRA] Use hadoop 2.10.2 for uniffle test#12111
wecharyu wants to merge 3 commits into
apache:mainfrom
wecharyu:uniffle_test

Conversation

@wecharyu
Copy link
Copy Markdown
Contributor

@wecharyu wecharyu commented May 19, 2026

What changes are proposed in this pull request?

  • Upgrade the hadoop home version from 2.8.5 to 2.10.2 for tpc-test-centos8-uniffle.

The 2.8.5 archive is in archive.apache.org, the download is much slow and is easy to cause the Build and Push Docker Image failed, details in https://github.com/apache/gluten/actions/workflows/docker_image.yml.

111.8 + wget -nv 'https://www.apache.org/dyn/closer.lua/hadoop/common/hadoop-2.8.5/hadoop-2.8.5.tar.gz?action=download' -O /opt/hadoop-2.8.5.tar.gz
247.6 failed: Connection timed out.
247.6 failed: Network is unreachable.

How was this patch tested?

Pass CI.

Was this patch authored or co-authored using generative AI tooling?

No.

@github-actions github-actions Bot added BUILD INFRA CORE works for Gluten Core labels May 19, 2026
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@wecharyu
Copy link
Copy Markdown
Contributor Author

@philo-he @zhouyuan could you help review this PR?

@philo-he
Copy link
Copy Markdown
Member

@wecharyu, Thanks for the PR. There's another PR that upgrades Maven to 3.9.15. Not sure which version we should target — might be worth coordinating to avoid conflicts.

@wecharyu
Copy link
Copy Markdown
Contributor Author

@philo-he The maven version in dlcdn.a.o is 3.9.16, we need move to this version for faster download.

@philo-he
Copy link
Copy Markdown
Member

@philo-he The maven version in dlcdn.a.o is 3.9.16, we need move to this version for faster download.

@wecharyu It sounds good to me. Could you review that PR and leave a comment there for alignment? Perhaps we should merge the dedicated Maven upgrade PR first. Thank you.

@zhouyuan
Copy link
Copy Markdown
Member

@wecharyu uniffle is using 2.8.5 in its main branch:
https://github.com/apache/uniffle/blob/master/pom.xml#L71

@github-actions github-actions Bot removed the CORE works for Gluten Core label May 27, 2026
@wecharyu
Copy link
Copy Markdown
Contributor Author

@zhouyuan Thanks for the information. Hadoop usually has compatibility guarantees within the same major version, details in doc, it should be safe to test uniffle in hadoop 2.10.2 env.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants