fix(build): stop hudi-presto-bundle depending on another bundle - #19490
fix(build): stop hudi-presto-bundle depending on another bundle#19490rangareddy wants to merge 1 commit into
Conversation
hudi-agent
left a comment
hudi-agent
left a comment
There was a problem hiding this comment.
Thanks for working on this! This PR removes the redundant-looking (but actually load-bearing) hudi-hadoop-mr-bundle dependency from hudi-presto-bundle and instead declares the Jackson 1.x (asl) artifacts directly, managing their version in the root pom. The reasoning about shade's silent-shrinkage behavior and the provided-vs-compile scope handling is sound, and the empirical entry-count verification is a nice touch. No issues flagged from this automated pass — a Hudi committer or PMC member can take it from here for a final review.
cc @yihua
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #19490 +/- ##
============================================
+ Coverage 76.97% 77.16% +0.19%
- Complexity 33855 33946 +91
============================================
Files 2575 2575
Lines 143378 143397 +19
Branches 17573 17640 +67
============================================
+ Hits 110362 110656 +294
+ Misses 24755 24481 -274
+ Partials 8261 8260 -1
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
voonhous
left a comment
voonhous
left a comment
There was a problem hiding this comment.
Reviewed the resolution graph rather than only the diff: A/B of mvn -o -B dependency:tree -Dincludes=org.codehaus.jackson -fae between base 637996c5ab20 and this branch, plus org/codehaus/jackson/ entry counts on the released bundles back to 0.15.1.
The core insight is right and well demonstrated: the dependency was load-bearing, not redundant. Two blockers inline though. The managed provided scope is not inert, it re-scopes jackson-asl in 7 other modules. And the jar-identical claim holds for the install-then-resolve recipe but not for the full-reactor build that produces the released artifact, where this goes from 0 to 623 Jackson 1.9.13 classes. Then a verification gap on the published POM, a wrong rationale in the comments, and three nits.
| <dependency> | ||
| <groupId>org.codehaus.jackson</groupId> | ||
| <artifactId>jackson-core-asl</artifactId> | ||
| <version>${jackson.asl.version}</version> | ||
| <scope>provided</scope> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.codehaus.jackson</groupId> | ||
| <artifactId>jackson-mapper-asl</artifactId> | ||
| <version>${jackson.asl.version}</version> | ||
| <scope>provided</scope> | ||
| </dependency> |
There was a problem hiding this comment.
Blocker. The managed <scope>provided</scope> is not inert. Maven applies a managed scope to transitive nodes, not only to modules that declare the artifact, so this re-scopes jackson-asl across the whole reactor.
A/B of base 637996c5ab20 against this branch:
mvn -o -B dependency:tree -Dincludes=org.codehaus.jackson -fae
compile/runtime becomes provided in 7 modules that never declare it: hudi-hadoop-mr-bundle, hudi-cli-bundle_2.12, hudi-flink, hudi-examples-flink, hudi-examples-k8s, hudi-adb-sync, hudi-tests-common.
Two of those have real consequences:
hudi-hadoop-mr-bundlesetscreateDependencyReducedPomandpromoteTransitiveDependenciestrue (packaging/hudi-hadoop-mr-bundle/pom.xml:139,142), and shade's goal isrequiresDependencyResolution=runtime, so provided artifacts are invisible to it. Both jackson artifacts silently drop out of that bundle's published POM. That partly undoes what fix(build): publish dependency-reduced POMs for shaded bundles #19433 wrote two lines above atpackaging/hudi-hadoop-mr-bundle/pom.xml:140("Keep dependencies that are not absorbed into the shaded jar, so the reduced POM still declares what consumers need at runtime"), for a bundle this PR never mentions.hudi-hive-sync-bundleandhudi-gcp-bundleconsume that POM.hudi-examples-k8sshades<include>*:*</include>(hudi-examples/hudi-examples-k8s/pom.xml:57). The flip splits the set:jackson-core-aslandjackson-mapper-aslgo provided whilejackson-jaxrsandjackson-xcstay runtime, so its fat jar keeps the JAX-RS provider without theorg.codehaus.jackson.mapclasses it calls into.
Managing the version only fixes this. I ran that variant against base: the reactor-wide diff is then exactly the two hudi-presto-bundle lines and nothing else, zero collateral flips.
| <dependency> | |
| <groupId>org.codehaus.jackson</groupId> | |
| <artifactId>jackson-core-asl</artifactId> | |
| <version>${jackson.asl.version}</version> | |
| <scope>provided</scope> | |
| </dependency> | |
| <dependency> | |
| <groupId>org.codehaus.jackson</groupId> | |
| <artifactId>jackson-mapper-asl</artifactId> | |
| <version>${jackson.asl.version}</version> | |
| <scope>provided</scope> | |
| </dependency> | |
| <dependency> | |
| <groupId>org.codehaus.jackson</groupId> | |
| <artifactId>jackson-core-asl</artifactId> | |
| <version>${jackson.asl.version}</version> | |
| </dependency> | |
| <dependency> | |
| <groupId>org.codehaus.jackson</groupId> | |
| <artifactId>jackson-mapper-asl</artifactId> | |
| <version>${jackson.asl.version}</version> | |
| </dependency> |
There was a problem hiding this comment.
Confirmed and dropped — the root pom is untouched in the new revision, so this collateral is gone entirely.
I reproduced your A/B on 637996c5ab20 vs the old branch tip and got your list exactly. Parsing the per-module trees into sets, 8 modules changed, 7 of which never declare the artifact:
hudi-adb-sync compile -> provided
hudi-cli-bundle_2.12 compile -> provided
hudi-examples-flink compile -> provided
hudi-examples-k8s runtime -> provided
hudi-flink compile -> provided
hudi-tests-common compile -> provided
hudi-hadoop-mr-bundle compile -> provided
hudi-presto-bundle (absent) -> compile
And the hudi-examples-k8s split is exactly as you described — jackson-core-asl and jackson-mapper-asl go provided while jackson-jaxrs:1.9.13 and jackson-xc:1.9.13 stay runtime, so its *:* fat jar would have kept the JAX-RS provider without org.codehaus.jackson.map. I had reasoned "managed scope is inert for modules that do not declare the artifact", which is simply not how Maven applies a managed scope to transitive nodes. Thanks for the correction.
I did not take the manage-version-only suggestion either, because your next comment made the dependencies unnecessary in the first place — see there.
| <dependency> | ||
| <groupId>org.codehaus.jackson</groupId> | ||
| <artifactId>jackson-core-asl</artifactId> | ||
| <scope>compile</scope> | ||
| </dependency> |
There was a problem hiding this comment.
Blocker. "No change to the presto bundle's jar contents" holds only for the install-then-resolve recipe. On the path that builds the released jar it goes 0 -> 623.
A reactor build resolves hudi-hadoop-mr-bundle from the reactor and never sees its reduced POM, so on master this bundle pulls no jackson-asl at all:
mvn -o -B dependency:tree -Dincludes=org.codehaus.jackson -fae
hudi-presto-bundle is absent from the jackson list on base 637996c5ab20 and appears only on this branch, at compile scope. The org.codehaus.jackson:* include at line 81 then matches, and the 623 classes land in the jar.
That is the path that ships. scripts/release/deploy_staging_jars.sh:71 deploys this bundle with -Dscala-2.12 -Dspark3.5 -Dflink1.20 -Ddocker.compose.skip=true, full reactor, no -pl. The released jars agree:
for v in 0.15.1 1.0.2 1.1.0-SNAPSHOT 1.2.0-SNAPSHOT; do
unzip -l hudi-presto-bundle-$v.jar | grep -c org/codehaus/jackson/
done
# 651, 0, 0, 0
0.15.1 shipped Jackson 1.x; every release since ships none, and a 1.3.0-SNAPSHOT reactor build from before #19433 also has 0. So the 7313/623 master baseline is a one-day-old side effect of #19433 (merged 2026-08-03), not this bundle's shipped contract, and this PR would make it permanent on both build paths.
Please re-measure with a full-reactor mvn install -DskipTests and put that number in the PR body beside the repository-resolution number. If it reads 0 -> 623, take the smaller fix described in my next comment rather than declaring these two artifacts. Worth noting too that #19491 asserts nothing about org/codehaus/jackson/** and only exercises the repository-resolution path, so it would not catch this either.
There was a problem hiding this comment.
You are right, and this was the important one. I re-measured with mvn install -DskipTests -Dscala-2.12 -Dspark3.5 -Dflink1.20 on both paths — -pl packaging/hudi-presto-bundle alone for repository resolution, and -pl packaging/hudi-hadoop-mr-bundle,packaging/hudi-presto-bundle so the mr-bundle comes from the reactor:
variant repository path reactor path (ships)
master 637996c5ab20 7273 cls / 651 jackson 6650 cls / 0
old revision of this PR 7273 / 651 7273 / 651 <-- 0 -> 651
new revision 6650 / 0 6650 / 0
So it read 0 -> 623 classes (651 zip entries) on the path that ships, exactly as you predicted, and my "7313 before and after" was a repository-path-only measurement. Both numbers are now in the PR body with the recipe, and I added an explicit retraction paragraph at the top so anyone arriving mid-thread is not misled by the original claim.
Also confirms your point about #19491: it asserts required classes and a org/apache/hudi/hadoop/** floor, and nothing about org/codehaus/jackson/**, so it would not have caught this. The new revision leaves that jar at 109 such entries, comfortably above the floor of 100.
| <dependency> | ||
| <groupId>org.codehaus.jackson</groupId> | ||
| <artifactId>jackson-mapper-asl</artifactId> | ||
| <scope>compile</scope> | ||
| </dependency> |
There was a problem hiding this comment.
Major. This re-litigates HUDI-4997 without citing it, and the history points at a smaller fix.
Commit 779a96506fb7 (#6893, 2022-10-20, "Use jackson-v2 import instead of jackson-v1") deleted exactly this pairing: the codehaus-jackson.version property and the jackson-core-asl / jackson-mapper-asl / jackson-jaxrs / jackson-xc block from the root pom, the org.codehaus.jackson. relocation from this file, and the two jackson-asl lines from dependencies/hudi-presto-bundle.txt. Stated reason in that PR: jackson-v1 has security risks.
gh api repos/apache/hudi/commits/779a96506fb7 --jq '.files[]|select(.filename=="pom.xml")|.patch'
git show 779a96506fb7 -- packaging/hudi-presto-bundle/pom.xml
The org.codehaus.jackson:* include at line 81 was added together with that relocation in 4e050cc2ba26 (#2816, 2021-04-17), back when Hudi's own code imported jackson-v1. #6893 removed the relocation and left the include orphaned. Nothing in the tree references the package today:
grep -rn "org\.codehaus\.jackson" --include="*.java" --include="*.scala" . | grep -v /target/ # no hits
So these are dead classes, and they ship unrelocated at their original coordinates onto Presto's classpath. jackson-mapper-asl 1.9.13 is also the top of CVE-2019-10172's affected range, with no fixed release in the ASL 1.x line.
Concrete alternative: rather than adding these two dependencies, delete the dead <include>org.codehaus.jackson:*</include> at line 81 in this PR. Both build paths then agree at 0 org/codehaus/jackson/ entries, matching 1.0.2 through 1.2.0 and pre-#19433 master, the root pom change becomes unnecessary, and it finishes the cleanup #6893 started. If you would rather keep shipping Jackson 1.x, cite #6893 in the PR body with the reason it is acceptable to re-add it, and add a relocation consistent with the avro and commons-lang3 ones already in this file.
There was a problem hiding this comment.
Taken — this is now what the PR does. Every claim here checks out:
779a96506fb7([HUDI-4997] use jackson-v2 replace jackson-v1 import. #6893) removed thecodehaus-jackson.versionproperty and all four asl artifacts from the root pom, theorg.codehaus.jackson.relocation from this file, and the two asl lines fromdependencies/hudi-presto-bundle.txt, in one commit.grep -rn "org\.codehaus\.jackson" --include="*.java" --include="*.scala" . | grep -v /target/returns nothing, so the include at line 81 has matched only transitively-reached classes since then, and they ship unrelocated because [HUDI-4997] use jackson-v2 replace jackson-v1 import. #6893 took the relocation with it.
So the PR is now two deletions in this file and nothing else: the hudi-hadoop-mr-bundle dependency, and the dead include. Both paths land at 0 org/codehaus/jackson/ entries and 6650 classes, matching 1.0.2 through 1.2.0 and master reactor builds. The root pom change is gone, which also disposes of your first comment and the two nits below it.
I chose deletion over re-adding with a relocation for the reason you give: jackson-mapper-asl 1.9.13 sits at the top of CVE-2019-10172 with no fixed ASL 1.x release, and nothing in the tree calls into it. #6893 is cited in both the commit message and the PR body.
| <dependency> | ||
| <groupId>org.apache.hudi</groupId> | ||
| <artifactId>hudi-hadoop-mr-bundle</artifactId> | ||
| <version>${project.version}</version> | ||
| </dependency> |
There was a problem hiding this comment.
Major, verification gap. The jar listing is only half the contract. #19433 was about the published dependency-reduced POM, and removing this dependency changes it: with promoteTransitiveDependencies=true, everything that reached this bundle only through hudi-hadoop-mr-bundle leaves the presto bundle's published POM. Diffing the two installed jars' META-INF/DEPENDENCIES, that includes it.unimi.dsi:fastutil:7.0.13 and org.apache.parquet:parquet-format:2.4.0, and it returns the parquet transitives from 1.10.1 to ${presto.parquet.version}.
That is probably a good change, but the PR neither claims nor measures it. Please build master and this branch and paste the diff of packaging/hudi-presto-bundle/target/dependency-reduced-pom.xml into the verification section, so the metadata half is covered the way the jar half is.
The same matrix should cover the two bundles that still depend on hudi-hadoop-mr-bundle and are affected by the root pom change in this PR: packaging/hudi-hive-sync-bundle/pom.xml:167 and packaging/hudi-gcp-bundle/pom.xml:157. #19469's question 3 is currently answered for one of the three bundles. Either extend the same pattern to those two here, or state in the PR body that they are deliberately left for a follow-up and link it.
There was a problem hiding this comment.
Measured and in the PR body. Parsing both reduced POMs into groupId:artifactId:version:scope sets rather than diffing the XML (the raw diff is noisy with reordering), master vs this branch:
Reactor path — 81 -> 80, exactly one entry, which is the whole point of the PR:
- org.apache.hudi:hudi-hadoop-mr-bundle:1.3.0-SNAPSHOT:compile
Repository path — 81 -> 80, and your prediction is precisely right:
- it.unimi.dsi:fastutil:7.0.13:compile
- org.apache.parquet:parquet-format:2.4.0:compile
- commons-lang:commons-lang:2.6, org.apache.orc:orc-shims:1.6.0
- org.apache.parquet:{column,common,encoding,hadoop,jackson}:1.10.1
+ org.apache.parquet:{column,common,encoding,hadoop}:1.15.2, parquet-format-structures:1.15.2, parquet-jackson:1.15.2:runtime
+ org.apache.orc:orc-shims:1.9.1, com.github.luben:zstd-jni:1.5.6-6, javax.annotation:javax.annotation-api:1.3.2
fastutil and parquet-format:2.4.0 leave, and the parquet transitives return from 1.10.1 to ${presto.parquet.version} — the version this bundle actually shades.
On hudi-hive-sync-bundle and hudi-gcp-bundle: since the root pom is no longer touched, nothing in this PR reaches them. I have stated in the PR body that they are deliberately left for a follow-up rather than swept in, because each needs its own before/after jar plus reduced-POM measurement on both paths — which is precisely the work this review round showed cannot be skipped. I will open the follow-up referencing #19469 question 3 and link it here.
There was a problem hiding this comment.
Follow-up filed as #19511, covering both bundles with the two-path jar + reduced-POM recipe from this thread as the required evidence. One thing that turned up while writing it, worth recording here: unlike the presto bundle, both of those reach hudi-hadoop-mr and hudi-hadoop-common through hudi-hive-sync rather than through the fat jar, so removing the dependency there should not empty their artifactSet — but that still needs measuring rather than assuming, which is the whole lesson of this round.
| <!-- Jackson 1.x (asl). Provided by the hadoop/hive runtime, so provided scope here; declared so a | ||
| bundle that shades org.codehaus.jackson can depend on the library directly rather than reaching it | ||
| through another bundle's published POM. --> |
There was a problem hiding this comment.
Minor, the stated rationale is not the actual path. "Provided by the hadoop/hive runtime" is what justifies the provided scope, but hadoop and hive are managed provided in this pom and do not propagate. The compile-scope path that actually exists in the reactor is parquet:
org.apache.parquet:parquet-avro:1.10.1:compile <- hudi-hadoop-mr-bundle, at ${hive.parquet.version} (pom.xml:131)
\- org.apache.parquet:parquet-hadoop:1.10.1:compile
+- org.codehaus.jackson:jackson-mapper-asl:1.9.13:compile
\- org.codehaus.jackson:jackson-core-asl:1.9.13:compile
parquet-hadoop 1.10.1 declares both artifacts; 1.15.2, which the presto bundle uses, declares neither. The other path is hadoop-common:2.10.2, and that one is profile-dependent: hadoop-common 3.3.6 and 3.4.0 declare no org.codehaus.jackson at all, so under the hadoop3 profiles nothing in the runtime provides it and provided is simply wrong.
If the root entries survive review, restate this as "reached at compile scope via parquet-hadoop 1.10.1 and hadoop-common 2.10.2 (hadoop2 profile only)". The same correction applies to the property comment at line 132. The durable fix for this whole class of problem is moving hive.parquet.version off 1.10.1, which is what drags Jackson 1.x in at all.
There was a problem hiding this comment.
Correct, and moot now — the root pom entries are gone, so there is no rationale left to restate. Recording the finding anyway since it outlives this PR: the compile-scope path is hudi-hadoop-mr-bundle -> parquet-avro:${hive.parquet.version}=1.10.1 -> parquet-hadoop:1.10.1 -> jackson-{core,mapper}-asl:1.9.13, and parquet-hadoop:1.15.2 declares neither, which is why the presto bundle at ${presto.parquet.version} has no path of its own. My "provided by the hadoop/hive runtime" was wrong on both counts — hadoop and hive are managed provided here and do not propagate, and hadoop-common 3.3.6/3.4.0 declare no org.codehaus.jackson at all, so provided would have been false under the hadoop3 profiles.
Agreed that moving hive.parquet.version off 1.10.1 is the durable fix for the class of problem. That is well outside this PR; worth its own issue.
| <version>${project.version}</version> | ||
| </dependency> | ||
|
|
||
| <!-- Jackson 1.x. Shaded via the org.codehaus.jackson:* include below and previously reached only |
There was a problem hiding this comment.
Nit, feel free to ignore: the include is at line 81, above this block, not below. The sibling comment at line 194 gets the direction right ("artifactSet above").
| <!-- Jackson 1.x. Shaded via the org.codehaus.jackson:* include below and previously reached only | |
| <!-- Jackson 1.x. Shaded via the org.codehaus.jackson:* include above and previously reached only |
There was a problem hiding this comment.
Right, and the block that comment belonged to is deleted in the new revision, so it is gone rather than corrected. Good catch either way — I had the direction backwards while the sibling comment at line 194 has it right.
| <!-- Jackson 1.x, reached through the hadoop/hive stack; shaded by hudi-presto-bundle --> | ||
| <jackson.asl.version>1.9.13</jackson.asl.version> |
There was a problem hiding this comment.
Nit, feel free to ignore: this splits the hive property block, landing between hive.parquet.version (line 131) and hive.avro.version (line 134). Moving it below hive.avro.version, or next to the fasterxml.jackson.* properties at lines 103-106, keeps the grouping intact.
There was a problem hiding this comment.
Moot — the root pom is untouched in the new revision, so the property is not added and the hive block stays intact.
|
|
||
| <!-- Jackson 1.x. Shaded via the org.codehaus.jackson:* include below and previously reached only | ||
| through hudi-hadoop-mr-bundle's published POM; declared directly so this bundle's shade inputs come | ||
| from libraries rather than from another bundle. compile scope because the classes are shaded in. --> |
There was a problem hiding this comment.
Minor: dependencies/hudi-presto-bundle.txt is the tracked dependency manifest for this bundle, and #6893 edited it in the same commit that removed these two artifacts (git show 779a96506fb7 -- dependencies/hudi-presto-bundle.txt). Making them direct compile dependencies leaves that file wrong again.
What keeps this minor: it is already stale (grep -c hbase dependencies/hudi-presto-bundle.txt returns 10 although HBase was removed in #12964) and scripts/dependency.sh is referenced by no workflow. So either regenerate it with scripts/dependency.sh, or add a line to the PR body noting that dependencies/ is unmaintained. Just do not leave it silently more wrong than it already is.
There was a problem hiding this comment.
Checked, and no edit is needed after all: dependencies/hudi-presto-bundle.txt lists neither jackson-asl (removed by #6893 in the same commit you cite) nor hudi-hadoop-mr-bundle, so this revision — which only deletes things — leaves it no more wrong than it was. Since the earlier revision would have added two direct compile dependencies, your point was correct against that version.
I noted in the PR body that the file is independently stale (10 HBase entries although HBase went in #12964) and that scripts/dependency.sh is referenced by no workflow, so a reader is not left thinking it is authoritative. Regenerating it is not something I want to fold into a two-line deletion.
hudi-hadoop-mr-bundle is a shaded fat jar, and hudi-presto-bundle depended on it while shading classes that jar already contains. apache#19433 declared hudi-hadoop-mr and hudi-hadoop-common directly, which left the bundle dependency redundant. Removing it also drops the org.codehaus.jackson:* entry from the artifactSet, because that include has been dead since apache#6893 (779a965, "Use jackson-v2 import instead of jackson-v1", 2022) deleted the org.codehaus.jackson. relocation next to it and the root pom's jackson-asl dependencies. No Java or Scala source in the tree references the package today, so the include only ever matched classes reached transitively. Both are removed together because the include is what makes the bundle dependency visible in the jar, and only on one of the two ways this bundle is built: variant repository-resolution full-reactor (what ships) master 7273 classes / 651 jackson 6650 classes / 0 jackson this change 6650 classes / 0 jackson 6650 classes / 0 jackson A reactor build resolves hudi-hadoop-mr-bundle from the reactor and never reads its published POM, so Jackson 1.x never reaches this bundle and the include matches nothing. Resolving from the repository reads the reduced POM, which lists jackson-core-asl and jackson-mapper-asl since apache#19433 turned on promoteTransitiveDependencies, and 623 classes land in the jar unrelocated under org/codehaus/jackson/. Releases confirm which of the two is the contract: scripts/release/deploy_staging_jars.sh builds the full reactor, and 1.0.2 through 1.2.0 all ship 0 such entries. After this change both paths agree, at the same 6650 classes the released jars carry, so the repository path no longer publishes a jar that differs from the released one, and jackson-mapper-asl 1.9.13 - the top of CVE-2019-10172's range, with no fix in the ASL 1.x line - stays off Presto's classpath. The published dependency-reduced POM loses exactly one entry on the reactor path, hudi-hadoop-mr-bundle itself. Closes apache#19469
569c9c8 to
6c68942
Compare
hudi-agent
left a comment
hudi-agent
left a comment
There was a problem hiding this comment.
Thanks for working on this! This PR removes the redundant hudi-hadoop-mr-bundle dependency from hudi-presto-bundle and drops the now-orphaned org.codehaus.jackson:* shade include. I verified that hudi-hadoop-mr and hudi-hadoop-common remain declared as direct compile-scope dependencies on the PR head, so the artifactSet includes still resolve and no non-Jackson classes are lost — consistent with the jar-content parity the author and reviewers measured. The build-path, root-pom retraction, and HUDI-4997 history concerns were already covered thoroughly in the prior round. No new issues flagged from this automated pass — a Hudi committer or PMC member can take it from here for a final review.
cc @yihua
github-actions
Bot
added
size:XS
5 participants
Describe the issue this Pull Request addresses
Closes #19469, a follow-up from #19433.
hudi-hadoop-mr-bundleis a shaded fat jar, andhudi-presto-bundledepended on it while shading classesthat jar already contains. #19433 declared
hudi-hadoop-mrandhudi-hadoop-commondirectly, which left thebundle dependency redundant. The task was to check whether it is, and drop it if so.
Removing it drops 623 classes from the jar, all
org/codehaus/jackson/**, but only on one of the two waysthis bundle gets built — and on the way that actually ships, master already has none of them. The revision
history explains why: the
artifactSetincludeorg.codehaus.jackson:*has been dead since #6893(
779a96506fb7, "Use jackson-v2 import instead of jackson-v1", 2022-10-20), which deleted theorg.codehaus.jackson.relocation sitting next to it, the root pom's four jackson-asl dependencies, and thetwo jackson-asl lines from
dependencies/hudi-presto-bundle.txt— stated reason, jackson-v1 security risks.The include was added alongside that relocation in
4e050cc2ba26(#2816, 2021) when Hudi's own code stillimported jackson-v1, and #6893 left it orphaned:
So both go: the bundle dependency and the include it kept alive.
Summary and Changelog
Two deletions in
packaging/hudi-presto-bundle/pom.xml, nothing else:hudi-hadoop-mr-bundledependency — the point of the issue;<include>org.codehaus.jackson:*</include>from theartifactSet, finishing the cleanup[HUDI-4997] use jackson-v2 replace jackson-v1 import. #6893 started.
Verification
Two resolution paths, because they disagree.
mvn install -DskipTests -Dscala-2.12 -Dspark3.5 -Dflink1.20, with-pl packaging/hudi-presto-bundlealone, sohudi-hadoop-mr-bundleresolves from~/.m2and its published reduced POM is read;-pl packaging/hudi-hadoop-mr-bundle,packaging/hudi-presto-bundle, so it resolves fromthe reactor and the reduced POM is never consulted. This is what ships:
scripts/release/deploy_staging_jars.sh:71deploys this bundle from a full reactor with no-pl.637996c5ab20)Both paths now agree, at exactly the class count master's reactor build produces — so the repository path
stops publishing a jar that differs from the released one. Released artifacts confirm which number is the
contract:
0.15.1shipped 651 such entries, and1.0.2,1.1.0and1.2.0all ship 0.Required contents still present in the rebuilt jar:
Published dependency-reduced POM, the metadata half of the contract that the earlier revision did not
measure. Parsed into
groupId:artifactId:version:scopesets and diffed, master vs this branch:Reactor path — exactly one entry leaves, which is the entire point of the PR:
Repository path — 81 → 80 entries, the larger delta @voonhous predicted:
Those were promoted into this bundle's POM only because it consumed another bundle's reduced POM; dropping
the dependency returns the parquet transitives to the version this bundle actually shades.
Impact
The released jar is unchanged — 6650 classes on the reactor path, before and after. What changes is that the
repository-resolution path stops producing a different jar from the released one, so #19433's
promoteTransitiveDependenciesno longer leaks Jackson 1.x into this bundle.jackson-mapper-asl1.9.13 isthe top of CVE-2019-10172's affected range with no fix in the ASL 1.x line, so keeping it off Presto's
classpath — where it would land unrelocated, since #6893 removed the relocation — is the point.
The published POM loses
hudi-hadoop-mr-bundle: correct, consumers were being told to resolve a fat jar thisbundle no longer needs.
dependencies/hudi-presto-bundle.txtneeds no edit — it has listed neither jackson-asl (since #6893) norhudi-hadoop-mr-bundle. Independently, that file is stale (it still lists 10 HBase entries although HBasewent away in #12964) and
scripts/dependency.shis referenced by no workflow; that is not this PR's to fix.Deliberately left for a follow-up:
packaging/hudi-hive-sync-bundle/pom.xml:167andpackaging/hudi-gcp-bundle/pom.xml:157still depend onhudi-hadoop-mr-bundle. This revision does not touchthe root pom, so neither is affected by anything here, and each needs its own before/after jar and reduced-POM
measurement rather than being swept in.
Risk Level
low — two deletions in one packaging POM, with the shipped jar's class listing measured identical to master on
the path that produces releases.
Documentation Update
none
Contributor's checklist