Skip to content

[ZEPPELIN-6311] Support JDK 17 - #5311

Draft
pan3793 wants to merge 4 commits into
apache:masterfrom
pan3793:jdk17
Draft

[ZEPPELIN-6311] Support JDK 17#5311
pan3793 wants to merge 4 commits into
apache:masterfrom
pan3793:jdk17

Conversation

@pan3793

@pan3793 pan3793 commented Jul 16, 2026

Copy link
Copy Markdown
Member

What is this PR for?

Enable Zeppelin to build, test, and run on JDK 17. This involves three independent areas of work:

  • Cassandra test infrastructure
  • Flink Scala version alignment
  • JPMS module-access flags

What type of PR is it?

Improvement

What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-6311

How should this be tested?

CI passes on both JDK 11 and JDK 17 across all modules. Verified locally:

  • cassandra module: 39 tests pass on JDK 17 with Docker
  • flink/flink-scala-2.12 module: 7 FlinkInterpreterTest tests pass on both JDK 11 and JDK 17
  • bin/common.sh syntax-checked; JPMS_JAVA_OPTS verified in JAVA_OPTS and JAVA_INTP_OPTS

Questions

  • Does the licenses file need update? No
  • Is there breaking changes for older versions? No
  • Does this needs documentation? No

Details

1. Cassandra: Testcontainers migration

Cassandra 3.11.5 (via cassandra-unit) crashes on JDK 17. During PREPARE statement storage, Cassandra 3.x serializes lambda expressions. JDK 17 represents lambdas as hidden classes, and sun.misc.Unsafe.objectFieldOffset() on hidden classes throws UnsupportedOperationException -- a hard JVM restriction that no flag can override.

Changes:

  • Replace cassandra-unit dependency with org.testcontainers:cassandra (cassandra:4.1.3 image)
  • Rewrite CassandraInterpreterTest to use CassandraContainer instead of EmbeddedCassandraServerHelper
  • Update test expectation HTML files (DescribeKeyspace_live_data.html, DescribeTable_live_data_complex_table.html, NoResultWithExecutionInfo.html) to match Cassandra 4.x table option output (e.g. additional_write_policy, read_repair = 'BLOCKING', speculative_retry = '99p', chunk_length_in_kb = 16)
  • Normalize localhost/<unresolved>:port in test assertions (JDK 17+ InetSocketAddress.toString() renders unresolved addresses differently)

2. Flink: Scala 2.12.7 -> 2.12.20

Flink 1.19/1.20 binary tgz bundles flink-scala_2.12-*.jar which shades scala-library, scala-compiler, and scala-reflect 2.12.7. Scala 2.12.7's JrtClassPath.asURLs() crashes on JDK 17+ with:

java.lang.RuntimeException: /packages cannot be represented as URI
    at java.base/jdk.internal.jrtfs.JrtPath.toUri(JrtPath.java:175)
    at scala.tools.nsc.classpath.JrtClassPath.asURLs(DirectoryClassPath.scala:204)

This was fixed in Scala 2.12.9 (scala/bug#11608).

Changes:

  • Change flink.scala.version from 2.12.7 to 2.12.20 in flink/pom.xml (removes the jdk17+ profile). The compile-time version must match the runtime version due to binary incompatibility in Settings.usejavacp() between 2.12.7 and 2.12.20.
  • Add maven-antrun-plugin to flink-1.19 and flink-1.20 profiles that patches the downloaded Flink tgz: strips scala/* classes from flink-scala_2.12-*.jar via zip -d, then copies scala-library/scala-compiler/scala-reflect 2.12.20 jars into FLINK_HOME/lib. Scoped to Flink 1.x only.
  • Add patchFlinkScala() in DownloadUtils for integration tests that download Flink via DownloadUtils.downloadFlink() to ~/.cache/. Guards on scalaVersion == "2.12" and flinkVersion < 2.0.

Experimental support for Java 17 was added in Flink 1.18. (FLINK-15736)

Flink 1.19/1.20 sticky with Scala 2.12.7 due to FLINK-12461, upgrade Scala is a necessary step for JDK 17 support.

3. JPMS args and JDK 17 CI

JDK 17 enforces strong module encapsulation. Many libraries (Netty, Mockito, etc.) require --add-opens flags to access internal JDK APIs. These flags were already needed for surefire tests but were either missing or duplicated per-module.

Assisted-by: GLM 5.2

@pan3793
pan3793 force-pushed the jdk17 branch 3 times, most recently from c9d672a to 43f0295 Compare August 10, 2026 09:15
Cassandra 3.x crashes on JDK 17 because Unsafe.objectFieldOffset() on
hidden classes (lambdas) throws UnsupportedOperationException during
PREPARE statement storage. This is a hard JVM restriction no flag can
override. Cassandra 4.x handles this correctly.

- Replace cassandra-unit with Testcontainers cassandra:4.1.3
- Rewrite CassandraInterpreterTest to use CassandraContainer
- Update test expectation HTML files for Cassandra 4.x table options
- Normalize localhost/<unresolved> InetSocketAddress format in assertions
  (JDK 17+ renders unresolved addresses differently)

Assisted-by: GLM 5.2
Flink 1.x binary tgz bundles flink-scala_2.12 which shades scala-library,
scala-compiler, and scala-reflect 2.12.7. Scala 2.12.7's
JrtClassPath.asURLs() crashes on JDK 17+ with "/packages cannot be
represented as URI" (scala/bug#11608, fixed in Scala 2.12.9).

- Change flink.scala.version from 2.12.7 to 2.12.20 for all JDKs
  (binary incompatibility in Settings.usejavacp() between 2.12.7 and 2.12.20)
- Add maven-antrun-plugin to flink-1.19/flink-1.20 profiles that strips
  scala/* from flink-scala_2.12-*.jar and copies 2.12.20 jars into FLINK_HOME/lib
- Add patchFlinkScala() in DownloadUtils for integration tests that download
  Flink to ~/.cache/ (guards on scalaVersion=="2.12" and flinkVersion<2.0)

Assisted-by: GLM 5.2
Add extraJavaTestArgs property to root pom.xml with --add-opens flags
for JDK 17+ module access, and reference it from surefire argLine in
all test modules. Remove the old per-module java-17 profile in
spark/interpreter/pom.xml that duplicated these flags.

Mirror the same flags in bin/common.sh and bin/common.cmd via
JPMS_JAVA_OPTS so Zeppelin server and interpreter processes launched
by bin scripts get the same module-access flags.

Enable JDK 17 in the flink CI matrix.

Assisted-by: GLM 5.2
@pan3793 pan3793 changed the title [DO-NOT-REVIEW] Test Java 17 [ZEPPELIN-6311] Support JDK 17 Aug 11, 2026
@pan3793

pan3793 commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

@jongyoul, the current changes in this PR should make Zeppelin compatible with JDK 17, the real hacky part is the patch for Flink - Flink does not work out of the box until 2.0, let me know your thoughts about this part. (I may not be able to add support for Flink 2.0, and I hope others can take this if you think we should not hack Flink 1.19/1.20 but upgrade to Flink 2.0 first)

@pan3793

pan3793 commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

I split the Cassandra part change into ZEPPELIN-6639

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant