Skip to content

Add test matrix for different java versions #1781

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zeitlinger merged 33 commits into from
Jan 26, 2026
Merged

Add test matrix for different java versions #1781

Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
3f2e73b
add test matrix
jaydeluca Jan 15, 2026
c709cc7
make graal conditional for java 17+
jaydeluca Jan 15, 2026
0cdbb3b
merge main
jaydeluca Jan 15, 2026
8472199
errorprone too
jaydeluca Jan 15, 2026
4d19fd7
fix spotless disable
jaydeluca Jan 15, 2026
18763cb
disable cache
jaydeluca Jan 15, 2026
4bcc776
try again
jaydeluca Jan 15, 2026
58e0b20
move errorprone into diff profile
jaydeluca Jan 15, 2026
69f7901
another try
jaydeluca Jan 15, 2026
36ed925
conditionally run additioanl profiles
jaydeluca Jan 20, 2026
ce39d7a
add flag to error prone
jaydeluca Jan 20, 2026
1f49ac5
move more dependencies
jaydeluca Jan 20, 2026
61df78b
merge main
jaydeluca Jan 20, 2026
2b2bbf5
fix benchmarks module
jaydeluca Jan 20, 2026
1df9c69
fix spring boot test module
jaydeluca Jan 20, 2026
4d1f6a3
linting
jaydeluca Jan 20, 2026
5e4bfdf
linting
jaydeluca Jan 20, 2026
690824f
linting
jaydeluca Jan 20, 2026
16d922a
linting
jaydeluca Jan 21, 2026
fe0f06a
change mvn package to install
jaydeluca Jan 21, 2026
780752c
skip gpg
jaydeluca Jan 21, 2026
beadc0a
undo
jaydeluca Jan 21, 2026
2b45c09
test install again and skip tests
jaydeluca Jan 21, 2026
e5c129e
exclude others from release build
jaydeluca Jan 21, 2026
ae836bf
editorconfig
jaydeluca Jan 21, 2026
c972680
Update .mise/tasks/build-release.sh
jaydeluca Jan 21, 2026
c6b6d5a
retry other java versions
jaydeluca Jan 21, 2026
efc9c9f
try and fix j8 and 11
jaydeluca Jan 21, 2026
9548485
undo 8 and 11
jaydeluca Jan 21, 2026
501afb0
editorconfig
jaydeluca Jan 21, 2026
7b3a748
editorconfig
jaydeluca Jan 21, 2026
e887c64
Merge branch 'main' into feature/add-multi-version-ci-testing
jaydeluca Jan 25, 2026
cd80efc
Update .github/workflows/multi-version-test.yml
jaydeluca Jan 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .editorconfig
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,12 @@ max_line_length = 200
max_line_length = 300

[pom.xml]
max_line_length = 110
max_line_length = 120

[*.py]
# checked by black
indent_size = 4
max_line_length = 120

[{.mise/tasks/build-release.sh,.github/workflows/multi-version-test.yml}]
max_line_length = 200
39 changes: 39 additions & 0 deletions .github/workflows/multi-version-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
name: Java-Version Compatibility Tests

on: [pull_request]

permissions: {}

jobs:
compatibility-test:
name: Test on Java ${{ matrix.java }}
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
java: [17, 21, 25]
Copy link
Copy Markdown
Member

@zeitlinger zeitlinger Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not 8, 11?

Copy link
Copy Markdown
Collaborator Author

@jaydeluca jaydeluca Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i couldnt get them to work, can't remember why, let me try again to refresh my memory

Copy link
Copy Markdown
Collaborator Author

@jaydeluca jaydeluca Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

theres a bunch of test code that uses switch statements and """...""" text blocks. I can refactor them all if we think it's worth it?

Copy link
Copy Markdown
Collaborator Author

@jaydeluca jaydeluca Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, JUnit Jupiter 6.0.2 requires Java 17+ to run

Copy link
Copy Markdown
Collaborator Author

@jaydeluca jaydeluca Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ive put a lot of effort into getting this to work for 8 and 11, and it's becoming a massive messy change. and adds a ton of complexity to the pom files. I propose that we keep the scope of this PR to what it is now, running the tests on 17+, and i can open an issue to track a followup where instead of running our entire test suite on every java version, we create some smoke tests that are more multi-version friendly in terms of dependencies etc, and then we can run that across 8 and 11 as well

steps:
- name: Check out
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6
with:
persist-credentials: false

- name: Set up Java ${{ matrix.java }}
id: setup-java
uses: actions/setup-java@f2beeb24e141e01a676f977032f5a29d81c9e27e # v5
Comment thread
jaydeluca marked this conversation as resolved.
Outdated
with:
distribution: "temurin"
java-version: ${{ matrix.java }}

- name: Cache local Maven repository
uses: actions/cache@8b402f58fbc84540c8b491a91e594a4576fec3d7 # v5.0.2
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-java${{ matrix.java }}-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-java${{ matrix.java }}-
${{ runner.os }}-maven-

- name: Build and test on Java ${{ matrix.java }}
run: ./mvnw clean install -Dtest.java.version=${{ matrix.java }} -Dspotless.skip=true -Dcheckstyle.skip=true -Dwarnings=-nowarn -Dcoverage.skip=true
2 changes: 1 addition & 1 deletion .mise/tasks/build-release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ set -euo pipefail
VERSION=${usage_tag#v}

mise run set-version "$VERSION"
mvn -B package -P 'release,!default' -Dmaven.test.skip=true
mvn -B package -P 'release,!default,!examples-and-integration-tests' -Dmaven.test.skip=true -Dgpg.skip=true
Copy link
Copy Markdown
Member

@zeitlinger zeitlinger Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this needed?

Copy link
Copy Markdown
Collaborator Author

@jaydeluca jaydeluca Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding the profiles has added some issues to the way some of these other tasks resolve everything.

from:
https://github.com/prometheus/client_java/actions/runs/21207964631/job/61008982563?pr=1781

Error:  Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.14.1:testCompile (default-testCompile) on project prometheus-metrics-exposition-formats: Compilation failure: Compilation failure: 
Error:  /home/runner/work/client_java/client_java/prometheus-metrics-exposition-formats-shaded/target/metrics-exposition-formats/src/test/java/io/prometheus/metrics/expositionformats/ProtobufExpositionFormatsTest.java:[11,45] cannot find symbol
Error:    symbol: class ExpositionFormatsTest
Error:  /home/runner/work/client_java/client_java/prometheus-metrics-exposition-formats-shaded/target/metrics-exposition-formats/src/test/java/io/prometheus/metrics/expositionformats/ProtobufExpositionFormatsTest.java:[13,3] method does not override or implement a method from a supertype

the shaded module's tests need a test-jar dependency that's explicitly excluded from the release profile, so thats why i added the !examples-and-integration-tests to disable them (i assumed that would be ok to skip for testing the release build)

I was also encountering gpg issues both in CI and locally, not sure if this is a "me" thing, but didn't think it was necessary for this test step

Comment thread
jaydeluca marked this conversation as resolved.
4 changes: 2 additions & 2 deletions benchmarks/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,8 @@
<configuration>
<source>1.8</source>
<target>1.8</target>
<compilerArgs>
<!-- need to add parameters to prevent inheritance -->
<compilerArgs combine.self="override">
<!-- Override parent to prevent ErrorProne inheritance -->
<arg>-parameters</arg>
</compilerArgs>
<annotationProcessorPaths>
Expand Down
181 changes: 97 additions & 84 deletions integration-tests/it-spring-boot-smoke-test/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,95 +89,108 @@

<build>
<plugins>
<plugin>
<groupId>org.graalvm.buildtools</groupId>
<artifactId>native-maven-plugin</artifactId>
<configuration>
<!-- editorconfig-checker-disable -->
<buildArgs>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.api.DisplayNameGenerator$IndicativeSentences
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor$ClassInfo
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor$LifecycleMethods
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.ClassTemplateInvocationTestDescriptor
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.ClassTemplateTestDescriptor
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.DynamicDescendantFilter$Mode
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.ExclusiveResourceCollector$1
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.MethodBasedTestDescriptor$MethodInfo
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.discovery.ClassSelectorResolver$DummyClassTemplateInvocationContext
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.platform.engine.support.store.NamespacedHierarchicalStore$EvaluatedValue
</buildArg>
<buildArg>--initialize-at-build-time=org.junit.platform.launcher.core.DiscoveryIssueNotifier
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.platform.launcher.core.HierarchicalOutputDirectoryProvider
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.platform.launcher.core.LauncherDiscoveryResult$EngineResultInfo
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.platform.suite.engine.SuiteTestDescriptor$LifecycleMethods
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.platform.commons.logging.LoggerFactory$DelegatingLogger
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.execution.ConditionEvaluator
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.execution.InterceptingExecutableInvoker
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.api.extension.ConditionEvaluationResult
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.execution.InvocationInterceptorChain
</buildArg>
</buildArgs>
<!-- editorconfig-checker-enable -->
</configuration>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>com.diffplug.spotless</groupId>
<artifactId>spotless-maven-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<java>
<googleJavaFormat/>
</java>
</configuration>
<executions>
<execution>
<phase>verify</phase>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

<profiles>
<profile>
<id>java17-plus</id>
<activation>
<jdk>[17,)</jdk>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.graalvm.buildtools</groupId>
<artifactId>native-maven-plugin</artifactId>
<configuration>
<!-- editorconfig-checker-disable -->
<buildArgs>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.api.DisplayNameGenerator$IndicativeSentences
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor$ClassInfo
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor$LifecycleMethods
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.ClassTemplateInvocationTestDescriptor
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.ClassTemplateTestDescriptor
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.DynamicDescendantFilter$Mode
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.ExclusiveResourceCollector$1
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.descriptor.MethodBasedTestDescriptor$MethodInfo
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.discovery.ClassSelectorResolver$DummyClassTemplateInvocationContext
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.platform.engine.support.store.NamespacedHierarchicalStore$EvaluatedValue
</buildArg>
<buildArg>--initialize-at-build-time=org.junit.platform.launcher.core.DiscoveryIssueNotifier
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.platform.launcher.core.HierarchicalOutputDirectoryProvider
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.platform.launcher.core.LauncherDiscoveryResult$EngineResultInfo
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.platform.suite.engine.SuiteTestDescriptor$LifecycleMethods
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.platform.commons.logging.LoggerFactory$DelegatingLogger
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.execution.ConditionEvaluator
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.execution.InterceptingExecutableInvoker
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.api.extension.ConditionEvaluationResult
</buildArg>
<buildArg>
--initialize-at-build-time=org.junit.jupiter.engine.execution.InvocationInterceptorChain
</buildArg>
</buildArgs>
<!-- editorconfig-checker-enable -->
</configuration>
</plugin>
<plugin>
<groupId>com.diffplug.spotless</groupId>
<artifactId>spotless-maven-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<java>
<googleJavaFormat/>
</java>
</configuration>
<executions>
<execution>
<phase>verify</phase>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>

</project>
Loading