Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 14 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ jobs:
with:
distribution: temurin
java-version: 21
cache: 'maven'
- name: Setup Maven
uses: stCarolas/setup-maven@v5
with:
Expand All @@ -54,8 +55,9 @@ jobs:
with:
distribution: temurin
java-version: 21
cache: 'gradle'
- name: Run Gradle example benchmarks
run: ./gradlew :example-gradle:jmh
run: ./gradlew :examples:example-gradle:jmh

build-and-run-maven:
strategy:
Expand All @@ -70,32 +72,39 @@ jobs:
with:
distribution: temurin
java-version: 21
cache: 'maven'
- name: Setup Maven
uses: stCarolas/setup-maven@v5
with:
maven-version: 3.9.9
- name: Publish to mavenLocal
run: ./gradlew -p jmh-fork publishToMavenLocal
- name: Build Maven example
run: cd example-maven && mvn package -q
run: cd examples/example-maven && mvn package -q
- name: Run Maven example benchmarks
run: java -jar example-maven/target/example-maven-1.0-SNAPSHOT.jar
run: java -jar examples/example-maven/target/example-maven-1.0-SNAPSHOT.jar

walltime-benchmarks:
runs-on: codspeed-macro
strategy:
matrix:
distribution: [corretto, temurin, oracle, zulu, microsoft, semeru]
steps:
- uses: actions/checkout@v4
with:
submodules: true
- uses: actions/setup-java@v4
with:
distribution: temurin
distribution: ${{ matrix.distribution }}
java-version: 21
cache: 'gradle'
- name: Run the benchmarks
uses: CodSpeedHQ/action@main
env:
CODSPEED_SKIP_UPLOAD: ${{ matrix.distribution != 'temurin' }}
with:
mode: walltime
run: ./gradlew :example-gradle:jmh
run: ./gradlew :examples:example-gradle:jmh

check:
runs-on: ubuntu-latest
Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
[submodule "jmh-fork/jmh-core/src/main/java/io/codspeed/instrument_hooks/instrument-hooks"]
path = jmh-fork/jmh-core/src/main/java/io/codspeed/instrument_hooks/instrument-hooks
url = https://github.com/CodSpeedHQ/instrument-hooks.git
[submodule "examples/TheAlgorithms-Java"]
path = examples/TheAlgorithms-Java
url = https://github.com/TheAlgorithms/Java.git
131 changes: 131 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

**codspeed-jvm** is a CodSpeed integration for JVM-based projects that use JMH (Java Microbenchmark Harness). It provides walltime benchmarking support and performance tracking in CI.

### Key Components

- **`jmh-fork/`**: Forked OpenJDK JMH with CodSpeed walltime result collection. Maven multi-module project used as a composite build dependency.
- **`examples/example-gradle/`**: Gradle-based example benchmarks using the custom JMH fork
- **`examples/example-maven/`**: Maven-based example benchmarks
- **`examples/TheAlgorithms-Java/`**: TheAlgorithms submodule with additional benchmarks

## Build & Development

### Prerequisites

- **Java**: JDK 21+ (for main project), JDK 11+ (for jmh-fork modules)
- **Maven**: 3.9.9+ (for jmh-fork)
- **Gradle**: Uses gradlew wrapper (no separate installation needed)

### Common Commands

#### Building

```bash
# Build and run Gradle example benchmarks
./gradlew :examples:example-gradle:jmh

# Build JAR artifact only (useful for local profiling)
./gradlew :examples:example-gradle:jmhJar

# Build and install JMH fork to local Maven repo (required before Maven example)
cd jmh-fork && mvn clean install -DskipTests -q

# Build Maven example (after installing JMH fork)
cd examples/example-maven && mvn package -q
java -jar examples/example-maven/target/example-maven-1.0-SNAPSHOT.jar
```

#### Testing

```bash
# Run CodSpeed tests for the JMH fork integration
cd jmh-fork && mvn test -pl jmh-core -Dtest="io.codspeed.*.*"

# Run with CodSpeed locally (walltime mode)
codspeed run --mode walltime --skip-upload -- ./gradlew :examples:example-gradle:jmh

# Run with specific pattern filtering
CODSPEED_LOG=trace codspeed run --mode walltime --skip-upload -- java -jar examples/example-gradle/build/libs/example-gradle-1.0-SNAPSHOT-jmh.jar ".*BacktrackingBenchmark.*"
```

#### Code Quality

```bash
# Run pre-commit hooks manually (linting and formatting)
pre-commit run --all-files
# Uses Google Java Formatter v1.25.2
```

### Build Structure

- **Root**: Gradle root project (`settings.gradle.kts`) includes:
- `examples:example-gradle` as a regular subproject
- `jmh-fork` as a composite build (allows independent versioning)
- **jmh-fork**: Maven multi-module project with submodules `jmh-core`, `jmh-generator-*`, etc.

### Publishing JMH Fork

The JMH fork is published under group ID `io.codspeed.jmh`. When updating versions:

```bash
cd jmh-fork && mvn versions:set -DnewVersion=x.y.z
```

Then update the version reference in root `build.gradle.kts`.

## Architecture & Key Patterns

### Multi-Build Structure

The repository combines Maven and Gradle via Gradle's composite build feature:
- **jmh-fork** (Maven multi-module) is built as a composite dependency for independent versioning
- Example projects depend on the forked JMH packages

### CodSpeed Integration Points

1. **Walltime Collection**: Modified JMH fork in `jmh-fork/jmh-core` collects walltime metrics during benchmark execution
2. **JNI Bindings**: Native code hooks (C) for precise instrumentation in `jmh-fork/jmh-core/src/main/java/io/codspeed/`
3. **CI Integration**: GitHub Actions workflows trigger CodSpeed measurement runs on `codspeed-macro` runners

### Benchmark Architecture

- Benchmarks use standard JMH annotations (`@Benchmark`, `@Fork`, etc.)
- The `gradle-jmh-plugin` (v0.7.2) generates benchmark JARs from sources
- Both Gradle (`me.champeau.jmh`) and Maven profiles support building executable benchmark JARs

## CI/CD & Testing

### GitHub Actions Workflows (`.github/workflows/ci.yml`)

- **lint**: Pre-commit hooks (Java formatting, YAML validation, file endings)
- **test**: CodSpeed-specific unit tests (jmh-core integration tests)
- **build-and-run-gradle**: Matrix test across Linux, macOS, Windows
- **build-and-run-maven**: Matrix test for Maven example, including publishToMavenLocal step
- **walltime-benchmarks**: Runs on CodSpeed infrastructure for real performance tracking

## Modifying JMH Fork

1. Make changes in `jmh-fork/`
2. Bump version: `cd jmh-fork && mvn versions:set -DnewVersion=X.Y.Z`
3. Publish locally: `cd jmh-fork && mvn clean install -DskipTests -q`
4. Update version reference in root `build.gradle.kts` if needed
5. Test both Gradle and Maven examples to verify compatibility

## Profiling & Debugging

Use `just` commands for quick profiling workflows (see `Justfile`):
- `just profile-codspeed` — walltime profiling via CodSpeed runner
- `just profile-perf` — Linux perf + flamegraph (requires Linux + JDK perf integration)
- `just profile-asprof` — async-profiler flamegraph

## Notes

- **Submodules**: Repository uses Git submodules (`.gitmodules`). Clone with `--recurse-submodules`.
- **Java Versions**: Root project uses Java 21; jmh-fork runs on Java 11+.
- **Pre-commit Exclusions**: jmh-fork is mostly excluded from pre-commit hooks except for `jmh-core/src/main/java/io/codspeed/`.
- **CodSpeed Credentials**: CI uses OIDC for authentication.
1 change: 1 addition & 0 deletions CLAUDE.md
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

## Bumping the JMH fork version

The JMH fork uses a versioning scheme like `1.37.0-codspeed.N`. To bump the version across all modules:
The JMH fork is published under group ID `io.codspeed.jmh`. To bump the version across all modules:

```bash
cd jmh-fork && mvn versions:set -DnewVersion=1.37.0-codspeed.2
cd jmh-fork && mvn versions:set -DnewVersion=x.y.z
```

This updates all `pom.xml` files in the multi-module project. After bumping, also update the version reference in `build.gradle.kts`.
11 changes: 0 additions & 11 deletions example-gradle/build.gradle.kts

This file was deleted.

1 change: 0 additions & 1 deletion example-maven/src/main/java/bench/FibBenchmark.java

This file was deleted.

1 change: 0 additions & 1 deletion example-maven/src/main/java/bench/Rle.java

This file was deleted.

1 change: 0 additions & 1 deletion example-maven/src/main/java/bench/RleBenchmark.java

This file was deleted.

1 change: 0 additions & 1 deletion example-maven/src/main/java/bench/SleepBenchmark.java

This file was deleted.

1 change: 1 addition & 0 deletions examples/TheAlgorithms-Java
Submodule TheAlgorithms-Java added at 5e06b1
24 changes: 24 additions & 0 deletions examples/example-gradle/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
plugins {
java
id("me.champeau.jmh") version "0.7.2"
}

group = "io.codspeed"
version = "1.0-SNAPSHOT"

jmh {
jmhVersion.set("0.1.0")
}

sourceSets {
named("jmh") {
java {
srcDir("../TheAlgorithms-Java/src/main/java")
}
}
}

dependencies {
jmh("org.apache.commons:commons-lang3:3.20.0")
jmh("org.apache.commons:commons-collections4:4.5.0")
}
120 changes: 120 additions & 0 deletions examples/example-gradle/src/jmh/java/bench/BacktrackingBenchmark.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
package bench;

import com.thealgorithms.backtracking.*;
import java.util.List;
import java.util.concurrent.TimeUnit;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.infra.Blackhole;

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 1, time = 1)
@Measurement(iterations = 3, time = 1)
@Fork(1)
public class BacktrackingBenchmark {

// -- N-Queens --

@State(Scope.Benchmark)
public static class NQueensState {
@Param({"4", "5", "6", "7", "8"})
public int nQueens;
}

@Benchmark
public List<List<String>> nQueensSolver(NQueensState state) {
return NQueens.getNQueensArrangements(state.nQueens);
}

// -- Parentheses Generation --

@State(Scope.Benchmark)
public static class ParenthesesState {
@Param({"3", "4", "5", "6"})
public int nParens;
}

@Benchmark
public List<String> generateParentheses(ParenthesesState state) {
return ParenthesesGenerator.generateParentheses(state.nParens);
}

// -- Combinations --

@State(Scope.Benchmark)
public static class CombinationsState {
@Param({"5", "6", "7", "8", "9"})
public int nCombinations;
}

@Benchmark
public void generateCombinations(CombinationsState state, Blackhole bh) {
Integer[] arr = new Integer[state.nCombinations];
for (int i = 0; i < state.nCombinations; i++) {
arr[i] = i;
}
bh.consume(Combination.combination(arr, 3));
}

// -- Permutations --

@State(Scope.Benchmark)
public static class PermutationsState {
@Param({"3", "4", "5", "6", "7"})
public int nPermutations;
}

@Benchmark
public void permutations(PermutationsState state, Blackhole bh) {
Integer[] arr = new Integer[state.nPermutations];
for (int i = 0; i < state.nPermutations; i++) {
arr[i] = i;
}
bh.consume(Permutation.permutation(arr));
}

// -- Sudoku Solver --

@State(Scope.Benchmark)
public static class SudokuState {
public int[][] sudokuBoard;

@Setup(Level.Invocation)
public void setupSudoku() {
sudokuBoard =
new int[][] {
{3, 0, 6, 5, 0, 8, 4, 0, 0},
{5, 2, 0, 0, 0, 0, 0, 0, 0},
{0, 8, 7, 0, 0, 0, 0, 3, 1},
{0, 0, 3, 0, 1, 0, 0, 8, 0},
{9, 0, 0, 8, 6, 3, 0, 0, 5},
{0, 5, 0, 0, 9, 0, 6, 0, 0},
{1, 3, 0, 0, 0, 0, 2, 5, 0},
{0, 0, 0, 0, 0, 0, 0, 7, 4},
{0, 0, 5, 2, 0, 6, 3, 0, 0},
};
}
}

@Benchmark
public boolean sudokuSolver(SudokuState state) {
return SudokuSolver.solveSudoku(state.sudokuBoard);
}

// -- Subsequences --

@State(Scope.Benchmark)
public static class SubsequencesState {
@Param({"8", "10", "12"})
public int nSubsequences;
}

@Benchmark
public void generateSubsequences(SubsequencesState state, Blackhole bh) {
List<Integer> seq = new java.util.ArrayList<>();
for (int i = 0; i < state.nSubsequences; i++) {
seq.add(i);
}
bh.consume(SubsequenceFinder.generateAll(seq));
}
}
Loading
Loading