Enable TestLens by britter · Pull Request #2314 · spockframework/spock

britter · 2026-03-05T09:22:11Z

This adds the testlens-app/setup-testlens action to all workflows running tests.

The app posts a summary of all test failures as a PR comment and provides means to mute unrelated test failures and faster reruns. For more information, please refer to the announcement on our website.

Summary by CodeRabbit

Chores
- Added a TestLens setup step to the CI pipeline, placed before the main build to prepare testing infrastructure.
- The TestLens step runs only for pull requests; build and upload steps remain unchanged.

coderabbitai · 2026-03-05T09:22:28Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 46b172c2-bd15-4a9b-9843-92ca2371ec97

📥 Commits

Reviewing files that changed from the base of the PR and between ad552d5 and 17fe6dd.

📒 Files selected for processing (2)

.github/workflows/branches-and-prs.main.kts
.github/workflows/branches-and-prs.yaml

🚧 Files skipped from review as they are similar to previous changes (2)

.github/workflows/branches-and-prs.main.kts
.github/workflows/branches-and-prs.yaml

📝 Walkthrough

Walkthrough

Adds a new "Set up TestLens" step to the Build and Verify job in GitHub Actions workflows (Kotlin DSL and YAML), adds an import for CustomAction in the Kotlin DSL, and conditions the TestLens step to run only on pull requests.

Changes

Cohort / File(s)	Summary
GitHub Actions (Kotlin DSL) `.github/workflows/branches-and-prs.main.kts`	Added `import io.github.typesafegithub.workflows.domain.actions.CustomAction` and inserted a "Set up TestLens" step using `CustomAction` with condition `github.event_name == 'pull_request'`.
GitHub Actions (YAML) `.github/workflows/branches-and-prs.yaml`	Inserted `Set up TestLens` step (`testlens-app/setup-testlens@v1`) before the build step with a PR-only condition; subsequent step identifiers renumbered.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐇 I hopped through code at dawn's first light,

I nudged a step so PRs shine bright,
TestLens greets builds before they start,
Pipelines hum with a hopeful heart,
A rabbit's skip — the workflow's right! 🥕✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Enable TestLens' directly and specifically describes the main change in the pull request: adding TestLens setup to GitHub workflows.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

📝 Coding Plan

Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Tip

CodeRabbit can suggest fixes for GitHub Check annotations.

Configure the reviews.tools.github-checks setting to adjust the time to wait for GitHub Checks to complete.

greptile-apps · 2026-03-05T09:24:11Z

Greptile Summary

This PR integrates the TestLens action into Spock's reusable setup-build-env composite action, enabling automatic PR comment summaries of test failures for all workflows that use this shared setup step.

The new step is correctly pinned to a full commit SHA (7f83d4feb810f54f9a4ccc1df7cbc79ee3cd836b) with a version comment (# v1.7.1), following supply-chain security best practices. It is appropriately named ('Set up TestLens') and guarded with if: ${{ github.event_name == 'pull_request' }} so it only activates in PR contexts.

Confidence Score: 4/5

This PR is safe to merge; the change is minimal, well-structured, and follows security best practices for pinning third-party actions.
The PR introduces a single new step in a composite action, pinned to a commit SHA with clear conditional execution. The logic is straightforward with no breaking changes. The change integrates cleanly into the existing workflow setup pattern.
No files require special attention.

Sequence Diagram

sequenceDiagram
    participant W as Calling Workflow
    participant C as setup-build-env (composite)
    participant JDK as setup-jdks
    participant GR as gradle/actions/setup-gradle
    participant TL as testlens-app/setup-testlens
    participant GJ as gradlew javaToolchains

    W->>C: uses: ./.github/actions/setup-build-env
    C->>JDK: Set up JDKs
    JDK-->>C: done
    C->>GR: Init Gradle Build Action
    GR-->>C: done
    C->>TL: Set up TestLens (only if pull_request event)
    TL-->>C: registered / token acquired
    C->>GJ: Gradle javaToolchains
    GJ-->>C: done
    C-->>W: build env ready

_{Last reviewed commit: 1eabd8d}

.github/actions/setup-build-env/action.yml

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/actions/setup-build-env/action.yml:
- Line 20: Add an optional boolean input (e.g., inputs.run_testlens default:
"false") to the composite action and guard the step that invokes
testlens-app/setup-testlens (the step using the symbol "uses:
testlens-app/setup-testlens@7f83d4f...") with an if that requires the input to
be true and that the workflow is a PR/test context (for example check
github.event_name == 'pull_request' or another explicit test/event check).
Update the action.yml inputs section to declare run_testlens with default false
and change the step invoking testlens-app/setup-testlens to only run when the
input is true and the event is a pull_request/test context so it won't execute
in release jobs with secrets.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: aec62f6e-0720-4043-8ef2-af62109aaf97

📥 Commits

Reviewing files that changed from the base of the PR and between 383b3ea and d5cd984.

📒 Files selected for processing (1)

.github/actions/setup-build-env/action.yml

.github/actions/setup-build-env/action.yml

leonard84 · 2026-03-12T09:41:00Z

Please move the testlens setup to this location

spock/.github/workflows/branches-and-prs.main.kts

Line 115 in 2db4303

)

britter · 2026-03-12T10:34:50Z

@leonard84 like 0fef76f?

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/branches-and-prs.main.kts:
- Around line 117-120: The workflow currently references the TestLens action
using a moving tag ("v1") in the CustomAction call (CustomAction("testlens-app",
"setup-testlens", "v1")); replace that tag with the full commit SHA for the
intended release (e.g., CustomAction("testlens-app", "setup-testlens",
"<full-commit-sha>")) so the uses(...) entry is pinned immutably; ensure the SHA
is full-length and corresponds to the action repository commit you want to lock
to.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c5939c39-e1fc-419c-8b69-11d7c3784ddf

📥 Commits

Reviewing files that changed from the base of the PR and between 1eabd8d and 0fef76f.

📒 Files selected for processing (2)

.github/workflows/branches-and-prs.main.kts
.github/workflows/branches-and-prs.yaml

.github/workflows/branches-and-prs.main.kts

codecov · 2026-03-12T10:48:53Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.16%. Comparing base (23858ab) to head (17fe6dd).

Additional details and impacted files

@@            Coverage Diff            @@
##             master    #2314   +/-   ##
=========================================
  Coverage     82.16%   82.16%           
  Complexity     4822     4822           
=========================================
  Files           472      472           
  Lines         15036    15036           
  Branches       1905     1905           
=========================================
+ Hits          12354    12355    +1     
  Misses         1989     1989           
+ Partials        693      692    -1

see 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

leonard84

Unfortunately, you've not enabled edits by maintainers, so I couldn't do this by myself. Nor could I rebase/update the PR, to be able to merge it.

.github/workflows/branches-and-prs.main.kts

marcphilipp · 2026-03-15T07:30:16Z

Unfortunately, you've not enabled edits by maintainers, so I couldn't do this by myself. Nor could I rebase/update the PR, to be able to merge it.

That's due to a limitation of GitHub:

https://github.com/orgs/community/discussions/5634

britter · 2026-03-15T08:43:13Z

@leonard84 done!

testlens-app · 2026-03-15T12:28:09Z

✅ All tests passed ✅

⚠️ TestLens detected flakiness ⚠️

Test Summary

Check	Project/Task	Test	Runs
Verify Branches and PRs / Build and Verify (2.5, 8, windows-latest)	:spock-specs:test	ParallelSpec > @ResourceLock with only READ allows parallel execution of data-driven features	⚠️ ⚠️
Verify Branches and PRs / Build and Verify (5.0, 11, ubuntu-latest)	:spock-specs:test	PollingConditionsSpec > can override timeout per invocation with condition method '#conditionMethod' and local variable > can override timeout per invocation with condition method '.within' and local variable	❌ ✅

🏷️ Commit: 17fe6dd
▶️ Tests: 106975 executed
⚪️ Checks: 33/33 completed

Test Failure

ParallelSpec > @ResourceLock with only READ allows parallel execution of data-driven features (:spock-specs:test in Verify Branches and PRs / Build and Verify (2.5, 8, windows-latest))

Condition not satisfied:

atomicInteger.get() == 3
|             |     |
3             2     false

	at apackage.ASpec.writeA(script17735779789941057490708.groovy:14)

expected	actual
3	2

Learn more about TestLens at testlens.app.

This comment has been minimized.

Sign in to view

greptile-apps bot reviewed Mar 5, 2026

View reviewed changes

.github/actions/setup-build-env/action.yml Outdated Show resolved Hide resolved

coderabbitai bot reviewed Mar 5, 2026

View reviewed changes

.github/actions/setup-build-env/action.yml Outdated Show resolved Hide resolved

This comment was marked as outdated.

Sign in to view

britter marked this pull request as draft March 5, 2026 09:33

This comment has been minimized.

Sign in to view

britter marked this pull request as ready for review March 6, 2026 18:21

britter requested a review from leonard84 March 6, 2026 18:21

britter force-pushed the britter/setup-testlens branch from 1eabd8d to 0fef76f Compare March 12, 2026 10:34

This comment has been minimized.

Sign in to view

coderabbitai bot reviewed Mar 12, 2026

View reviewed changes

.github/workflows/branches-and-prs.main.kts Show resolved Hide resolved

leonard84 approved these changes Mar 14, 2026

View reviewed changes

leonard84 requested changes Mar 14, 2026

View reviewed changes

.github/workflows/branches-and-prs.main.kts Outdated Show resolved Hide resolved

britter force-pushed the britter/setup-testlens branch from 0fef76f to ad552d5 Compare March 15, 2026 08:42

This comment has been minimized.

Sign in to view

britter requested a review from leonard84 March 15, 2026 08:42

ci: enable TestLens

17fe6dd

britter force-pushed the britter/setup-testlens branch from ad552d5 to 17fe6dd Compare March 15, 2026 12:28

Conversation

britter commented Mar 5, 2026 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

This comment has been minimized.

coderabbitai bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Walkthrough

Changes

Estimated code review effort

Poem

Uh oh!

greptile-apps bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 4/5

Sequence Diagram

Uh oh!

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment was marked as outdated.

This comment has been minimized.

leonard84 commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

britter commented Mar 12, 2026

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

leonard84 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

marcphilipp commented Mar 15, 2026

Uh oh!

This comment has been minimized.

britter commented Mar 15, 2026

Uh oh!

testlens-app bot commented Mar 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ All tests passed ✅

Test Summary

Test Failure

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

britter commented Mar 5, 2026 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Mar 5, 2026 •

edited

Loading

greptile-apps bot commented Mar 5, 2026 •

edited

Loading

leonard84 commented Mar 12, 2026 •

edited

Loading

codecov bot commented Mar 12, 2026 •

edited

Loading

testlens-app bot commented Mar 15, 2026 •

edited

Loading