Publish TestingBot results as a GitHub check (testingbotChecks)#30
Conversation
Adds a TestingBotChecksPublisher (freestyle post-build action + the testingbotChecks pipeline step) that publishes the build's TestingBot outcome as a GitHub check via the checks-api plugin. The check name (its context) and summary message are configurable — the modern equivalent of the custom context/message Sauce exposes for GitHub Pull Request Builder jobs. The conclusion reflects the TestingBot sessions in the build (all passed -> success), falling back to the overall build result when none are present, and the check links to the embedded TestingBot build report. Delivery to GitHub is provided at runtime by the github-checks plugin; when absent the publish is a safe no-op (NullChecksPublisher). Depends on checks-api + display-url-api (both BOM-managed).
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@README.md`:
- Line 151: Update the “Publishing results as a GitHub check” heading in
README.md to use the repository’s required setext heading style instead of an
ATX heading, preserving the heading text and section structure.
In `@src/main/java/testingbot/TestingBotChecksPublisher.java`:
- Around line 95-103: Update the result-processing loop in
TestingBotChecksPublisher to normalize and deduplicate the session IDs returned
by TestingBotReportFactory.findSessionIDs(cr), then count and render each unique
session rather than each CaseResult. Ensure total, passed, rows, and the
resulting conclusion all reflect the unique-session counts while preserving
existing handling for cases with no sessions.
In `@src/main/resources/testingbot/TestingBotChecksPublisher/help-message.html`:
- Around line 2-4: Update the help text describing the TestingBot check link to
state that it links to the embedded TestingBot build report when available, and
otherwise falls back to the Jenkins run URL.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e49c33b6-93ff-4f40-a78d-f44d8957762e
📒 Files selected for processing (6)
README.mdpom.xmlsrc/main/java/testingbot/TestingBotChecksPublisher.javasrc/main/resources/testingbot/TestingBotChecksPublisher/config.jellysrc/main/resources/testingbot/TestingBotChecksPublisher/help-message.htmlsrc/main/resources/testingbot/TestingBotChecksPublisher/help-name.html
| } | ||
| ``` | ||
|
|
||
| ## Publishing results as a GitHub check |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Use the repository’s required heading style.
markdownlint-cli2 reports MD003 here because this section uses an ATX heading while the repository expects setext headings. Convert it to:
Proposed fix
-## Publishing results as a GitHub check
+Publishing results as a GitHub check
+-----------------------------------📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ## Publishing results as a GitHub check | |
| Publishing results as a GitHub check | |
| ----------------------------------- |
🧰 Tools
🪛 markdownlint-cli2 (0.23.0)
[warning] 151-151: Heading style
Expected: setext; Actual: atx
(MD003, heading-style)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@README.md` at line 151, Update the “Publishing results as a GitHub check”
heading in README.md to use the repository’s required setext heading style
instead of an ATX heading, preserving the heading text and section structure.
Source: Linters/SAST tools
| if (TestingBotReportFactory.findSessionIDs(cr).isEmpty()) { | ||
| continue; | ||
| } | ||
| total++; | ||
| boolean casePassed = cr.isPassed(); | ||
| if (casePassed) { | ||
| passed++; | ||
| } | ||
| rows.add("| " + cr.getFullName() + " | " + (casePassed ? "✅ passed" : "❌ failed") + " |"); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Count unique TestingBot sessions, not merely test cases.
TestingBotReportFactory.findSessionIDs(cr) returns every matching session token, but this code only checks whether the list is non-empty and increments total once per CaseResult. Multiple sessions in one case, or repeated tokens across fields, therefore produce incorrect totals, pass counts, rows, and potentially the conclusion. Normalize/deduplicate the IDs and count/render each session, or enforce a one-session-per-case contract upstream.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/main/java/testingbot/TestingBotChecksPublisher.java` around lines 95 -
103, Update the result-processing loop in TestingBotChecksPublisher to normalize
and deduplicate the session IDs returned by
TestingBotReportFactory.findSessionIDs(cr), then count and render each unique
session rather than each CaseResult. Ensure total, passed, rows, and the
resulting conclusion all reflect the unique-session counts while preserving
existing handling for cases with no sessions.
| Optional summary shown on the check. When left empty, a summary is generated from the TestingBot | ||
| sessions in the build (for example, <code>3 of 4 TestingBot session(s) passed.</code>). | ||
| The check also links to the embedded TestingBot build report. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Document the report-link fallback.
The publisher links to the embedded TestingBot report only when available; otherwise the check links to the Jenkins run URL. Update this text to reflect both cases.
Proposed wording
- The check also links to the embedded TestingBot build report.
+ When available, the check links to the embedded TestingBot build report;
+ otherwise it links to the Jenkins run.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| Optional summary shown on the check. When left empty, a summary is generated from the TestingBot | |
| sessions in the build (for example, <code>3 of 4 TestingBot session(s) passed.</code>). | |
| The check also links to the embedded TestingBot build report. | |
| Optional summary shown on the check. When left empty, a summary is generated from the TestingBot | |
| sessions in the build (for example, <code>3 of 4 TestingBot session(s) passed.</code>). | |
| When available, the check links to the embedded TestingBot build report; | |
| otherwise it links to the Jenkins run. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/main/resources/testingbot/TestingBotChecksPublisher/help-message.html`
around lines 2 - 4, Update the help text describing the TestingBot check link to
state that it links to the embedded TestingBot build report when available, and
otherwise falls back to the Jenkins run URL.
…iew on #30) - TestingBotChecksPublisher now counts and renders each distinct session id (trimmed + deduplicated across cases) instead of one entry per CaseResult, so a case logging several sessions, or a session shared by several cases, is reflected accurately in total/passed/rows and the conclusion. Cases without sessions still contribute nothing. - help-message.html now states the check links to the embedded build report when available and otherwise to the Jenkins run. Skipped the README setext-heading suggestion: the repo uses ATX headings (## / ###) for every section and subsection; only the H1 title is setext, so ## here is consistent.
PRs #29 and #30 were merged into the build-report-embed branch, but that branch had already been squash-merged to master as #28 beforehand, so their code never reached master. This brings the stranded changes forward, on top of the current master (which already has #28 and the #31 inline-media feature): - #29: run the freestyle tunnel on the build's agent (TunnelManager.startOnChannel/ stopOnChannel + shared registry; wrapper + testingbotTunnel step delegate to it). - #30: TestingBotChecksPublisher (testingbotChecks) publishing results as a GitHub check, plus the checks-api + display-url-api dependencies and docs. No behavior change versus the already-reviewed #29/#30; only rebased onto master. The inline-media (#31) files are left untouched.
1 participant
What
Publishes the TestingBot outcome of a build as a GitHub check (✅/❌ on the commit & PR), the modern equivalent of Sauce Labs' "custom context + message" GitHub Pull Request Builder option.
Change
New
TestingBotChecksPublisher(Recorder+SimpleBuildStep,@Symbol("testingbotChecks")) — works as a freestyle post-build action and as a pipeline step:name— the check name/context on the commit/PR (defaultTestingBot).message— optional summary; when empty, a summary is generated from the TestingBot sessions in the build.Delivery to GitHub is handled at runtime by the
github-checksplugin + a GitHub App; when absent,ChecksPublisherFactory.fromRunreturns aNullChecksPublisherso the step is a safe no-op. Depends onchecks-api+display-url-api(both BOM-managed).Validation
mvn -DskipTests verify(SpotBugs on) — clean.hpi:runscript console:name/messageconfig round-trips; default name isTestingBot.testingbotChecks(...)runs green as a no-op with no GitHub Checks impl installed.ChecksPublisherFactory, the publishedChecksDetailsis verified end-to-end: passing build → nameTestingBot, statusCOMPLETED, conclusionSUCCESS, configured summary, run-URL details link; failing build → conclusionFAILURE.Note
Stacked on #28 (
build-report-embed) for the build-report details URL. Retarget tomasteronce #28 merges.Summary by CodeRabbit