Skip to content

ENH: Add collapsible CI log sections for dashboard phases#5973

Open
hjmjohnson wants to merge 2 commits intoInsightSoftwareConsortium:dashboardfrom
hjmjohnson:dash-board-github-ci-instrument-groupings
Open

ENH: Add collapsible CI log sections for dashboard phases#5973
hjmjohnson wants to merge 2 commits intoInsightSoftwareConsortium:dashboardfrom
hjmjohnson:dash-board-github-ci-instrument-groupings

Conversation

@hjmjohnson
Copy link
Member

Summary

  • Adds ci_section_start() / ci_section_end() helper functions to itk_common.cmake
  • Wraps configure, build, test, coverage, memcheck, and submit phases in collapsible log sections
  • Detects CI environment via standard env vars and emits platform-appropriate markers:
    • GitHub Actions: ::group:: / ::endgroup::
    • Azure DevOps: ##[group] / ##[endgroup]
    • GitLab CI: ANSI escape section_start / section_end with [collapsed=true]
    • Local/other: plain --- Title --- delimiters

Motivation

Dashboard CI logs can be very long, making it hard to find the relevant phase (e.g., jumping straight to test failures). Collapsible sections let developers expand only the phase they care about.

References

Test plan

  • Verify collapsible sections appear correctly in GitHub Actions CI logs
  • Verify collapsible sections appear correctly in Azure DevOps CI logs
  • Verify local runs show plain --- Title --- delimiters

🤖 Generated with Claude Code

@github-actions github-actions bot added the type:Enhancement Improvement of existing methods or implementation label Mar 20, 2026
@hjmjohnson hjmjohnson requested a review from thewtex March 20, 2026 00:20
@hjmjohnson
Copy link
Member Author

@thewtex, @blowekamp, @dzenanz I am hoping that this will make navigating the CI infrastructure easier by breaking up the testing/building phases into sub-sections that can be hidden.

I'd love you feedback on this suggestion. I'll be away from the office for the next week, so my responses may be slow.

@blowekamp
Copy link
Member

Is there a sample run to look at someplace?

I get the problem you are trying to solve, with the difficulty with the logs. And using the groups look like a nice approach.

I have wanted to split the ctest steps into senate CI steps. The current driver monolithically does all steps. CTest has Start, Update, Configure, Build, Test, Coverage, MemCheck, and Submit steps, were submit cat be run after many of the steps or in parts. Just having separate configure, build and test steps in the CI would help along too IMHO. This would do a similar thing.

I am not sure parsing the xml in the cmake code is the right things to do. Perhaps using that python script in the dashboard branch, and directly calling it similar to the other PR would better keep components separated.

@hjmjohnson
Copy link
Member Author

I am not sure parsing the xml in the cmake code is the right things to do. Perhaps using that python script in the dashboard branch, and directly calling it similar to the other PR would better keep components separated.

@blowekamp One benefit of parsing the XML is that it clearly list the exact warning or error that causes the build indicate failure AFTER the regular expressions had been filtered out. It should make adding specific regex easier to debug.

I like your idea of moving the python script to the dashboard branch and making it part of the dashboard steps. I'll try that next week after some more testing in this draft mode.

hjmjohnson added a commit to hjmjohnson/ITK that referenced this pull request Mar 20, 2026
Point all CI pipelines at hjmjohnson/ITK dash-board-github-ci-instrument-groupings
branch instead of upstream dashboard to test collapsible CI log sections
and build diagnostics reporting from PR InsightSoftwareConsortium#5973.

This commit must NOT be merged — revert the dashboard clone URL
before merging PR InsightSoftwareConsortium#5971.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@hjmjohnson
Copy link
Member Author

Attempt to use dashboard to review this PR

@hjmjohnson hjmjohnson force-pushed the dash-board-github-ci-instrument-groupings branch from 2aa5545 to 5e6b883 Compare March 21, 2026 23:20
hjmjohnson added a commit to hjmjohnson/ITK that referenced this pull request Mar 22, 2026
Point all CI pipelines at hjmjohnson/ITK dash-board-github-ci-instrument-groupings
branch instead of upstream dashboard to test collapsible CI log sections
and build diagnostics reporting from PR InsightSoftwareConsortium#5973.

This commit must NOT be merged — revert the dashboard clone URL
before merging PR InsightSoftwareConsortium#5971.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@hjmjohnson hjmjohnson marked this pull request as ready for review March 22, 2026 13:21
@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 22, 2026

Greptile Summary

This PR adds ci_section_start() / ci_section_end() helper functions to itk_common.cmake and wraps each dashboard phase (init, diagnostics, configure, build, test, coverage, memcheck, submit) in collapsible CI log sections, targeting GitHub Actions, Azure DevOps, GitLab CI, and local runs. It also adds ci_report_build_diagnostics() to surface compiler warnings and errors directly in the CI log by parsing the CTest-generated Build.xml.

Key observations:

  • The platform detection logic and marker syntax for all three CI systems look correct; \r in CMake double-quoted strings is a proper carriage return, so the GitLab ANSI escape sequences are well-formed.
  • Continuous/loop mode reuses section IDs — when dashboard_loop is non-zero the while loop iterates, emitting the same fixed IDs (configure, build, test, etc.) on each pass. GitLab CI requires section IDs to be unique per job, so repeated IDs will corrupt the log rendering. The fix is to append a per-iteration counter to every section ID.
  • ci_report_build_diagnostics is called after ci_section_end(build), so compiler warnings and errors appear outside the collapsible build group. This may be intentional (to keep errors always visible), but deserves an explanatory comment.
  • ci_section_end emits no output for the local/fallback case, leaving a visual asymmetry with the --- Title --- opener printed by ci_section_start.
  • The %s format specifier passed to string(TIMESTAMP ...) is not supported on Windows before CMake 3.25; if the GitLab CI branch were ever exercised on a Windows runner with an older CMake, the epoch field in the marker would be empty.

Confidence Score: 3/5

  • Safe to merge for single-run dashboards, but continuous-mode dashboards on GitLab CI will produce broken log sections due to duplicate section IDs.
  • The CI helper functions are correct for single-run (non-continuous) pipelines, which is the common case. The duplicate section ID bug only manifests in continuous (dashboard_loop > 0) mode on GitLab CI — a narrower scenario — but it's a real functional defect that should be addressed before merging. The remaining issues are minor style/clarity items.
  • itk_common.cmake — specifically the section ID strings inside the while(NOT dashboard_done) loop

Important Files Changed

Filename Overview
itk_common.cmake Adds CI section helpers and a build-diagnostics reporter; duplicate fixed section IDs in continuous/loop mode will break GitLab CI rendering, and build diagnostics are placed outside the collapsible build section.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A([ctest -S itk_common.cmake]) --> B{COMMAND dashboard_hook_init?}
    B -- yes --> C[ci_section_start: dashboard_init\ndashboard_hook_init\nci_section_end: dashboard_init]
    B -- no --> D

    C --> D[while NOT dashboard_done]
    D --> E[ci_section_start: dashboard_diagnostics\nctest_start / ctest_update\nci_section_end: dashboard_diagnostics]

    E --> F{fresh OR continuous\nOR changes?}
    F -- no --> G{dashboard_loop?}

    F -- yes --> H[ci_section_start: configure\nctest_configure\nci_section_end: configure]
    H --> I[ci_section_start: build\nctest_build\nci_section_end: build]
    I --> J[ci_report_build_diagnostics\nOUTSIDE section]
    J --> K[ci_section_start: test\nctest_test\nci_section_end: test]
    K --> L{dashboard_do_coverage?}
    L -- yes --> M[ci_section_start: coverage\nctest_coverage\nci_section_end: coverage]
    L -- no --> N
    M --> N{dashboard_do_memcheck?}
    N -- yes --> O[ci_section_start: memcheck\nctest_memcheck\nci_section_end: memcheck]
    N -- no --> P
    O --> P[ci_section_start: submit\nctest_submit\nci_section_end: submit]
    P --> G

    G -- yes --> D
    G -- no --> Q([Done])

    style J fill:#f9d,stroke:#c00,color:#000
Loading

Reviews (1): Last reviewed commit: "ENH: Print build warnings and errors fro..." | Re-trigger Greptile

hjmjohnson and others added 2 commits March 22, 2026 08:38
Wrap configure/build/test/coverage/memcheck/submit phases in
collapsible log sections so CI output is easier to navigate.

Detects the CI environment via standard environment variables
and emits the appropriate markers:
- GitHub Actions: ::group::/::endgroup::
- Azure DevOps: ##[group]/##[endgroup]
- GitLab CI: ANSI escape section_start/section_end (collapsed)
- Other/local: plain "--- Title ---" / "--- end id ---" delimiters

Section IDs include a per-iteration counter so that GitLab CI
(which requires unique IDs within a job log) renders correctly
when dashboard_loop runs multiple iterations in continuous mode.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ctest_build() only exposes warning/error counts via NUMBER_WARNINGS
and NUMBER_ERRORS.  The actual compiler messages are written to
Build.xml for CDash submission but never printed to the CI log,
making it hard to see what went wrong without visiting CDash.

Add ci_report_build_diagnostics() which reads Build.xml after the
build step, extracts <Text> content from <Warning> and <Error>
elements, and prints them directly to stdout.

The call is intentionally placed outside the collapsible build
section so that warnings and errors are always visible without
having to expand any section.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@hjmjohnson hjmjohnson force-pushed the dash-board-github-ci-instrument-groupings branch from 5e6b883 to 773bdf8 Compare March 22, 2026 13:43
hjmjohnson added a commit to hjmjohnson/ITK that referenced this pull request Mar 22, 2026
Point all CI pipelines at hjmjohnson/ITK dash-board-github-ci-instrument-groupings
branch instead of upstream dashboard to test collapsible CI log sections
and build diagnostics reporting from PR InsightSoftwareConsortium#5973.

This commit must NOT be merged — revert the dashboard clone URL
before merging PR InsightSoftwareConsortium#5971.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@dzenanz
Copy link
Member

dzenanz commented Mar 23, 2026

Are this PR and #5974 competing or complimentary PRs?

@blowekamp
Copy link
Member

Are this PR and #5974 competing or complimentary PRs?

This PR is for the dashboard branch. The other is a demo of using it.

@dzenanz
Copy link
Member

dzenanz commented Mar 23, 2026

The other branch seems to have some non-trivial changes, that's why I am asking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:Enhancement Improvement of existing methods or implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants