Skip to content

Implementation ci dataset#1474

Draft
marco66colombo wants to merge 27 commits into
fastmachinelearning:mainfrom
marco66colombo:implementation-ci-dataset
Draft

Implementation ci dataset#1474
marco66colombo wants to merge 27 commits into
fastmachinelearning:mainfrom
marco66colombo:implementation-ci-dataset

Conversation

@marco66colombo
Copy link
Copy Markdown
Contributor

Description

This PR adds a dedicated GitLab CI path for collecting full implementation-flow artifacts from hls4ml models.

The first supported target is VivadoAccelerator on zcu102, using small Keras models from the example-models submodule. The implementation jobs run outside the regular pytest matrix and execute the complete backend flow: HLS synthesis, cosimulation, Vivado synthesis, implementation, bitstream generation, report parsing, and artifact collection.

The goal is to make it possible to run selected end-to-end implementation checks periodically or on demand, and to preserve structured data that can be compared across hls4ml releases, backend/tool versions, boards, and model families. This should help track changes in resource usage, timing, generated reports, and other implementation metrics over time.

This is intended as the first step of an extensible implementation dataset CI suite. The helper and CI layout are written so that additional backends can be added with backend-specific templates and static job entries. Follow-up work is already planned/in development for VitisUnified, with possible future extensions to Bambu and other supported backends.

PR #1451 was propedeutic for this work, since it tested the runtime environment for these workflows.

Main changes:

  • Add CI_PIPELINE_MODE=implementation as a separate GitLab pipeline mode.
  • Keep implementation jobs out of the default pytest CI generator.
  • Add implementation-specific CI templates and a static implementation job list.
  • Add Vivado/VivadoAccelerator runtime setup for the implementation jobs.
  • Add implementation dataset helpers that collect:
    • parsed hls4ml reports
    • build logs
    • dataset JSON records
    • generated bitstream metadata
    • zipped generated project directories with SHA256 checksums
  • Add two initial VivadoAccelerator implementation tests using example-models Keras models.
  • Add documentation for adding and running implementation CI jobs.

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • Documentation update
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • A new research paper code implementation
  • CI improvements: add implementation test coverage

Tests

This CI path is intended to run in GitLab, because it depends on the CERN CI environment, CVMFS tool installations, and Vivado runtime.

To run the implementation CI, start a GitLab pipeline on this branch with:

CI_PIPELINE_MODE=implementation

Expected behavior:

  • the regular pytest generator and normal pytest pipeline are skipped
  • the implementation-pytests trigger job runs
  • the implementation child pipeline is created from:
    • test/pytest/implementation/ci-template.yml
    • test/pytest/implementation/pytests.yml
  • the current implementation jobs run as separate jobs:
    • pytest.implementation.keras_1layer_vivadoacc
    • pytest.implementation.keras_conv1d_small_vivadoacc

Expected artifacts from each successful job include:

  • *_dataset.json
  • *_hls4ml_report.json
  • *_build.log
  • *_project.zip

Checklist

  • I have read the guidelines for contributing.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • My changes generate no new warnings.
  • I have installed and run pre-commit on the files I edited or added.
  • I have added tests that prove my fix is effective or that my feature works.

@marco66colombo marco66colombo self-assigned this May 14, 2026
@marco66colombo marco66colombo requested a review from nghielme May 14, 2026 19:13
@marco66colombo marco66colombo added the please test Trigger testing by creating local PR branch label May 15, 2026
@marco66colombo marco66colombo requested a review from vloncar May 20, 2026 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

please test Trigger testing by creating local PR branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant