Skip to content

fix: auto-detect image compression in StorageMux flashing - #974

Open
mmahut wants to merge 1 commit into
jumpstarter-dev:mainfrom
mmahut:mmahut/detect-compression
Open

fix: auto-detect image compression in StorageMux flashing#974
mmahut wants to merge 1 commit into
jumpstarter-dev:mainfrom
mmahut:mmahut/detect-compression

Conversation

@mmahut

@mmahut mmahut commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Fixes #54

Flashing a compressed image through the StorageMux drivers (SDWire, DUTLink) wrote the compressed bytes to the device unless the user passed --compression xz.

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@mmahut, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 37 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2cb57ed3-79cc-47e7-b568-10bf11437e22

📥 Commits

Reviewing files that changed from the base of the PR and between c18ac85 and 9ea6fad.

📒 Files selected for processing (5)
  • python/packages/jumpstarter-driver-opendal/jumpstarter_driver_opendal/client.py
  • python/packages/jumpstarter-driver-opendal/jumpstarter_driver_opendal/driver.py
  • python/packages/jumpstarter-driver-opendal/jumpstarter_driver_opendal/driver_test.py
  • python/packages/jumpstarter/jumpstarter/common/storage.py
  • python/packages/jumpstarter/jumpstarter/common/storage_test.py
📝 Walkthrough

Walkthrough

StorageMux storage writes now use AutoDecompressIterator to detect and decompress gzip, XZ, bzip2, and zstd images. Tests cover local compressed images and an HTTP-served XZ image. The flashing documentation lists supported formats.

Changes

StorageMux decompression

Layer / File(s) Summary
Decompression write path
python/packages/jumpstarter/jumpstarter/common/storage.py, python/packages/jumpstarter-driver-opendal/jumpstarter_driver_opendal/driver.py
Storage writes pass resources through AutoDecompressIterator before writing them. Uncompressed data remains unchanged.
Compression coverage and API documentation
python/packages/jumpstarter-driver-opendal/jumpstarter_driver_opendal/driver_test.py, python/packages/jumpstarter-driver-opendal/jumpstarter_driver_opendal/client.py
Tests cover gzip, XZ, bzip2, and HTTP-served XZ images. The flash documentation lists supported formats.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
  participant StorageMuxFlasherClient
  participant MockStorageMux
  participant AutoDecompressIterator
  participant StorageFile
  StorageMuxFlasherClient->>MockStorageMux: flash image resource
  MockStorageMux->>AutoDecompressIterator: inspect and iterate resource
  AutoDecompressIterator-->>MockStorageMux: decompressed or unchanged chunks
  MockStorageMux->>StorageFile: write image chunks
Loading

Possibly related PRs

Suggested reviewers: mangelajo

Poem

A rabbit packed images snug and tight,
gzip and XZ now stream just right.
bzip2 hops through the storage door,
HTTP brings XZ to the floor.
The mux writes clean, from start to end.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: automatic image compression detection during StorageMux flashing.
Description check ✅ Passed The description directly explains issue #54 and the failure fixed by this pull request.
Linked Issues check ✅ Passed The changes add automatic decompression for supported image formats and tests for StorageMux flashing, addressing issue #54.
Out of Scope Changes check ✅ Passed The implementation, documentation, and tests are related to automatic compression detection for StorageMux flashing.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@python/packages/jumpstarter-driver-opendal/jumpstarter_driver_opendal/driver_test.py`:
- Around line 183-196: Extend
test_driver_mock_storage_mux_flasher_auto_decompress’s compression parameter
matrix with a zstd-compressed input and matching test ID, using the project’s
existing zstd dependency or helper. Preserve the same flash/dump round-trip
assertion so StorageMuxFlasherClient zstd decompression is covered, then run
make pkg-test-jumpstarter-driver-opendal.
- Around line 230-231: Update the test server cleanup around server and
server_thread so the HTTP server is managed with a context manager or
contextlib.closing, and in finally call server.shutdown() followed by
server_thread.join(). Ensure the listening socket and serving thread are both
closed before the test exits.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c65a073f-c44d-4dbf-b279-5bfd2b12c1bc

📥 Commits

Reviewing files that changed from the base of the PR and between c18ac85 and d75ef71.

📒 Files selected for processing (4)
  • python/packages/jumpstarter-driver-opendal/jumpstarter_driver_opendal/client.py
  • python/packages/jumpstarter-driver-opendal/jumpstarter_driver_opendal/driver.py
  • python/packages/jumpstarter-driver-opendal/jumpstarter_driver_opendal/driver_test.py
  • python/packages/jumpstarter/jumpstarter/common/storage.py

@mmahut
mmahut force-pushed the mmahut/detect-compression branch from d75ef71 to c6add4f Compare August 5, 2026 11:00
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@mmahut
mmahut force-pushed the mmahut/detect-compression branch from c6add4f to 9ea6fad Compare August 5, 2026 11:04
@@ -786,7 +786,11 @@ def flash(
operator: Operator | None = None,
compression: Compression | None = None,
):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not super familiar with compression, but how likely is it, that those magic bytes might appear randomly in some files? If its somewhat likely, we might want to check extensions and magic bytes.

[gzip.compress, lambda data: lzma.compress(data, format=lzma.FORMAT_XZ), bz2.compress, zstd.compress],
ids=["gzip", "xz", "bz2", "zstd"],
)
def test_driver_mock_storage_mux_flasher_auto_decompress(tmp_path, compress):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are no tests for the explicit compression argument override in flash().

Comment on lines 17 to +20

if sys.version_info >= (3, 14):
from compression import zstd
else:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

import here but is not declared as a dev dependency in this package's own pyproject.toml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

StorageMux flashing (SDWire, SDMux, DUTLink) flashing does not auto-detect compression

2 participants