Skip to content

Add lossless JPEG compression for raw denoise DNG output#21448

Draft
andriiryzhkov wants to merge 2 commits into
darktable-org:masterfrom
andriiryzhkov:nr_dng_compression
Draft

Add lossless JPEG compression for raw denoise DNG output#21448
andriiryzhkov wants to merge 2 commits into
darktable-org:masterfrom
andriiryzhkov:nr_dng_compression

Conversation

@andriiryzhkov

Copy link
Copy Markdown
Collaborator

Sibling to #21435 (TIFF compression for denoise/upscale). That PR handles the TIFF outputs; this one does the equivalent for raw denoise, which writes DNG.

The naïve fix – COMPRESSION_JPEG + tile writes – doesn't work: libtiff's JPEG codec hard-rejects 16-bit. So I encode each 256×256 tile with libjpeg-turbo (which does support 16-bit lossless), tell libtiff COMPRESSION_ADOBE_DEFLATE so it accepts the setup, hand it the JPEG bytes via TIFFWriteRawTile() (libtiff stores opaque bytes without invoking its codec), then post-patch the Compression tag from 8 → 7 in the closed file. The on-disk data is genuine lossless JPEG; the deflate codec is never actually invoked. Documented in detail at the top of src/imageio/imageio_dng.c.

A new compression combo in the raw-denoise output panel (uncompressed / lossless JPEG, default lossless) mirrors the TIFF combo from #21435.

File size on a noisy ISO 6400 Canon R7: ~50 MB → ~41 MB. Cleaner sources compress better.

@andriiryzhkov andriiryzhkov added the scope: AI features AI features related issues and PR label Jun 29, 2026
@andriiryzhkov andriiryzhkov marked this pull request as draft June 29, 2026 09:33
@andriiryzhkov

Copy link
Copy Markdown
Collaborator Author

Moving to draft — the implementation hits a Linux distribution problem I didn't catch before.

jpeg_enable_lossless() requires libjpeg-turbo 3.0+ (Oct 2023). The AppImage builds on Ubuntu LTS, which ships libjpeg-turbo 2.x. So as-is, every Linux user via AppImage would get a build failure or "no compression" fallback.

Path options:

  1. Feature-detect + fallback to uncompressed — Linux AppImage users get no compression. Defeats the point.
  2. Switch to DEFLATE compression — rejected: rawspeed enforces the DNG spec strictly and refuses to read integer DNG with DEFLATE (DngDecoder.cpp:362), so the files wouldn't open in darktable itself.
  3. Vendor liblj92 — small standalone lossless JPEG encoder (used by Magic Lantern). True lossless JPEG everywhere, spec-compliant, ~1500 lines in src/external/. Even though this looks like the best option, it's a lot of effort compared to the size of the issue it solves.

Parking the PR until I have a clearer answer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope: AI features AI features related issues and PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant