Hdr fixes by kode54 · Pull Request #3023 · WayfireWM/wayfire

kode54 · 2026-05-09T01:08:40Z

This branch was:

Co-authored-by: Claude Opus 4.7 (1M context) noreply@anthropic.com

Some of the changes were hand edits or tweaks, such as the gamma 2.2 change to the add_rect color, which required multiple rewrites before I realized what was up with that:

The target blend operation is alpha blended with pre-multiplied alpha, so the color that is coming in needs to have its R/G/B divided by the alpha before adjusting it, then scaled by the color multiplier in linear space, then converted back to sRGB, then pre-multiplied by the alpha again. I opted to use gamma 2.2 instead of piecewise sRGB for the space conversions.
There was a place where the agent decided to try setting a NULL preferred image description, which I discovered is not the legal way to unset the preferred image description. So instead, I opted to write it to set a fallback of sRGB / gamma 2.2, in the case where a "best" format is not found for a surface.
All the rest of the code did what it was supposed to right out of the generator, so it didn't need any major alterations.
Except for Uncrustify. I had to fix up a few files with that, and the line breaking comment wrap found a bug in the Uncrustify implementation that needs to run a second pass to check for discontinuity with equals sign spacing. I'll report that on ammen99's uncrustify repository.

Also store color representation v1 pointer.

For future proofing, in case they become useful to the render setup.

Defaulting to sRGB/gamma22 for non-matching parameters.

The wlroots Vulkan renderer already composites every render pass into an FP16 R16G16B16A16_SFLOAT blend image and applies the inverse-EOTF (or 3D LUT) selected by the pass's color_transform when writing to the output buffer. Allocating our own linear intermediate and running a second render pass to encode it duplicated this work — and actively suppressed wlroots' two-pass by passing EXT_LINEAR as the target transfer function. Render directly to the output buffer with the real output transfer function instead. Per-source luminance bridging in add_texture still runs against that transfer function, so SDR-into-PQ gets the correct 0.0203 multiplier. GLES2 behavior is unchanged — that path was already this code. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Plugin-managed linear-space buffers (Expo's per-workspace aux, the cube's per-workspace framebuffers, the grid crossfade snapshot, view snapshots, and transformer inner_content) had no opinion about their underlying format. With an HDR (PQ) output, the per-source luminance multiplier in render_pass_t::add_texture scales PQ contents up to ~49.26 in the SDR-relative linear domain (PQ peak / SDR reference white) before they land in those buffers. An 8-bit linear backing clipped that to 1.0, dimming HDR highlights. Pass hdr_linear through buffer_allocation_hints_t at each of those sites, gated on the output's committed transfer function being ST2084_PQ. SDR outputs keep their 8-bit allocations. transformer_base_node_t::get_updated_contents gains an optional wf::output_t* parameter so it can make the same decision; the call site in transformer_render_instance_t::get_texture forwards _shown_on. The default nullptr preserves the existing behavior for any out-of-tree callers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kode54 · 2026-05-10T00:28:34Z

Okay, this has an outstanding issue with direct scanout of size mismatched XWayland surfaces, possibly also size mismatched Wayland surfaces, I've committed a fix, will push it once I verify it works correctly.

Direct scanout commits the surface buffer straight to the primary plane via wlr_output_state_set_buffer, which leaves buffer_src_box / buffer_dst_box unset and lets wlroots default both to the buffer's pixel dimensions. On surfaces with buffer_scale != 1 the buffer is larger than the surface's logical size, and the resulting plane configuration mis-displays the buffer on at least some hardware paths — only the top-left logical-sized region reaches the screen, which was visible as alternating bad frames on HDR fullscreen XWayland games whenever the cursor was off the output (the sw-cursor path on the cursor's output forced compositing and masked the issue). Fall back to compositing for these surfaces so the renderer can sample the buffer correctly. SDR surfaces with scale 1 keep the scanout fast path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

ammen99

In general, I am highly doubtful about some of the latest changes. Why skip the zero-copy path if we don't need HDR / special color transfer functions? Maybe the best solution is something like this:

Extend get_texture() to also include a parameter saying which color spaces the caller expects
Make sure get_texture() does not go the zero-copy path if the texture does not match the expected color space
In the GLES2 codepath, expect SRGB or whatever
In the Vulkan codepath, say we support all transfer functions.

ammen99 · 2026-05-10T14:54:36Z

            auto buffer = find_buffer(state.get_context(), total_size);
            buffer->write(unified_buffer.data(), total_size);

-            auto texture  = get_texture(data.target.scale);


If I understand the motivation for the change above correctly, then I think this is not needed. Custom vulkan shaders do take color transfer functions in consideration - if what we have right now is not enough, then we need to extend that support, not force more copies.

ammen99 · 2026-05-10T15:10:53Z

+                // The aux buffer stores a linear-space scene composite (target_tf == EXT_LINEAR).
+                // When the output is HDR (PQ), HDR sources contribute SDR-relative linear values up to
+                // ~49.26 (PQ peak / SDR reference white), which would clip in an 8-bit linear buffer.
+                const auto *img_desc = wall->output->handle->image_description;


We have these checks so often that it would make sense to have a helper function output->render->is_hdr(), same for render_target->is_hdr().

ammen99 · 2026-05-10T15:13:01Z

+     * colors. Cached so that it is not recreated each frame.
+     */
+    wlr_color_transform *output_inverse_eotf = nullptr;
+    wlr_color_transfer_function output_inverse_eotf_tf = (wlr_color_transfer_function)0;


is 0 a valid color transfer function? if not, seems like a candidate for std::optional<wlr_color_transfer_function> instead of invalid values.

ammen99 · 2026-05-10T15:21:08Z

        render_pass_params_t params;
        params.instances = &damage_manager->instance_manager->get_instances();

+        // Render directly to the output target. wlroots' Vulkan renderer already runs a


I am not sure what the purpose of this comment is.

ammen99 · 2026-05-10T15:22:34Z

+    return wf::PQ_MAX_NITS / wf::SDR_REFERENCE_WHITE_NITS;
+}
+
+static float float_max(float a, float b)


This one looks really redundant in c++, where we have std::max.

kode54 · 2026-05-10T23:44:23Z

In general, I am highly doubtful about some of the latest changes. Why skip the zero-copy path if we don't need HDR / special color transfer functions? Maybe the best solution is something like this:

Extend get_texture() to also include a parameter saying which color spaces the caller expects

Make sure get_texture() does not go the zero-copy path if the texture does not match the expected color space

In the GLES2 codepath, expect SRGB or whatever

The altered wlroots code base expects linear in the GLES2 code path as well. And maybe it's worth just getting rid of this branch instead, since I doubt that will ever be merged into wlroots, considering it breaks on Nvidia.

In the Vulkan codepath, say we support all transfer functions.

kode54 · 2026-05-10T23:48:00Z

It's quite possible I should just discard everything after 2443c5f ... since that's mostly adapting everything to the GLES2 rewrite that will likely never be approved or employed anywhere.

wlr_output_state_set_image_description() neither sets allow_reconfiguration nor triggers wlroots' empty-buffer allocation path, so HDR toggles produced atomic commits with no fresh primary buffer and no DRM_MODE_ATOMIC_ALLOW_MODESET flag. amdgpu rejected those, leaving HDR_OUTPUT_METADATA unapplied. Re-pin the current render format and set allow_reconfiguration on the pending state so wlroots attaches a new primary FB and the DRM backend issues a modeset commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The wlroots Vulkan two-pass renderer composites every add_texture into an FP16 blend image with sRGB primaries (it builds a sRGB-target absolute-colorimetric matrix per source primaries in pass.c). Wayfire's output color_transform was a pure linear→inverse-EOTF transform, which the second pass applies with an identity color matrix — leaving the final output buffer with sRGB primaries values that an HDR display configured for Rec.2020 then over-saturates. Build a 2-stage [matrix(sRGB→output_primaries), inverse_eotf] pipeline when the output's committed image description advertises non-sRGB primaries (e.g. BT.2020 on HDR), and fall back to the bare inverse_eotf otherwise. The wlroots Vulkan renderer accepts that pipeline shape via unwrap_color_transform's COLOR_TRANSFORM_PIPELINE branch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kode54 · 2026-05-14T06:21:30Z

Could you please review again, I'm not sure how much of the above comments are regarding code I decided to move elsewhere for future dealings.

Signed-off-by: Christopher Snowhill <kode54@gmail.com>

Plugins computing the FP16-backing hint for linear aux buffers each inlined the same image-description / PQ-transfer check with a different verbose comment. Move the check to a single helper in render.hpp and keep the rationale comment there. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kode54 added 11 commits May 8, 2026 07:42

Implement color management v1

0074cfc

Also store color representation v1 pointer.

Add extra parameters to color_transform_t

616b817

For future proofing, in case they become useful to the render setup.

Set output color transfer function

7f9952e

Reset output transfer function on config change

ac86353

Adjust render transfer functions

46f2c03

Adjust luminance for sRGB/gamma22 brightness

e2e4450

Support surface preferred image description

40ec058

Defaulting to sRGB/gamma22 for non-matching parameters.

Compensate for pre-multiplied alpha and gamma22

9424c78

Linear space color rendering

79e7970

Uncrustify

3a41900

Uncrustify again

3efb789

ammen99 reviewed May 9, 2026

View reviewed changes

Comment thread src/output/render-manager.cpp Outdated

ammen99 reviewed May 9, 2026

View reviewed changes

Comment thread src/api/wayfire/render.hpp

ammen99 requested changes May 10, 2026

View reviewed changes

kode54 force-pushed the hdr-fixes branch from 5328e7e to 2443c5f Compare May 11, 2026 00:33

kode54 and others added 2 commits May 10, 2026 17:41

kode54 and others added 3 commits May 14, 2026 04:38

Remove unwanted comment block

2fd27f1

Signed-off-by: Christopher Snowhill <kode54@gmail.com>

Use std::max instead of custom function

5dcd537

Signed-off-by: Christopher Snowhill <kode54@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hdr fixes#3023

Hdr fixes#3023
kode54 wants to merge 19 commits into
WayfireWM:masterfrom
kode54:hdr-fixes

kode54 commented May 9, 2026

Uh oh!

Uh oh!

Uh oh!

kode54 commented May 10, 2026

Uh oh!

ammen99 left a comment

Uh oh!

ammen99 May 10, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ammen99 May 10, 2026

Uh oh!

ammen99 May 10, 2026

Uh oh!

ammen99 May 10, 2026

Uh oh!

ammen99 May 10, 2026

Uh oh!

kode54 commented May 10, 2026

Uh oh!

kode54 commented May 10, 2026

Uh oh!

kode54 commented May 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

kode54 commented May 9, 2026

Uh oh!

Uh oh!

Uh oh!

kode54 commented May 10, 2026

Uh oh!

ammen99 left a comment

Choose a reason for hiding this comment

Uh oh!

ammen99 May 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ammen99 May 10, 2026

Choose a reason for hiding this comment

Uh oh!

ammen99 May 10, 2026

Choose a reason for hiding this comment

Uh oh!

ammen99 May 10, 2026

Choose a reason for hiding this comment

Uh oh!

ammen99 May 10, 2026

Choose a reason for hiding this comment

Uh oh!

kode54 commented May 10, 2026

Uh oh!

kode54 commented May 10, 2026

Uh oh!

kode54 commented May 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants