Honor unaligned 16-bit RGB row strides - #3318
Closed
carrerasdarren-cell wants to merge 1 commit into
Closed
Conversation
carrerasdarren-cell
force-pushed
the
fix-unaligned-rgb16-strides
branch
from
August 7, 2026 07:25
a8916a1 to
72e07e3
Compare
Caller-supplied avifRGBImage buffers may use padded byte strides. Use unaligned-safe 16-bit sample access throughout RGB conversion and alpha handling, address F16 rows with byte arithmetic, and fall back from libyuv when its typed buffers would be unaligned. Add regression coverage for unaligned odd-stride conversion in both directions and for high-depth alpha multiply and unmultiply.
carrerasdarren-cell
force-pushed
the
fix-unaligned-rgb16-strides
branch
from
August 7, 2026 07:26
72e07e3 to
433c33d
Compare
Member
|
Thank you for the pull request. Alignment of high bit depth buffers is an API contract. We don't need to accommodate for unaligned buffers or odd strides. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
avifRGBImageallows caller-supplied buffers whoserowBytesis larger than the tight pixel width. For high-depth RGB, an odd padded stride makes successive rows alternate between aligned and unaligned addresses.The scalar RGB and alpha paths currently access these samples through
uint16_t *. UBSan reports misaligned stores inavifFillAlpha()and the 16-bit YUV-to-RGB writers.avifRGBImageToF16()also advances auint16_t *byrowBytes >> 1, so it discards the low stride bit. On untouched main, a valid 9-byte-stride RGBA output changes when only the row-padding byte changes:That can turn caller-owned row padding into visible float output and is undefined behavior on strict-alignment targets.
Change
memcpyhelpers for 16-bit samples in caller-owned RGB buffers while preserving native-endian behavior.The patched sanitizer run reports no alignment errors and the same reproducer becomes:
Verification
avifrgbtest,avifalphapremtest,avifrgbtoyuvtest, andavifrgbtoyuvthreadingtestpass.avifrgbtestpasses, including aligned F16 fast-path output versus the odd-stride scalar fallback.git diff --checkpass.Related work
PR #3312 touches
avifFillAlpha()and same-depthavifReformatAlpha()to add an AArch64 NEON path, but it does not address odd row alignment. If it lands after this change, its 16-bit dispatch and scalar tails will need to retain the unaligned-row behavior fixed here.