Skip to content

Fix integer arithmetic inconsistency in YUV/RGB conversion slow paths#3149

Open
YooLCD wants to merge 3 commits intoAOMediaCodec:mainfrom
YooLCD:main
Open

Fix integer arithmetic inconsistency in YUV/RGB conversion slow paths#3149
YooLCD wants to merge 3 commits intoAOMediaCodec:mainfrom
YooLCD:main

Conversation

@YooLCD
Copy link
Copy Markdown
Contributor

@YooLCD YooLCD commented Mar 30, 2026

In avifImageYUVAnyToRGBAnySlow() and the YUV420/422 write path of avifImageRGBToYUV(), chroma row offsets are computed as uvJ * uRowBytes where both operands are uint32_t. This means the multiplication is done in 32-bit arithmetic before being used as an array index into the UV planes.

The fast paths (avifImageYUV16ToRGB16Color, avifImageYUV8ToRGB8Color, etc.) already guard against this by casting to size_t first:

// fast path — already correct
(uint16_t *)&image->yuvPlanes[AVIF_CHAN_U][((size_t)uvJ * image->yuvRowBytes[AVIF_CHAN_U])]

// slow path — before this patch
&uPlane[(uvJ * uRowBytes) + (uvI * yuvChannelBytes)]

If yuvRowBytes is provided externally (it is a public field on avifImage), the 32-bit product can wrap on large images. This patch adds (size_t) casts to all 22 affected sites in the slow path and the encode write path to match the fast paths.

Also replaces strcpy with memcpy in read.c when copying AVIF_URN_ALPHA0 into alphaAuxProp->u.auxC.auxType. No overflow today (AVIF_URN_ALPHA0 is 44 bytes, AUXTYPE_SIZE is 64), but every other string copy in the file uses memcpy or avifROStreamReadString, so this makes it consistent.

Comment thread src/read.c Outdated
avifProperty * alphaAuxProp = avifMetaCreateProperty(meta, "auxC");
AVIF_CHECKERR(alphaAuxProp, AVIF_RESULT_OUT_OF_MEMORY);
strcpy(alphaAuxProp->u.auxC.auxType, AVIF_URN_ALPHA0);
// Use memcpy instead of strcpy to make the buffer boundary explicit.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Delete this comment.

One problem with this comment is that it describes the change rather than the resulting code.

In general, the change should be described in the commit message of a pull request.

The resulting code, if subtle, can be described in a comment.

Comment thread src/read.c Outdated
AVIF_CHECKERR(alphaAuxProp, AVIF_RESULT_OUT_OF_MEMORY);
strcpy(alphaAuxProp->u.auxC.auxType, AVIF_URN_ALPHA0);
// Use memcpy instead of strcpy to make the buffer boundary explicit.
// AVIF_URN_ALPHA0 is 43 bytes + null terminator, well within AUXTYPE_SIZE (64).
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Delete this comment

libavif recently switched to the ISO C11 standard. This is best valdiated with a static assertion. See pull request #3154.

Comment thread src/read.c Outdated
Comment thread src/reformat.c Outdated
const uint8_t * ptrU8 = uPlane ? &uPlane[(uvJ * uRowBytes)] : NULL;
const uint8_t * ptrV8 = vPlane ? &vPlane[(uvJ * vRowBytes)] : NULL;
const uint8_t * ptrA8 = aPlane ? &aPlane[j * aRowBytes] : NULL;
const uint8_t * ptrY8 = &yPlane[(size_t)j * yRowBytes];
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please undo the changes to the avifImageYUVAnyToRGBAnySlow() function. I have previously committed to reviewing pull request #3063 for this function.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the review! I've reverted the changes to avifImageYUVAnyToRGBAnySlow() and removed the comments.

@YooLCD YooLCD requested a review from wantehchang April 2, 2026 06:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants