Skip to content

fix BitBuffer overflow in rastertohp OutputLine deinterleave#1650

Open
aizu-m wants to merge 1 commit into
OpenPrinting:masterfrom
aizu-m:rastertohp-bitbuffer-bounds
Open

fix BitBuffer overflow in rastertohp OutputLine deinterleave#1650
aizu-m wants to merge 1 commit into
OpenPrinting:masterfrom
aizu-m:rastertohp-bitbuffer-bounds

Conversation

@aizu-m

@aizu-m aizu-m commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

ASan, feeding a crafted application/vnd.cups-raster page to a queue that uses an HP PCL PPD:

ERROR: AddressSanitizer: heap-buffer-overflow WRITE of size 1
  #0 OutputLine rastertohp.c:558
  #1 main rastertohp.c:727
0x... is located 0 bytes after 800-byte region
  #1 StartPage rastertohp.c:338

I was tracing the deep-colour (cupsBitsPerColor > 1) branch of OutputLine. StartPage sizes BitBuffer as ColorBits * (cupsWidth + 7) / 8, but the deinterleave loop runs count = cupsBytesPerLine / NumPlanes times and writes bit_ptr[0] and bit_ptr[bytes]. NumPlanes is 1 for any colourspace that is not CMY or KCMY, so an RGB or CMYK header (its cupsNumColors already folded into cupsBytesPerLine) makes count several times larger than the two bytes-wide sub-planes the buffer holds, and bit_ptr[bytes] runs past the allocation. The page above is 800px, 8-bit, RGB.

Same loop, second way in: count is unsigned and the loop only stops on count > 0 with count -= 2, so an odd cupsBytesPerLine / NumPlanes (e.g. a 12px 2-bit black line where cupsBytesPerLine is 3) underflows to ~2^31 and it keeps writing off the heap.

Both reduce to the loop overrunning the two sub-planes, so I bounded bit_ptr to BitBuffer + bytes. Valid input where count is exactly 2 * bytes reaches the bound just as count hits 0, so nothing changes there; I diffed a 2-bit page before and after and the output is byte-identical.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant