bound utf-8 reader to the terminating nul in extension-functions

[dxbjavid]

out-of-bounds read on a truncated utf-8 sequence in extension-functions

the multibyte reader behind reverse/proper/charindex/leftstr/rightstr/strfilter/difference reads the trailing bytes a lead byte announces without checking for the string's terminating nul, so a value ending in a truncated sequence such as reverse(cast(x'f0' as text)) makes it walk a few bytes past the allocation, a heap over-read reachable from ordinary sql. this stops the read as soon as the next byte is not a continuation byte, so a truncated character resolves to U+FFFD and well-formed utf-8 decodes exactly as before.

[gotson]

The extension functions come from the Sqlite repo, would you happen to know if that issue is also present there, or whether it has been fixed and this fix is the port of theirs ?

[dxbjavid]

good question. this isn't part of the core sqlite amalgamation. it's the contributed extension-functions.c (the LMH one from sqlite.org's contrib page), and the reader here was copied out of sqlite 3.3.13's internal utf8 handling, as the comment just above the tables notes.

core sqlite reworked that reader a long time ago. their version now takes an end pointer and stops on the first non-continuation byte, so the over-read isn't present upstream. the contrib file was never kept in sync, so the old unbounded copy is what still ships here. there's no upstream patch to this file to port, so this is a local fix, but it deliberately follows the same continuation-byte approach current sqlite uses.