Skip to content

Fix byte/character offset mixup in computed constants output - #148

Draft
rootkiller6788 wants to merge 1 commit into
google:mainfrom
rootkiller6788:fix-computed-constants-utf16-offsets
Draft

Fix byte/character offset mixup in computed constants output#148
rootkiller6788 wants to merge 1 commit into
google:mainfrom
rootkiller6788:fix-computed-constants-utf16-offsets

Conversation

@rootkiller6788

Copy link
Copy Markdown

Problem

When jsir_gen runs dynamic constant propagation on a script containing non-ASCII characters (e.g. Cyrillic text in a comment), the Computed constants section reports the wrong source segment.

Fixes #22.

Root cause

DumpJsirAnalysisResult in maldoca/js/ir/jsir_gen_lib.cc formats each computed constant with From [start, end): ... by calling original_source.substr(start_offset, end_offset - start_offset). However:

  • original_source is the raw UTF-8 byte string.
  • start_offset / end_offset are UTF-16 code-unit offsets. They come from JsirTriviaAttr locations, which are populated from Babel/ESTree positions (node->start() / node->end() in ast_to_jsir.handwritten.cc). ESTree source positions are UTF-16 code units.

Slicing a byte string with code-unit offsets diverges as soon as any non-ASCII character appears before the constant, producing the misaligned output described in the issue.

Fix

Convert the source to UTF-16 once with the existing Utf8ToUtf16 helper, slice it using the UTF-16 offsets, and convert the selected segment back to UTF-8 with Utf16ToUtf8 for output. Adds the :utf16 dependency to the jsir_gen_lib BUILD target.

Testing

Manual verification of the logic: offsets recorded by the analysis are UTF-16 code-unit offsets (confirmed against pass.cc and the Babel position provenance), and the fix applies the existing, already-used Utf8ToUtf16 / Utf16ToUtf8 round-trip so byte offsets and code-unit offsets no longer get conflated.

The dynamic constant propagation analysis records source locations as
UTF-16 code-unit offsets (Babel/ESTree positions), but
DumpJsirAnalysisResult sliced the original UTF-8 source with those
offsets directly. For sources containing non-ASCII characters this
misreported the source segment shown next to each computed constant.

Convert the source to UTF-16 before slicing and convert the selected
segment back to UTF-8 for output, reusing the existing Utf8ToUtf16 /
Utf16ToUtf8 helpers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Computed constants output misrepresents source locations if non-ASCII characters are present in the source

1 participant