add fill_tape_padded to skip the internal input copy - #472
Open
ranflarion wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First half of #469: the padded entry point, without the UTF-8 validation skip (independent, can follow separately).
fill_tape_padded(s, len, buffers, tape)plusINPUT_PADDING. Callers that control the input layout supply the SIMD over-read padding themselves, skipping the O(len) copyfill_tapemakes into its internal buffer: +2% to +7% on the shapes measured in #469, growing with document size.With that copy gone stage 2 reads the same buffer string unescaping writes into, so the read side can no longer be a
&[u8]argument (argument protectors under both borrow models, andnoalias readonly). It becomesInputView, apub(crate)pointer+len threaded throughbuild_tapeand the per-ISAparse_str, withbuild_tapetaking*mut u8for the write side.fill_tapestill passes a disjoint copy, so that path is unchanged.One correction to the issue: since 011d699 dropped NUL from the structural-or-whitespace tables the padding is not arbitrary,
s[len]has to be structural or whitespace (b' 'works). Documented, androot_scalars_terminated_by_paddingcovers it.tests/fill_tape_padded.rspins the result againstfill_tapenode-for-node over lane-boundary escape positions, escapes landing against the padding, shared multi-row scratches, error paths and a random corpus. Miri-clean under Stacked and Tree Borrows on both aarch64 and x86_64, which may be useful for #446/#450; full suite, feature matrix, clippy and fmt green.Happy to rename things or keep
InputViewout ofparse_str's signature if you'd prefer.