Add ANSI_QUOTES and SQL mode validation#452
Draft
JanJakes wants to merge 5 commits into
Draft
Conversation
Contributor
🤖 Lexer benchmarkChanges to lexer-related files were detected and triggered a benchmark:
Note: Hosted runners are noisy, and absolute numbers vary. Treat the results with caution and verify them locally. To reproduce locally: |
Map supported SQL mode names to their native MySQL bit values and use the bitmap as the driver's internal representation. Serialize active modes by bit position so duplicates collapse and @@sql_mode follows MySQL's canonical order.
Validate SQL mode names and numeric masks before changing the session state. Report invalid values with MySQL's SQLSTATE 42000 and error 1231, while retaining MySQL's handling for empty list components and incorrect value types.
When ANSI_QUOTES is active, MySQL treats a double-quoted sequence as a quoted identifier rather than a string literal. Emulate this in the lexer by emitting a backtick-quoted identifier token for double-quoted text when the mode is set, mirroring how NO_BACKSLASH_ESCAPES already alters tokenization. The driver already forwards its active SQL modes to the lexer, so no further wiring is needed. See: https://dev.mysql.com/doc/refman/8.4/en/sql-mode.html#sqlmode_ansi_quotes
MySQL's composite ANSI mode is shorthand for a set of component modes. Expand it when sql_mode is set and store the resulting list, so that "@@sql_mode" and individual mode checks reflect the components: REAL_AS_FLOAT, PIPES_AS_CONCAT, ANSI_QUOTES, IGNORE_SPACE, ONLY_FULL_GROUP_BY REAL_AS_FLOAT and ONLY_FULL_GROUP_BY are stored but not yet respected by the emulation. The driver is the source of truth for SQL modes, but the lexer also recognizes the composite ANSI mode directly so it stays correct when used standalone, applying the components that affect tokenization: PIPES_AS_CONCAT, IGNORE_SPACE, and ANSI_QUOTES. See: https://dev.mysql.com/doc/refman/8.4/en/sql-mode.html#sqlmode_ansi
MySQL does not process backslash escape sequences inside quoted identifiers; the bounding quote is escaped only by doubling it. The lexer was applying string-literal backslash escaping to backtick identifiers (and, with the new ANSI_QUOTES support, to double-quoted identifiers), both when scanning for the closing quote and when unquoting the value. As a result, an identifier like `a\nb` resolved to "a<newline>b", and `a\` (a trailing backslash) failed to tokenize because the backslash was treated as escaping the closing quote. Restrict backslash escaping to string literals so identifiers preserve backslashes verbatim. See: https://dev.mysql.com/doc/refman/8.4/en/identifiers.html
JanJakes
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ANSI_QUOTEShandling to the PHP and native MySQL lexersANSImode while retaining the original modeWhy
The driver previously stored arbitrary SQL mode names, accepted invalid assignments, treated double-quoted text as string literals regardless of SQL mode, did not expand
ANSI, and applied string backslash escaping rules to quoted identifiers. The native lexer also lacked the corresponding behavior.Impact
@@sql_modenow follows MySQL's bitmap order, duplicate modes collapse, invalid assignments fail without changing session state, and mode availability follows the emulated MySQL version. Queries usingANSI_QUOTESorANSIparse double-quoted identifiers correctly in PHP and native configurations, including identifiers containing or ending in backslashes.Testing
composer run check-cscargo fmt --checkandcargo test