Skip to content

fix: no connection character set reads a MySQL string differently - #34

Merged
christiangda merged 1 commit into
mainfrom
fix/a-multibyte-character-does-not-take-the-backslash
Oct 24, 2026
Merged

christiangda merged 1 commit into
mainfrom
fix/a-multibyte-character-does-not-take-the-backslash

Conversation

@christiangda

Copy link
Copy Markdown
Contributor

An independent review of the MySQL dialect (#32) ran the string-reading claim against MySQL 8.4.13, before v1.2.0 was tagged. The tag is held for this PR.

What it found

1. Security: the claim held for utf8mb4 and latin1 connections, and not for gbk, gb18030, sjis, cp932 or big5. In those character sets a backslash (0x5C) can be the last byte of a character that starts with any byte from 0x81. The server reads the two as one character, and the backslash escapes nothing.

Filter, allow-list ["name"] qfv (#32) MySQL, SET NAMES utf8mb4 MySQL, SET NAMES gbk
name = '丁\' OR id = 2 #' accepted, one comparison 0 rows 1 row: id = 2 ran

The same happens with an invalid byte before the backslash, and under gb18030, sjis and cp932.

2. ESCAPE '\%' and ESCAPE '\_': qfv counted each as one character. MySQL keeps them as two outside a pattern, and answers "Incorrect arguments to ESCAPE". The same goes for an escape byte that is not UTF-8.

3. Minor: LiteralNode.Value used strings.Trim, so a string ending on an escaped quote lost it.

What changes

  • The MySQL dialect refuses a backslash right after a character outside ASCII, with why: illegal string: a backslash right after a character outside ASCII, … put a space or another ASCII character between them. In UTF-8, every non-ASCII character ends in a byte from 0x80, and so does every invalid byte. A quote (0x27) is never part of a character in any of these sets, so nothing else can be read differently.
  • ESCAPE:
    • \% and \_ count as two characters, as MySQL counts them;
    • in every dialect, an escape string with a byte that is not UTF-8 is refused. The lexer notes which strings held one: the scanner reads such a byte as U+FFFD, and the input says which it was. Its ErrorCount cannot tell, because it decodes one character ahead.
  • Value takes one quote off each end.
  • Docs: dialects.md has a section on the connection character set and the new refusals (held to Parse by the docs test). The "never looser" line now says where the database's answer depends on the connection. migration.md and the DialectMySQL doc comment are updated too.

How it was checked

  • Differential, against the server: 6,000 filters the dialect accepts, built from quotes, backslashes, comments, and multi-byte and invalid bytes, run through MySQL 8.4.13 under seven connection character sets.

    Character set feat: the MySQL 8.4 dialect #32 as merged this PR
    utf8mb4, latin1, gb18030, cp932, big5 — 0 syntax errors, 0 unknown columns
    gbk 115 syntax errors 0
    sjis 49 syntax errors 0

    What remains under the multi-byte sets is "Illegal mix of collations", and, under latin1 and sjis, an escape like é being two characters: MySQL refusing, after parsing, depending on the connection. Under utf8mb4, nothing remains.

  • Tests:

    • TestNoConnectionCharacterSetReadsAStringDifferently: the review's inputs refused, an ASCII character between allowed, PostgreSQL unaffected;
    • TestTheMySQLEscapeCharacterIsOneCharacterToMySQL;
    • TestAStringsValueKeepsItsLastQuote.
  • Mutants: 7 of 7 caught, each compiling.

    • The multi-byte backslash allowed.
    • The previous character not kept after an escape.
    • \% counted as one.
    • A non-UTF-8 escape allowed.
    • The escaped byte not checked.
    • A real U+FFFD taken for a bad byte.
    • Value trimming every quote.
  • go test -race passes, coverage is 98.3%, and golangci-lint reports 0 issues. All 21 diagrams render after GitHub-style entity decoding.

After this merges, v1.2.0 is tagged.

🤖 Generated with Claude Code

The review of the MySQL dialect found that its string reading held for
utf8mb4 and latin1 connections and not for gbk, gb18030, sjis, cp932 or
big5, where a backslash can be the last byte of a character that starts
with any byte from 0x81: MySQL reads the two as one, and the backslash
escapes nothing. name = '丁\' OR id = 2 #' was one comparison to the
parser, and MySQL 8.4.13 ran id = 2 with SET NAMES gbk. A backslash right
after a character outside ASCII is refused, with why. 6,000 accepted
filters run under all seven character sets: none read differently (before
the rule, 115 under gbk and 49 under sjis).

And what MySQL refuses of an ESCAPE string is refused: \% and \_ are two
characters outside a pattern, and a byte that is not UTF-8 is none (the
scanner reads it as U+FFFD; the input says which it was). A string's Value
takes one quote off each end, where Trim took every one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@christiangda christiangda self-assigned this Oct 24, 2026
@christiangda
christiangda merged commit 1603dac into main Oct 24, 2026
1 check passed
@christiangda
christiangda deleted the fix/a-multibyte-character-does-not-take-the-backslash branch October 24, 2026 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant