Repository navigation
fix(parser): stop the lexers from hanging on a NUL character - #661
Conversation
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Angular's cursor peeks a NUL as `$EOF`, but `advance()` still moves past it.
So only the loops that check `peek() === $EOF` stop there: the main loop,
text, interpolation, tag and attribute names, unquoted values, and block
parameters outside quotes. Comments, CDATA, DOCTYPE, raw text (`<style>`,
`<textarea>`, `<title>`, `<script>`), quoted attribute values, `[...]`
attribute names and ICU values read a NUL as text.
Cutting the input at the first NUL broke the second group. For example,
`<!--a\0b--><p>{{x}}</p>` gave an error, and `<textarea>a\0b</textarea>`
silently dropped the rest of the template, while ngtsc compiles both.
- HTML lexer: stop the main loop and interpolations at a NUL. The scanners
that read to a closing marker now check the index, not the character.
- Interpolation: read the character after `//` before checking again, as
Angular does (`{{ a //}}b` is one interpolation).
- Expression lexer: skip whitespace the same way in `${...}` as in
`scanToken` (every character up to the space, and NBSP), so `${a\0}`
compiles like it does in Angular.
- Tests: check the tokens and that there are no errors for each case.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@codex review |
|
Codex Review: Something went wrong. Try again later by commenting “@codex review”. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review |
|
Codex Review: Something went wrong. Try again later by commenting “@codex review”. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
A NUL character in a template or a binding made the compiler loop forever.
Both lexers use
'\0'as their end-of-input sentinel. In the HTML lexer the scanners stop at a NUL without moving, so the main loop (while index < length) never ends. In the expression lexeradvance()refused to move past a character equal toEOF, so a host binding such as'[x]': 'a\0b'hung in the same way.What Angular does
Angular's cursor peeks a NUL as
$EOF(0), butadvance()still moves past it (it only checks the offset). So a NUL stops only the loops that checkpeek() === $EOF:<style>,<textarea>,<title>,<script>[...]attribute names, ICU valuesChanges
<b>{{a}}</b>\0<i>{{b}}</i>keeps only<b>,<!--a\0b--><p>{{x}}</p>keeps the<p>, and<textarea>a\0b</textarea>keepsa\0b.//before checking again, as Angular does. So{{ a //}}bis one interpolation, and{{ a //\0 }}is not cut at the NUL.advance()checks the index, not the character. Whitespace is skipped the same way inscanTokenand inside${...}: every character up to the space (NUL and the other control characters), plus NBSP. A NUL inside a string is still an unterminated string.Checks
${a\0}. All of them fail on the first version of this PR, and the first two hang before it.tokenize()for 29 templates, with a NUL put at every position (743 cases). The only differences left also show up without a NUL (see below). The same check for the expression lexer found no NUL-only differences.@ifdrops expression errors, and there is no DOM schema check (NG8002).cargo test --all-features(3760 tests), no-features (3618), conformance 1264/1264, napi vitest 401/401, and clippy on the changed lines all pass.Not in this PR (also happen without a NUL)
<style>a</stylex>) gets no error. Angular reportsUnexpected character "EOF".<!x>is an error for us, but a DOCTYPE token for Angular.🤖 Generated with Claude Code