Skip to content

Commit 0b8bb71

Browse files
authored
gh-160284: Number each keyword once in the PEG generator (#160290)
`KeywordCollectorVisitor` checked the quoted spelling of a keyword against keys stored without quotes, so the check never matched and every occurrence of a keyword in the grammar took a new token type. The 35 hard keywords were numbered between 500 and 721 instead of 500 to 534.
1 parent 0b676e3 commit 0b8bb71

3 files changed

Lines changed: 257 additions & 247 deletions

File tree

‎Lib/test/test_peg_generator/test_c_generator.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -277,6 +277,15 @@ def test_lowering_snapshots_symbols_and_tokens(self):
277277
self.assertEqual(lowerer.prepare_rule(atom).signature.return_type, "expr_ty")
278278
self.assertEqual(lowerer.prepare_rule(start), expected)
279279

280+
def test_keywords_are_numbered_once(self):
281+
generator = self.make_generator("""
282+
start: keywords | "soft" "soft"
283+
keywords: 'if' NAME | 'if' NUMBER | 'else' NAME | 'pass' | 'else' 'pass'
284+
""")
285+
generator.prepare(PurePath("example.gram"))
286+
self.assertEqual(generator.keywords, {"if": 500, "else": 501, "pass": 502})
287+
self.assertEqual(generator.soft_keywords, {"soft"})
288+
280289

281290
if __name__ == "__main__":
282291
unittest.main()

0 commit comments

Comments
 (0)