fix(revise): a suffix value derives its entries from its own commas (#511) - #512
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #512 +/- ##
=======================================
Coverage 98.66% 98.66%
=======================================
Files 45 45
Lines 3219 3232 +13
=======================================
+ Hits 3176 3189 +13
Misses 43 43 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
A pure move: the pass at the tail of post_rules that tags a SUFFIX token "joined" when it continues the entry before it is now a module-level worker, _mark_suffix_entries, in place over the stage's token list as every other post rule writes, with suffix_entries() the state-in/state-out wrapper. Parser.revise is the wrapper's caller in the next commit (#511): it needs to run the pass over a sub-parse whose roles it has forced, which the tail of a stage cannot offer. Predicate unedited. Why two spellings: a first draft made post_rules call the state wrapper, and the second ParseState build cost three more calls per parse against the band tests/v2/test_benchmark.py holds (py3.11, 2026-09-06: 450 calls/name before, 451 with the worker, 454 with the draft; the facade band tops at 456). AGENTS.md's one-public-function stage rule gains the sanctioned exception in the same commit. Four stage tests exercise the wrapper over forced-role states, which is the shape revise() will hand it, including a forced non-suffix role whose sub-parse had tagged a word on its own. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Parser.revise(name, suffix=value) forced every harvested token to SUFFIX after the sub-parse had already run the entry pass, so a bare 'MD PhD' -- read there as a title and a family name -- came back as two entries and rendered 'MD, PhD'. revise(p, suffix=p.suffix) was not the identity on 38 of the 1117 corpus names (2026-09-06). revise now sub-parses to a state, forces the named role on every non-dropped token, and runs suffix_entries over the forced state, so a comma in the value parts two credentials and a space joins them -- the rule a whole name uses (rules.md#R1). 'Ph. D.' joins by the same rule; the 2026-08-31 acceptance of 'Ph., D.' on this path is superseded, the head-position reasoning behind it untouched. Two limits pinned: a policy-named delimiter inside a value with no comma of its own is a word of the run, the drop that would part it happening only on a segment after a comma; and a glued CJK honorific the whole name kept on an initial peels in the value's own parse. Round-trip: 1117 1 after, 1117 38 before -- the one left is that honorific, '김민준씨, J.씨', right entries and one word read on its own. Gate byte-identical at all four baselines: revise is not on the compare path. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The #511 decision under C1 (R1's history pointer) with the 38 -> 1 measurement, its recipe and the one name left; the phd-merge acceptance of 'Ph., D.' marked superseded with its reasoning left standing; the ONE-PREDICATE-PER-QUESTION instance suffix_entries now is; and the 2.3.0 release bullet, the #436 bullet losing the sentence that recorded this limit as unfixed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
derek73
force-pushed
the
claude/511-revise-entries
branch
from
September 7, 2026 02:05
a51a280 to
9900f05
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #511.
Problem
Parser.revise(name, suffix=value)classified the value by a full sub-parse and then forced every harvested token toSUFFIX— after the sub-parse had already run the #436 entry pass, which keys onRole.SUFFIX. A bare"MD PhD"reads in that sub-parse as a title plus a family name, so the pass joined nothing, and the suffix view comma-joined two unjoined tokens. Feeding a name's own suffix back was not the identity:p.suffixrevise(p, suffix=p.suffix).suffixbeforeJohn Smith MD PhDMD PhDMD, PhDJohn Smith, Ph. D.Ph. D.Ph., D.Doe, John, MD PhD - FACS FellowMD PhD - FACS FellowMD, PhD, -, FACS, FellowMeasured 2026-09-06 over the 1117 distinct names in
tools/differential/corpus*.jsonl(368 carry a suffix): 38 did not round-trip.Fix
Three commits, one concern each, bisectable in order.
post_rulesbecomes_mark_suffix_entries(in place over the stage's token list, as every other post rule writes) plus the wrappersuffix_entries(state). Pure move; the predicate is unedited. A first draft hadpost_rulescall the state wrapper and cost three more calls per parse against the benchmark band (py3.11: 450 → 454 calls/name; the band tops at 455.9), so the worker spelling was chosen (451). Four stage tests exercise the wrapper over forced-role states.revisesub-parses each value to a state, forces the named role on every non-dropped token, runssuffix_entriesover the forced state, assembles and harvests. So a comma in the value parts two credentials and a space joins them: the rule a whole name uses (rules.md#R1, which gains anAccepted:clause and_parser.pyinimplemented:).parse()is untouched; a draft_runhelper shared withparsecost a frame on the hot path and was dropped.Parser.revise(name, suffix=name.suffix)be the identity? (John Smith MD PhDgivesMD PhD, and revising with that string givesMD, PhDback) #511 bullet under C1 (R1's history pointer) with the measurement and recipe, the phd-merge acceptance ofPh., D.marked superseded with its head-position reasoning left standing, the ONE-PREDICATE-PER-QUESTION instance, and the 2.3.0 release bullet.Behavior
revise(n, suffix="MD PhD")→MD PhD(wasMD, PhD);"MD, PhD"stays two entries.revise(n, suffix="Ph. D.")→Ph. D.(wasPh., D., the 2026-08-31 ACCEPTED outcome). The Ph. D. merge is still a head-position rule and still does not fire in a value; the entry pass joins the pair because they share a comma bucket. Superseded, not reversed in reasoning.김민준씨, J.씨: the whole-name parse keepsJ.씨one glued suffix token while the bare value's sub-parse peels the honorific off the initial, so the revised field renders씨, J. 씨(was씨, J., 씨) — right entries, a word read differently on its own. That is the "classified ON ITS OWN" limit the docstring records; CJK honorific peeling is a W-rule question, not moved. Pinned.extra_suffix_delimitersis a word of the run rather than a separator (the drop that parts entries happens only on a segment after a comma). The round-trip is unaffected: the whole-name view already renders that boundary as a comma.FOLDED_TAG, as before. A draft that cleared the sub-parse's own"joined"was measured and backed out: it destroyed within-piece merge marks on non-suffix values, and no view reads the tag off a non-suffix token.Verification
reviseis not on the compare path.pytest: 6584 passed (6576 at master; one test replaced, eight added). mypy and ruff clean. Sphinx build clean.tools/perf/call_count.py --against 330ee55: parse 414 / facade 451 vs 413 / 450.Parser.revise(name, suffix=name.suffix)be the identity? (John Smith MD PhDgivesMD PhD, and revising with that string givesMD, PhDback) #511 issue body).🤖 Generated with Claude Code