fix(parser): suffix entries follow the commas the writer typed (#436, #437) - #510
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #510 +/- ##
=======================================
Coverage 98.66% 98.66%
=======================================
Files 45 45
Lines 3219 3219
=======================================
Hits 3176 3176
Misses 43 43 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
rules.md#R1 has said it since 2026-08-23 -- a run of post-nominals
written with spaces renders with spaces, one written with commas
keeps them -- and every example under it was a comma path. The
NO-COMMA path violated it at 1.4.0, 2.0.0, 2.1.0 and 2.2.0 alike:
HumanName("John Smith MD PhD").suffix was 'MD, PhD', a comma nobody
typed. #429 fixed the family-comma path and recorded this one as
left alone.
The entry boundary is now read off the text instead of off segment
shape. A post-nominal continues the one before it iff they sit in
the same comma bucket -- comma_bucket, the function segment builds
segments with -- and nothing between them parts the run. group loses
the between-piece marking whole -- `tail` by index, #429's
segment_suffix_reading verdict, the per-piece gate and the
stickiness across an interleaved title -- and keeps core dropping
and the within-piece Ph. D. heal. The stickiness survives by
construction: 'Smith, MD Dr. PhD' has no comma between MD and PhD.
What parts a run is a name word -- GIVEN, MIDDLE or FAMILY -- or a
dropped delimiter core (#206), which is what 'Smith, MD - PhD -
FACS' puts between PhD and FACS under every policy, the half a
dropped-core test cannot reach. What does not part it is a token
whose role renders into a field other than the name and the suffix:
_RENDERS_ELSEWHERE beside the pass names all three of them, TITLE,
NICKNAME and MAIDEN. Nor does a dropped token that is NOT a core,
which is the maiden marker of a bare-marker clause -- dropped with
no role at all, so the role test cannot see it -- and that is why
the dropped arm reads the core set rather than treating every drop
as a boundary. The set is _vocab.delimiter_cores, group's own
derivation off Policy.extra_suffix_delimiters, imported rather than
repeated so the drop site and this pass cannot disagree. Measured
2026-09-06 against f5deae0: 'Smith, MD "Doc" PhD',
'Smith, MD (nee Jones) PhD' and 'Smith, MD nee Jones PhD' each
render 'MD PhD' at the parent and at HEAD, where a title-only
transparency would have regressed all three to 'MD, PhD'; and
'John Smith MD "Doc" PhD' rendered 'MD, PhD' at the parent -- the
bug this commit exists to fix, left unfixed by that draft.
Thirteen pre-existing corpus names move, all from a comma-joined
render to a space-joined one, none the other way, none with a
comma in the original between the two words. Nine cases.py rows
are re-pinned; 'MD PhD Jr., John' keeps its name and its point,
segment 0 being the family segment, and only its separator moves.
Eight assertions outside that table are re-pinned too: four in the
v1-parity suite (tests/test_suffixes.py x3 and
tests/test_capitalization.py) and four in the v2 API suite
(tests/v2/test_parser.py). v1 inserted a comma into a run the
writer had spaced, and that is a deliberate deviation recorded in
the release log by this bundle's records commit, not a parity
break. Four rules.md example lines outside R1 (P2's 'John Smith Mc
V', P5's 'abdul Smith Jr Ma', C1's 'Smith, John PhD I.', W3's
'田中さん 様.') carried the same comma and are corrected; all four are
corpus names, so the differential observed those edits
independently -- '田中さん 様.' sits in corpus_cjk_tolerated.jsonl and
carries its own ledger rule. C1's closing qualifier and W3's
tolerated: line were read and left verbatim: both describe a
reading, not a separator. Every *_list view other than suffix_list
is byte-identical over the corpus names, measured before and
after.
Round-tripping is stable on the shapes #429 fixed: str() of a fixed
parse is a no-comma string that used to re-parse with the comma
back.
Gate, with the new R1 example in the rules corpus: 360 / 254 / 166 /
28 intentional at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0, 0 unexplained, 0
radar unclassified. One name changes rule -- 'abdul Smith Jr V' at
2.0.0 and 2.1.0, whose diff grows past fix(#401)'s fields and lands
on a compound rule naming both mechanisms.
Refs #436, refs #437
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A dated bullet under decisions.md#C1, where #429 and #430/#432 sit and where rules.md#R1's history: already points: the rule as it shipped -- same comma bucket, and nothing between the two post-nominals but a token that renders into another field, with a dropped index parting the run only when it is a delimiter core -- the measured movers with their recipes, the nine cases.py re-pins, what group lost and why the interleaved-title stickiness needs no code, the round-trip claim with its test and its limit, and the four Declined shapes. It records the FIND that R1 stated the rule in words on 2026-08-23 while the no-comma path violated it, and the two places the design was amended by measurement. The second amendment is the second FIND and is recorded as one. The spec's predicate parted only at a dropped core, and the two delimiter-core corpus names drop none under the default policy; then the first draft of this bundle's code commit made TITLE the only transparent role and every dropped index a boundary, which regressed 'Smith, MD "Doc" PhD', 'Smith, MD (nee Jones) PhD' and 'Smith, MD nee Jones PhD' from 'MD PhD' to 'MD, PhD' and left 'John Smith MD "Doc" PhD' unfixed -- measured 2026-09-06 against the parent tree, witnessed by no corpus name, so the gate was green over it and the code review is what caught it. Sites the first commit made false, found by its comment-accuracy review and fixed here with dated clauses: #429's "left alone" paragraph (all three of its open clauses are spent -- filed, fixed, and round-tripping pinned), its present-tense description of the between-piece marking, and its "the predicate moved down into group" sentence; #430/#432's "THREE readers" paragraph, which says two remain; the Declined note that had the "joined" tag applied by grouping alone. mechanisms.md#ONE-PREDICATE-PER-QUESTION moves segment_suffix_reading out of the called-by-both list and adds delimiter_cores, the new instance, to the vocabulary layer's; the predicate itself is not moved. The #498 bullet's 'Smith, PhD Jr.' quotation and the matching _CROSS_RULE_WINNERS row comment both said the ledger discrepancy was unrepaired. Half of it is repaired: Task 1 corrected 'PhD, Jr.' to 'PhD Jr.' in the three ledgers carrying the comment, stale since #429. The "title 'PhD' at every baseline" half stands and is still live, re-measured here, and both sites now say which half is which. One release-log bullet in the 2.3.0 block, above the 2.2.0 #429 bullet so that one reads as history: the before/after, the unchanged comma forms, the deliberate v1 deviation, and the round trip fixed for these runs with the one name that goes the other way. Gate byte-identical to the previous commit -- nothing here is an input the gate reads. Closes #436, closes #437. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
derek73
force-pushed
the
claude/436-suffix-entries
branch
from
September 6, 2026 20:43
f22fa4f to
9e3834b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The suffix view now separates post-nominals the way the writer did. Two consecutive suffix tokens render as one space-joined entry iff they sit in the same comma-delimited part of the name and nothing between them belongs to the name or separates entries; otherwise a comma.
HumanName("John Smith MD PhD").suffixisMD PhD, as it has been forJohn Smith, MD PhDsince 1.4.0 and forSmith, MD PhDsince #429. Closes #436, closes #437.What was wrong
rules.md#R1 already said it: "a run of post-nominals written with spaces renders with spaces, and one written with commas keeps them." Every example under it was a comma path, and the no-comma path violated the sentence at every release since 1.4.0. #429 fixed the family-comma path and recorded this one as "left alone"; it is also why round-tripping was unstable on the shapes #429 fixed, since
str()of a fixed parse is a no-comma string.The cause was where the entry boundary was decided: in
group, from segment SHAPE —tailby index, ORed since #429 with a content verdict, gated per piece, sticky across an interleaved title, closed by a delimiter core. Three shape-derived answers to one question the input answers directly.The change
A pass at the end of
post_rules, after roles are assigned, walks the SUFFIX tokens in string order and tags a continuationjoinediff the pair shares a comma bucket (comma_bucket, the functionsegmentbuilds segments with) and nothing parts them. Transparent: a token that renders into another field — title, nickname, maiden — and a dropped maiden marker. Parting: a surviving name word, or a dropped delimiter core, read through the samedelimiter_cores()helpergroupuses to drop it.grouploses its between-piece marking whole and keeps core dropping and the within-piecePh. D.heal. The tag stays the recorded form, so pickled names keep working.Measured over the 1116 corpus names: 13 move, every one from a comma the writer never typed to the spacing they wrote; no name moves the other way; every list view other than
suffix_listis byte-identical. Ninecases.pyrows, four rules.md examples and eight older assertions carried the artifact and are re-pinned with notes. Round-tripping is stable on the shapes #429 fixed and a test pins those three; over the corpora it is a rendering, not a canonical form — 35 names failed to round-trip at the parent and 23 do now, 13 fixed and one newly visible,Smith, John PhD I., whose render used to hide a parse disagreement behind the inserted comma. The decisions entry carries the measurement.What review caught
The first draft of the code commit made only a title transparent. Measured against the parent tree, that regressed
Smith, MD "Doc" PhDand both spellings ofSmith, MD nee Jones PhDfromMD PhDtoMD, PhDand leftJohn Smith MD "Doc" PhDunfixed. Four stage tests and a case row now pin the transparent set, and the decisions entry records the miss. A four-reviewer pass over the PR then added a mixed-comma three-suffix test (the one mutant that had survived, measuring the comma bucket against the run's first suffix), pinned that a marker the policy itself names as a delimiter parts the run, and corrected five figures and sentences in the records. One limit is recorded rather than fixed:Parser.revise(n, suffix=n.suffix)is not the identity on a space-joined run, since the bare string has no comma to route by; pre-existing, and reachable on more inputs now (24 of the parent's corpus names, 38 here).Records
R1 gains its no-comma example and a sentence naming the mechanism. Six ledger rules at fourteen placements, each with its claim roster; the differential gate reads 360 / 254 / 166 / 28 intentional at the four baselines, up from 352 / 247 / 155 / 14, with 0 unexplained and exactly one name changing rule (
abdul Smith Jr Vat 2.x, whose diff grew pastfix(#401)'s fields). A dated bullet under C1 in decisions.md carries the decision, the figures, the four declined shapes and the review finding; the #429 and #430 entries are amended where this made them history; a 2.3.0 release-log bullet records the deviation from 1.4.0.Verification
Full suite green, ruff and mypy clean, gate exit 0 at four baselines with no finding block. Mutation controls: dropping the comma-bucket conjunct joins
Smith Jr., Mr. Jr.across the writer's comma; dropping the parting conjunct joinsPhD FACSacross a delimiter; running the pass beforeassigncollapsesJohn A B Smith's middle names. Each is killed by a named test.🤖 Generated with Claude Code