Skip to content

fix(parser): suffix entries follow the commas the writer typed (#436, #437) - #510

Merged
derek73 merged 2 commits into
masterfrom
claude/436-suffix-entries
Sep 6, 2026
Merged

fix(parser): suffix entries follow the commas the writer typed (#436, #437)#510
derek73 merged 2 commits into
masterfrom
claude/436-suffix-entries

Conversation

@derek73

@derek73 derek73 commented Sep 6, 2026

Copy link
Copy Markdown
Owner

The suffix view now separates post-nominals the way the writer did. Two consecutive suffix tokens render as one space-joined entry iff they sit in the same comma-delimited part of the name and nothing between them belongs to the name or separates entries; otherwise a comma. HumanName("John Smith MD PhD").suffix is MD PhD, as it has been for John Smith, MD PhD since 1.4.0 and for Smith, MD PhD since #429. Closes #436, closes #437.

What was wrong

rules.md#R1 already said it: "a run of post-nominals written with spaces renders with spaces, and one written with commas keeps them." Every example under it was a comma path, and the no-comma path violated the sentence at every release since 1.4.0. #429 fixed the family-comma path and recorded this one as "left alone"; it is also why round-tripping was unstable on the shapes #429 fixed, since str() of a fixed parse is a no-comma string.

The cause was where the entry boundary was decided: in group, from segment SHAPE — tail by index, ORed since #429 with a content verdict, gated per piece, sticky across an interleaved title, closed by a delimiter core. Three shape-derived answers to one question the input answers directly.

The change

A pass at the end of post_rules, after roles are assigned, walks the SUFFIX tokens in string order and tags a continuation joined iff the pair shares a comma bucket (comma_bucket, the function segment builds segments with) and nothing parts them. Transparent: a token that renders into another field — title, nickname, maiden — and a dropped maiden marker. Parting: a surviving name word, or a dropped delimiter core, read through the same delimiter_cores() helper group uses to drop it. group loses its between-piece marking whole and keeps core dropping and the within-piece Ph. D. heal. The tag stays the recorded form, so pickled names keep working.

Measured over the 1116 corpus names: 13 move, every one from a comma the writer never typed to the spacing they wrote; no name moves the other way; every list view other than suffix_list is byte-identical. Nine cases.py rows, four rules.md examples and eight older assertions carried the artifact and are re-pinned with notes. Round-tripping is stable on the shapes #429 fixed and a test pins those three; over the corpora it is a rendering, not a canonical form — 35 names failed to round-trip at the parent and 23 do now, 13 fixed and one newly visible, Smith, John PhD I., whose render used to hide a parse disagreement behind the inserted comma. The decisions entry carries the measurement.

What review caught

The first draft of the code commit made only a title transparent. Measured against the parent tree, that regressed Smith, MD "Doc" PhD and both spellings of Smith, MD nee Jones PhD from MD PhD to MD, PhD and left John Smith MD "Doc" PhD unfixed. Four stage tests and a case row now pin the transparent set, and the decisions entry records the miss. A four-reviewer pass over the PR then added a mixed-comma three-suffix test (the one mutant that had survived, measuring the comma bucket against the run's first suffix), pinned that a marker the policy itself names as a delimiter parts the run, and corrected five figures and sentences in the records. One limit is recorded rather than fixed: Parser.revise(n, suffix=n.suffix) is not the identity on a space-joined run, since the bare string has no comma to route by; pre-existing, and reachable on more inputs now (24 of the parent's corpus names, 38 here).

Records

R1 gains its no-comma example and a sentence naming the mechanism. Six ledger rules at fourteen placements, each with its claim roster; the differential gate reads 360 / 254 / 166 / 28 intentional at the four baselines, up from 352 / 247 / 155 / 14, with 0 unexplained and exactly one name changing rule (abdul Smith Jr V at 2.x, whose diff grew past fix(#401)'s fields). A dated bullet under C1 in decisions.md carries the decision, the figures, the four declined shapes and the review finding; the #429 and #430 entries are amended where this made them history; a 2.3.0 release-log bullet records the deviation from 1.4.0.

Verification

Full suite green, ruff and mypy clean, gate exit 0 at four baselines with no finding block. Mutation controls: dropping the comma-bucket conjunct joins Smith Jr., Mr. Jr. across the writer's comma; dropping the parting conjunct joins PhD FACS across a delimiter; running the pass before assign collapses John A B Smith's middle names. Each is killed by a named test.

🤖 Generated with Claude Code

@derek73 derek73 added bug docs Documentation fixes and updates tooling tests labels Sep 6, 2026
@derek73 derek73 self-assigned this Sep 6, 2026
@codecov

codecov Bot commented Sep 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.66%. Comparing base (f5deae0) to head (9e3834b).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #510   +/-   ##
=======================================
  Coverage   98.66%   98.66%           
=======================================
  Files          45       45           
  Lines        3219     3219           
=======================================
  Hits         3176     3176           
  Misses         43       43           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

derek73 and others added 2 commits September 6, 2026 13:41
rules.md#R1 has said it since 2026-08-23 -- a run of post-nominals
written with spaces renders with spaces, one written with commas
keeps them -- and every example under it was a comma path. The
NO-COMMA path violated it at 1.4.0, 2.0.0, 2.1.0 and 2.2.0 alike:
HumanName("John Smith MD PhD").suffix was 'MD, PhD', a comma nobody
typed. #429 fixed the family-comma path and recorded this one as
left alone.

The entry boundary is now read off the text instead of off segment
shape. A post-nominal continues the one before it iff they sit in
the same comma bucket -- comma_bucket, the function segment builds
segments with -- and nothing between them parts the run. group loses
the between-piece marking whole -- `tail` by index, #429's
segment_suffix_reading verdict, the per-piece gate and the
stickiness across an interleaved title -- and keeps core dropping
and the within-piece Ph. D. heal. The stickiness survives by
construction: 'Smith, MD Dr. PhD' has no comma between MD and PhD.

What parts a run is a name word -- GIVEN, MIDDLE or FAMILY -- or a
dropped delimiter core (#206), which is what 'Smith, MD - PhD -
FACS' puts between PhD and FACS under every policy, the half a
dropped-core test cannot reach. What does not part it is a token
whose role renders into a field other than the name and the suffix:
_RENDERS_ELSEWHERE beside the pass names all three of them, TITLE,
NICKNAME and MAIDEN. Nor does a dropped token that is NOT a core,
which is the maiden marker of a bare-marker clause -- dropped with
no role at all, so the role test cannot see it -- and that is why
the dropped arm reads the core set rather than treating every drop
as a boundary. The set is _vocab.delimiter_cores, group's own
derivation off Policy.extra_suffix_delimiters, imported rather than
repeated so the drop site and this pass cannot disagree. Measured
2026-09-06 against f5deae0: 'Smith, MD "Doc" PhD',
'Smith, MD (nee Jones) PhD' and 'Smith, MD nee Jones PhD' each
render 'MD PhD' at the parent and at HEAD, where a title-only
transparency would have regressed all three to 'MD, PhD'; and
'John Smith MD "Doc" PhD' rendered 'MD, PhD' at the parent -- the
bug this commit exists to fix, left unfixed by that draft.

Thirteen pre-existing corpus names move, all from a comma-joined
render to a space-joined one, none the other way, none with a
comma in the original between the two words. Nine cases.py rows
are re-pinned; 'MD PhD Jr., John' keeps its name and its point,
segment 0 being the family segment, and only its separator moves.
Eight assertions outside that table are re-pinned too: four in the
v1-parity suite (tests/test_suffixes.py x3 and
tests/test_capitalization.py) and four in the v2 API suite
(tests/v2/test_parser.py). v1 inserted a comma into a run the
writer had spaced, and that is a deliberate deviation recorded in
the release log by this bundle's records commit, not a parity
break. Four rules.md example lines outside R1 (P2's 'John Smith Mc
V', P5's 'abdul Smith Jr Ma', C1's 'Smith, John PhD I.', W3's
'田中さん 様.') carried the same comma and are corrected; all four are
corpus names, so the differential observed those edits
independently -- '田中さん 様.' sits in corpus_cjk_tolerated.jsonl and
carries its own ledger rule. C1's closing qualifier and W3's
tolerated: line were read and left verbatim: both describe a
reading, not a separator. Every *_list view other than suffix_list
is byte-identical over the corpus names, measured before and
after.

Round-tripping is stable on the shapes #429 fixed: str() of a fixed
parse is a no-comma string that used to re-parse with the comma
back.

Gate, with the new R1 example in the rules corpus: 360 / 254 / 166 /
28 intentional at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0, 0 unexplained, 0
radar unclassified. One name changes rule -- 'abdul Smith Jr V' at
2.0.0 and 2.1.0, whose diff grows past fix(#401)'s fields and lands
on a compound rule naming both mechanisms.

Refs #436, refs #437

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A dated bullet under decisions.md#C1, where #429 and #430/#432 sit
and where rules.md#R1's history: already points: the rule as it
shipped -- same comma bucket, and nothing between the two
post-nominals but a token that renders into another field, with a
dropped index parting the run only when it is a delimiter core --
the measured movers with their recipes, the nine cases.py re-pins,
what group lost and why the interleaved-title stickiness needs no
code, the round-trip claim with its test and its limit, and the
four Declined shapes. It records the FIND that R1 stated the rule
in words on 2026-08-23 while the no-comma path violated it, and
the two places the design was amended by measurement.

The second amendment is the second FIND and is recorded as one.
The spec's predicate parted only at a dropped core, and the two
delimiter-core corpus names drop none under the default policy;
then the first draft of this bundle's code commit made TITLE the
only transparent role and every dropped index a boundary, which
regressed 'Smith, MD "Doc" PhD', 'Smith, MD (nee Jones) PhD' and
'Smith, MD nee Jones PhD' from 'MD PhD' to 'MD, PhD' and left
'John Smith MD "Doc" PhD' unfixed -- measured 2026-09-06 against
the parent tree, witnessed by no corpus name, so the gate was
green over it and the code review is what caught it.

Sites the first commit made false, found by its comment-accuracy
review and fixed here with dated clauses: #429's "left alone"
paragraph (all three of its open clauses are spent -- filed,
fixed, and round-tripping pinned), its present-tense description
of the between-piece marking, and its "the predicate moved down
into group" sentence; #430/#432's "THREE readers" paragraph, which
says two remain; the Declined note that had the "joined" tag
applied by grouping alone. mechanisms.md#ONE-PREDICATE-PER-QUESTION
moves segment_suffix_reading out of the called-by-both list and
adds delimiter_cores, the new instance, to the vocabulary layer's;
the predicate itself is not moved.

The #498 bullet's 'Smith, PhD Jr.' quotation and the matching
_CROSS_RULE_WINNERS row comment both said the ledger discrepancy
was unrepaired. Half of it is repaired: Task 1 corrected
'PhD, Jr.' to 'PhD Jr.' in the three ledgers carrying the comment,
stale since #429. The "title 'PhD' at every baseline" half stands
and is still live, re-measured here, and both sites now say which
half is which.

One release-log bullet in the 2.3.0 block, above the 2.2.0 #429
bullet so that one reads as history: the before/after, the
unchanged comma forms, the deliberate v1 deviation, and the round
trip fixed for these runs with the one name that goes the other
way.

Gate byte-identical to the previous commit -- nothing here is an
input the gate reads. Closes #436, closes #437.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@derek73
derek73 force-pushed the claude/436-suffix-entries branch from f22fa4f to 9e3834b Compare September 6, 2026 20:43
@derek73 derek73 added this to the v2.3 milestone Sep 6, 2026
@derek73
derek73 merged commit 330ee55 into master Sep 6, 2026
11 checks passed
@derek73
derek73 deleted the claude/436-suffix-entries branch September 6, 2026 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug docs Documentation fixes and updates tests tooling

Projects

None yet

1 participant