Skip to content

fix(revise): a suffix value derives its entries from its own commas (#511) - #512

Merged
derek73 merged 3 commits into
masterfrom
claude/511-revise-entries
Sep 7, 2026
Merged

fix(revise): a suffix value derives its entries from its own commas (#511)#512
derek73 merged 3 commits into
masterfrom
claude/511-revise-entries

Conversation

@derek73

@derek73 derek73 commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Closes #511.

Problem

Parser.revise(name, suffix=value) classified the value by a full sub-parse and then forced every harvested token to SUFFIX — after the sub-parse had already run the #436 entry pass, which keys on Role.SUFFIX. A bare "MD PhD" reads in that sub-parse as a title plus a family name, so the pass joined nothing, and the suffix view comma-joined two unjoined tokens. Feeding a name's own suffix back was not the identity:

input p.suffix revise(p, suffix=p.suffix).suffix before
John Smith MD PhD MD PhD MD, PhD
John Smith, Ph. D. Ph. D. Ph., D.
Doe, John, MD PhD - FACS Fellow MD PhD - FACS Fellow MD, PhD, -, FACS, Fellow

Measured 2026-09-06 over the 1117 distinct names in tools/differential/corpus*.jsonl (368 carry a suffix): 38 did not round-trip.

Fix

Three commits, one concern each, bisectable in order.

  1. refactor(post_rules) — the entry pass at the tail of post_rules becomes _mark_suffix_entries (in place over the stage's token list, as every other post rule writes) plus the wrapper suffix_entries(state). Pure move; the predicate is unedited. A first draft had post_rules call the state wrapper and cost three more calls per parse against the benchmark band (py3.11: 450 → 454 calls/name; the band tops at 455.9), so the worker spelling was chosen (451). Four stage tests exercise the wrapper over forced-role states.
  2. fix(revise)revise sub-parses each value to a state, forces the named role on every non-dropped token, runs suffix_entries over the forced state, assembles and harvests. So a comma in the value parts two credentials and a space joins them: the rule a whole name uses (rules.md#R1, which gains an Accepted: clause and _parser.py in implemented:). parse() is untouched; a draft _run helper shared with parse cost a frame on the hot path and was dropped.
  3. docs — decisions.md Should Parser.revise(name, suffix=name.suffix) be the identity? (John Smith MD PhD gives MD PhD, and revising with that string gives MD, PhD back) #511 bullet under C1 (R1's history pointer) with the measurement and recipe, the phd-merge acceptance of Ph., D. marked superseded with its head-position reasoning left standing, the ONE-PREDICATE-PER-QUESTION instance, and the 2.3.0 release bullet.

Behavior

  • revise(n, suffix="MD PhD")MD PhD (was MD, PhD); "MD, PhD" stays two entries.
  • revise(n, suffix="Ph. D.")Ph. D. (was Ph., D., the 2026-08-31 ACCEPTED outcome). The Ph. D. merge is still a head-position rule and still does not fire in a value; the entry pass joins the pair because they share a comma bucket. Superseded, not reversed in reasoning.
  • Round-trip: 38 → 1 of 1117. The one left is 김민준씨, J.씨: the whole-name parse keeps J.씨 one glued suffix token while the bare value's sub-parse peels the honorific off the initial, so the revised field renders 씨, J. 씨 (was 씨, J., 씨) — right entries, a word read differently on its own. That is the "classified ON ITS OWN" limit the docstring records; CJK honorific peeling is a W-rule question, not moved. Pinned.
  • Limit, pinned: in a value with no comma of its own, a delimiter named through extra_suffix_delimiters is a word of the run rather than a separator (the drop that parts entries happens only on a segment after a comma). The round-trip is unaffected: the whole-name view already renders that boundary as a comma.
  • The sub-parse's tags are kept minus FOLDED_TAG, as before. A draft that cleared the sub-parse's own "joined" was measured and backed out: it destroyed within-piece merge marks on non-suffix values, and no view reads the tag off a non-suffix token.

Verification

🤖 Generated with Claude Code

@derek73 derek73 added the bug label Sep 6, 2026
@derek73 derek73 self-assigned this Sep 6, 2026
@codecov

codecov Bot commented Sep 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.66%. Comparing base (330ee55) to head (9900f05).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #512   +/-   ##
=======================================
  Coverage   98.66%   98.66%           
=======================================
  Files          45       45           
  Lines        3219     3232   +13     
=======================================
+ Hits         3176     3189   +13     
  Misses         43       43           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@derek73 derek73 added this to the v2.3 milestone Sep 7, 2026
derek73 and others added 3 commits September 6, 2026 19:05
A pure move: the pass at the tail of post_rules that tags a SUFFIX
token "joined" when it continues the entry before it is now a
module-level worker, _mark_suffix_entries, in place over the stage's
token list as every other post rule writes, with suffix_entries() the
state-in/state-out wrapper. Parser.revise is the wrapper's caller in
the next commit (#511): it needs to run the pass over a sub-parse
whose roles it has forced, which the tail of a stage cannot offer.
Predicate unedited.

Why two spellings: a first draft made post_rules call the state
wrapper, and the second ParseState build cost three more calls per
parse against the band tests/v2/test_benchmark.py holds (py3.11,
2026-09-06: 450 calls/name before, 451 with the worker, 454 with the
draft; the facade band tops at 456). AGENTS.md's one-public-function
stage rule gains the sanctioned exception in the same commit.

Four stage tests exercise the wrapper over forced-role states, which
is the shape revise() will hand it, including a forced non-suffix
role whose sub-parse had tagged a word on its own.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Parser.revise(name, suffix=value) forced every harvested token to
SUFFIX after the sub-parse had already run the entry pass, so a bare
'MD PhD' -- read there as a title and a family name -- came back as
two entries and rendered 'MD, PhD'. revise(p, suffix=p.suffix) was
not the identity on 38 of the 1117 corpus names (2026-09-06).

revise now sub-parses to a state, forces the named role on every
non-dropped token, and runs suffix_entries over the forced state, so
a comma in the value parts two credentials and a space joins them --
the rule a whole name uses (rules.md#R1). 'Ph. D.' joins by the same
rule; the 2026-08-31 acceptance of 'Ph., D.' on this path is
superseded, the head-position reasoning behind it untouched. Two
limits pinned: a policy-named delimiter inside a value with no comma
of its own is a word of the run, the drop that would part it
happening only on a segment after a comma; and a glued CJK honorific
the whole name kept on an initial peels in the value's own parse.

Round-trip: 1117 1 after, 1117 38 before -- the one left is that
honorific, '김민준씨, J.씨', right entries and one word read on its
own. Gate byte-identical at all four baselines: revise is not on the
compare path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The #511 decision under C1 (R1's history pointer) with the 38 -> 1
measurement, its recipe and the one name left; the phd-merge
acceptance of 'Ph., D.' marked superseded with its reasoning left
standing; the ONE-PREDICATE-PER-QUESTION instance suffix_entries now
is; and the 2.3.0 release bullet, the #436 bullet losing the sentence
that recorded this limit as unfixed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@derek73
derek73 force-pushed the claude/511-revise-entries branch from a51a280 to 9900f05 Compare September 7, 2026 02:05
@derek73
derek73 merged commit 79160f3 into master Sep 7, 2026
11 checks passed
@derek73
derek73 deleted the claude/511-revise-entries branch September 7, 2026 02:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Should Parser.revise(name, suffix=name.suffix) be the identity? (John Smith MD PhD gives MD PhD, and revising with that string gives MD, PhD back)

1 participant