Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 16 additions & 6 deletions docs/design/decisions.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/design/mechanisms.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Problem shape. "Which stage does X?" — asked before attributing behavior in pr

## ONE-PREDICATE-PER-QUESTION — one predicate answers it, and every other site calls that

Problem shape. Two stages need the same answer about the same input, and the one that does not own the decision is about to test for it. Contract statement. Where two sites ask the same question, exactly one predicate answers it and every other site calls that one — never a condition written to match it. The predicate belongs to the QUESTION, not to whichever stage decides: it may sit in a leaf both stages import, and for the leading-title test it must, since the deciding stage is assign and group cannot import assign. How it works. A hand-written mirror agrees with its original only until one of them moves, and the drift is invisible in both directions: each site keeps passing its own tests while they disagree about an input neither covers. Five instances, every one found as a defect before it was found as a pattern — #319 lifted the wholly-suffix predicate into the vocabulary layer "so the comma decision and the honorific peel's segment test cannot drift apart"; #401/#421 lifted the trailing-numeral fork out of assign so the bound-given reserve stopped carrying a copy, its hand-written mirror having been falsified in review more than once — the lesson recorded there being that what must be mirrored is assign's WALK, not merely its condition; #425 replaced that reserve's hand re-derivation of the trailing peel with one function over the view the join would leave; #424 moved assign's leading-title test down because group's own `title()` does not see H2's unlisted abbreviations, so `Xyz. van Johnson` chained where `Dr. van Johnson` did not; #429 moved the no-name-segment test down because group asked by segment INDEX where assign asks by CONTENT. The destination follows the LAYER, not the topic: a predicate over token text goes to `_vocab`, one over pieces and tags to `_pieces`. Both are leaves the stages sit on. The piece layer got its own module only in #439 — until then those predicates collected in `_group`, not because grouping owned them but because `_assign` imports `_group` and cannot be imported back, so group was the one place both stages could reach; five had accumulated across four PRs before the module existed. Stage order is this mechanism's limit, and it forecloses the alternative: where the reader comes AFTER the decider, record the answer on the state instead — `ParseState.order` is that shape, "Recorded rather than recomputed downstream, because the two can differ" — which is unavailable whenever the EARLIER stage is the one asking. (The concrete assign→group import that forced the `_group` collection is gone since #439; what remains is the ordering it was a symptom of, and tests/v2/test_layering.py is where the leaf's contract is now written down.) The cost is a second evaluation of the same predicate, measured for #429 at 1.2–2.2% of a family-comma parse and 0% of every other; recording that number was the right answer there over plumbing a state field the two sites would not otherwise share. Lives in. nameparser/_pipeline/_vocab.py over text (is_wholly_suffix; is_trailing_numeral_suffix — the #401/#421 instance, whose only caller since #439 is the shared peel rather than a stage; and maiden_marker_run, the #434 instance and the clearest two-stage case, called by classify over token texts and by extract over a clause's whitespace words, with group reading the tags classify recorded because it runs later) and nameparser/_pipeline/_pieces.py over pieces: is_suffix_piece, is_leading_title, leading_titles, peel_walk, peel_trailing and segment_suffix_reading are called by both stages — the last of those is #430's instance, where THREE readers share one answer, the render join being the third — while is_title_piece and trailing_start are called by group alone — `trailing_start` being the one to know, since it answers where the trailing run begins and is what P2's chain and M2's walk stop at. tests/v2/test_layering.py holds each module's contract, and a piece predicate growing a dependency on a STAGE shows up there as a widened entry. Reach for it when. You are about to write a condition that mirrors, matches or "does what X does" — or you find a comment saying one does. Grep for the other site's predicate and call it instead.
Problem shape. Two stages need the same answer about the same input, and the one that does not own the decision is about to test for it. Contract statement. Where two sites ask the same question, exactly one predicate answers it and every other site calls that one — never a condition written to match it. The predicate belongs to the QUESTION, not to whichever stage decides: it may sit in a leaf both stages import, and for the leading-title test it must, since the deciding stage is assign and group cannot import assign. How it works. A hand-written mirror agrees with its original only until one of them moves, and the drift is invisible in both directions: each site keeps passing its own tests while they disagree about an input neither covers. Five instances, every one found as a defect before it was found as a pattern — #319 lifted the wholly-suffix predicate into the vocabulary layer "so the comma decision and the honorific peel's segment test cannot drift apart"; #401/#421 lifted the trailing-numeral fork out of assign so the bound-given reserve stopped carrying a copy, its hand-written mirror having been falsified in review more than once — the lesson recorded there being that what must be mirrored is assign's WALK, not merely its condition; #425 replaced that reserve's hand re-derivation of the trailing peel with one function over the view the join would leave; #424 moved assign's leading-title test down because group's own `title()` does not see H2's unlisted abbreviations, so `Xyz. van Johnson` chained where `Dr. van Johnson` did not; #429 moved the no-name-segment test down because group asked by segment INDEX where assign asks by CONTENT. The destination follows the LAYER, not the topic: a predicate over token text goes to `_vocab`, one over pieces and tags to `_pieces`. Both are leaves the stages sit on. The piece layer got its own module only in #439 — until then those predicates collected in `_group`, not because grouping owned them but because `_assign` imports `_group` and cannot be imported back, so group was the one place both stages could reach; five had accumulated across four PRs before the module existed. Stage order is this mechanism's limit, and it forecloses the alternative: where the reader comes AFTER the decider, record the answer on the state instead — `ParseState.order` is that shape, "Recorded rather than recomputed downstream, because the two can differ" — which is unavailable whenever the EARLIER stage is the one asking. (The concrete assign→group import that forced the `_group` collection is gone since #439; what remains is the ordering it was a symptom of, and tests/v2/test_layering.py is where the leaf's contract is now written down.) The cost is a second evaluation of the same predicate, measured for #429 at 1.2–2.2% of a family-comma parse and 0% of every other; recording that number was the right answer there over plumbing a state field the two sites would not otherwise share. Lives in. nameparser/_pipeline/_vocab.py over text (is_wholly_suffix; is_trailing_numeral_suffix — the #401/#421 instance, whose only caller since #439 is the shared peel rather than a stage; and maiden_marker_run, the #434 instance and the clearest two-stage case, called by classify over token texts and by extract over a clause's whitespace words, with group reading the tags classify recorded because it runs later; and delimiter_cores, the #436/#437 instance, read by group where a tail segment DROPS a configured delimiter core and by post_rules where the suffix view's entry boundary asks whether a dropped token was one, with a third reader inside this same module, is_wholly_suffix, where a configured core counts as suffix-shaped) and nameparser/_pipeline/_pieces.py over pieces: is_suffix_piece, leading_titles, peel_walk and peel_trailing are called by both stages, while is_leading_title, is_title_piece and trailing_start are called by group alone (measured 2026-09-06 by call site: `is_leading_title` has no caller in `_assign.py`, which reads `leading_titles` instead — a first draft of this clause listed it among the shared ones) — `trailing_start` being the one to know, since it answers where the trailing run begins and is what P2's chain and M2's walk stop at — and segment_suffix_reading by assign alone since #436/#437, that last one being #430's instance, where THREE readers shared one answer until the render join, group's third, was replaced by a rule over the commas the writer typed (decisions.md#C1, 2026-09-06); it stays where it is, one call site being no reason to move a predicate that two sites will contest again. tests/v2/test_layering.py holds each module's contract, and a piece predicate growing a dependency on a STAGE shows up there as a widened entry. Reach for it when. You are about to write a condition that mirrors, matches or "does what X does" — or you find a comment saying one does. Grep for the other site's predicate and call it instead.

## RENDER-HONORS-THE-PARSE — the parse decides it, the views honor it

Expand Down
16 changes: 10 additions & 6 deletions docs/design/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@ P2. Rationale: a particle is written as part of the surname it
trailing run the chain stops before it as before any suffix
word, and the peel takes it; where it continues a prefix run,
the run takes it as a particle, as P6 reads it after a comma.
"John Smith Mc V" → suffix="Mc, V"
"John Smith Mc V" → suffix="Mc V"
"John van Mc" → family="van Mc"
Accepted: a caller wanting the combined double-surname reading
(#132's ask) has it as the surnames view rather than the
Expand Down Expand Up @@ -339,7 +339,7 @@ P5. Rationale: some given-name words are incomplete alone — "abdul"
"abdul Smith V" → suffix="V"
"abdul Smith Jr V" → family="Smith"
"abdul Smith Jr Ma" → family="Smith"
"abdul Smith Jr Ma" → suffix="Jr, Ma"
"abdul Smith Jr Ma" → suffix="Jr Ma"
"abdul Smith Ma" → suffix="Ma"
"abdul Smith Berg Ma" → family="Berg" · boundary
"abdul Sir Smith Berg" → given="abdul Sir"
Expand Down Expand Up @@ -876,7 +876,7 @@ C1. Rationale: a credential run after the comma means the name is in
"Smith, PSM I." → suffix="PSM I."
"Smith, PSM I." strict-comma-suffixes → suffix="I."
"Smith, John V." → middle="V."
"Smith, John PhD I." → suffix="PhD, I."
"Smith, John PhD I." → suffix="PhD I."
"Smith, John V" → suffix="V" · boundary
"Smith, Ph. D. Jr." → suffix="Ph. D. Jr."
"Smith, MD PhD" → suffix="MD PhD"
Expand Down Expand Up @@ -1122,7 +1122,7 @@ W3. Rationale: a family name declared by a comma is the writer's
"남궁민수, 지훈" → family="남궁민수" · boundary
"田中さん, Dr." → family="田中さん"
"田中さん, PhD" → suffix="さん, PhD"
"田中さん 様." → suffix="さん, 様."
"田中さん 様." → suffix="さん 様."
tolerated: native CJK writing has neither a family-comma convention nor a period standing after an honorific, so the four comma lines above and the period line under them illustrate current behavior — changeable without notice — rather than promise it; the line carrying neither, beside them, is W1's claim, which is normative. All five stay watched at every released baseline on the differential's radar tier (tools/differential/corpus_cjk_tolerated.jsonl, projected from the `tolerated` rows of tests/v2/cases.py) instead of its contract tier, and those rows pin them at HEAD.
history: decisions.md#W3 · interacts: W1, W2, C1 · implemented: nameparser/_pipeline/_script_segment.py

Expand Down Expand Up @@ -1225,13 +1225,17 @@ R1. Rationale: a field is a way of reading the parse, not a stored
is the one place this is visible, because it is the only field
that can hold parts the writer comma-separated: a run of
post-nominals written with spaces renders with spaces, and one
written with commas keeps them.
written with commas keeps them. The separator is the comma the
writer typed, read off the words' positions once their roles are
known; a delimiter the configuration names is a separator too,
and so is any name word standing between two post-nominals.
"Dr. Juan Q. Xavier de la Vega III" → family="de la Vega"
"Hassan, Mohamad Ahmad Ali" middle_as_family → family="Ahmad Ali Hassan"
"Hassan, Mohamad Ahmad Ali" → family="Hassan" · boundary
"Smith, MD PhD" → suffix="MD PhD"
"John Smith MD PhD" → suffix="MD PhD"
"John Smith, MD, Bart" → suffix="MD, Bart"
history: decisions.md#C1 · interacts: O3, P6, R3 · implemented: nameparser/_types.py
history: decisions.md#C1 · interacts: O3, P6, R3 · implemented: nameparser/_pipeline/_post_rules.py, nameparser/_types.py

R2. Rationale: callers need the surname with and without its
particles — sorting wants "Vega", display wants "de la Vega".
Expand Down
11 changes: 8 additions & 3 deletions docs/modules.rst
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,14 @@ Results
actual family particles),
``conjunction`` (a joining word, "and"/"y"), ``initial`` (an
initial-shaped word in a script that HAS initials — "J." or "А.",
never "씨."), and ``joined`` (a continuation of the
previous token within one merged piece, so the suffix view renders
"Ph. D." as one credential). Every other tag is namespaced
never "씨."), and ``joined`` (a continuation of the token before it
— within one merged piece the tag is role-blind and every view
joins the pair with a space, so it renders "Ph. D." as one
credential and ``Smith, Ph. D. Smith`` gives ``first_list``
``['Ph. D.']``, and since #436 it also spans the pieces of one
SUFFIX entry, the run of post-nominals the writer wrote without a
comma — "MD PhD" — which is the half the suffix view alone reads
and joins with a space instead of ", "). Every other tag is namespaced
(``vocab:...``) and unstable — never match against those.

.. autoclass:: nameparser.Span
Expand Down
2 changes: 2 additions & 0 deletions docs/release_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ Release Log

- **Record a 2.0.0 change to HumanName.initials() that no release note had classified:** since 2.0.0 the facade initials each WORD of a name part, where 1.4.0 initialed a joined run as one group -- ``HumanName("Juan Velasquez y Garcia").initials()`` is ``J. V. G.`` and was ``J. V G.``; ``Abdul Salam Hassan`` is ``A. S. H.`` and was ``A S. H.``. Nothing changes in 2.3.0; the differential gate now compares ``initials()`` (#484) and this is what it found. See the ``differential-ledger, the initials view`` entry of ``docs/design/decisions.md``

- **Fix a space-separated run of post-nominals rendering with a comma the writer never typed.** ``HumanName("John Smith MD PhD").suffix`` is ``MD PhD`` and was ``MD, PhD`` at every release since 1.4.0; ``Kenneth Clarke QC MP`` gives ``QC MP``, and the CJK honorific runs (``田中さん II``, ``김민준 박사 씨``) follow the same rule. This is a deliberate deviation from 1.4.0, which inserted a comma into a run the writer had spaced; the v1-parity suite is re-pinned to match. The comma forms are unchanged -- ``HumanName("Smith, MD, PhD").suffix`` is still ``MD, PhD`` -- because the separator is now the comma the writer typed rather than the shape of the comma segments, and a configured suffix delimiter still parts a run, as does a name word standing between two post-nominals. Round-tripping is fixed for these runs, which the 2.2.0 note below recorded as broken: ``str(HumanName("Smith, MD PhD"))`` is ``Smith MD PhD`` and re-parses to suffix ``MD PhD``, where the comma-written and space-written spellings of one run used to disagree. ``str()`` is still a rendering rather than a canonical form, and one name goes the other way: ``HumanName("Smith, John PhD I.")`` renders ``John Smith PhD I.``, which reads middle ``Smith PhD`` and family ``I.`` as it always has, so that name round-tripped only on the strength of the comma this fix removes. Thirteen names that predate this change move in the differential corpora (fourteen with the example this change adds), all in the same direction, and no other field view moves. One limit is not fixed here: ``Parser.revise(n, suffix=n.suffix)`` is not the identity on a space-joined run, because the bare string has no comma to route by and the words come back as separate entries — pre-existing, ``revise``'s own docstring recording that a value is classified on its own, and reachable on more inputs now. See the ``C1`` entry of ``docs/design/decisions.md`` (closes #436, closes #437)

* 2.2.0 - August 31, 2026

nameparser 2.2 is a rename plus about thirty parsing fixes.
Expand Down
Loading