Skip to content

Commit 4e929cf

Browse files
derek73claude
andcommitted
fix(particles): a tussenvoegsel after a family comma joins the family
"Beethoven, Ludwig van" is how "Ludwig van Beethoven" is filed in a Dutch or Belgian alphabetized listing, and the trailing particle run was read as a middle name. It now attaches to the family the comma has already named and renders before it, so the listing and the plain spelling agree: Beethoven, Ludwig van -> given 'Ludwig', family 'van Beethoven' Berg, Jan van der -> given 'Jan', family 'van der Berg' rules.md#P6 has stated this since the 2026-08-16 keystone; both its deviates: #379 markers come out, and its pointer moves from tracked: to implemented:. The derived views move with the parse, which closes the half of #130 that asked for them (family_particles 'van', family_base 'Beethoven', where both were empty and 'Beethoven'). Keyed on the token's VOCABULARY, not its assigned role: that is what gives P6's stated precedence over S2 its effect, since assign reads a trailing `vd`/`mc`/`do` as a post-nominal. "Berg, Jan vd" therefore reads family 'vd Berg' where 1.4.0 and 2.1 alike gave suffix 'vd', which closes #380 on the same shape. The words-to-spare guard is a piece test rather than a count, so "Nguyen, Van" keeps its only given word. One of 751 corpus names moves at all three baselines, and needed a ledger rule in each. The ledger guards forced the regex to narrow: the first draft reused the neighbouring comma rules' bare-comma pattern, which claimed 215 names and reached two protected exclusions; the member `des` came out too, being broad enough to match the name "Des". Closes #379 Closes #380 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 5b1d8b6 commit 4e929cf

10 files changed

Lines changed: 346 additions & 14 deletions

File tree

docs/design/decisions.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,11 @@ Open: [#360](https://github.com/derek73/python-nameparser/issues/360) which part
8080

8181
### P6 — the trailing orphan particle
8282

83+
- 2026-08-18 #379/#380 IMPLEMENTED — the rule shipped as stated, both `deviates:` markers out. Keyed on the token's VOCABULARY rather than its assigned role, which is what gives the S2 precedence its effect: assign reads a trailing `vd`/`mc`/`do` as a post-nominal, and the attachment overrides that after a comma. The words-to-spare guard is a PIECE test, not a count — every trailing piece that is wholly particles attaches, and the run must leave a GIVEN or MIDDLE word ahead of it, so "Nguyen, Van" keeps its only given word. Not "a name word": see the defect below, where that wider reading passed on family text P1 had just produced.
84+
Measured: ONE of 751 corpus names moves ("Vega, Juan de la"), at all three baselines, and it needed a new ledger rule in each. The LEDGER rule's regex — not the parse rule, which has no regex and fires on the whole particle vocabulary through the tag — is a PARTIAL copy of PARTICLES, deliberately: the words that actually end a Dutch, Iberian or German listing, not all 70. The ledger guards forced that narrowing: the first draft used the neighbours' bare comma regex, which claimed 215 names and reached two protected exclusions. A member `des` had to come out as well, and for a reason worth keeping straight: it is not in PARTICLES at all, so it could copy nothing, and the text it reached was "Designated" (from a corpus name) rather than a person's name. `_reaches_non_vocabulary("des", PARTICLES)` is the one-line recomputation.
85+
Two defects found in review, both from the rule's neighbours rather than from its own logic. P1's fold runs EARLIER in the same function and retags all of segment 1 to FAMILY, so a words-to-spare test written as "some name role remains" passed on family text P1 had just produced — and the attachment then hoisted the particle in front of a base it never preceded ("Smith, de Mesnil van" → 'van Smith de Mesnil'). 1.4 gives 'de Mesnil van Smith' for that input and 2.1 gives 'Smith de Mesnil van', so neither the defect nor the fix restores 1.4 — what the fix restores is 2.1, the pre-comma-family reading having moved back in 2.0. The test has to name GIVEN and MIDDLE, which is what the rule says. And a post-nominal sits BEHIND the tussenvoegsel in this listing, so the backward walk stopped on it and the attachment never fired: "Berg, Jan van Jr." and "Berg, Jan van, Jr." parsed two ways on nothing but a comma. The walk now steps over a trailing piece holding no name, unless it is itself particle vocabulary — `vd` arrives suffix-roled and IS the run.
86+
Fallout worth its own record: "Vega, Juan de la" was the input tests/test_initials.py used to build an ALL-PARTICLE middle name, and this rule takes that shape away from THAT input. It does not remove the shape — "Vega, Santa de la", "Berg, Van der" and "Nguyen, Van Le" all still produce one in the default order, because the trailing run swallows the given word too and the attachment then declines. The test is repointed at the last of those rather than deleted. Whether such a middle should exist at all is #402 — for a never-given particle it is #365's defect, and for an ambiguous one the parser reports it as a middle name and then declines to initial it, which is a miscount either way.
87+
8388
- 2026-08-16 (order-precedence keystone; #379, #380, #365) — a particle ending the name has nothing to link forward to, and no particle is a name by itself, so it attaches to the family name standing beside it and renders BEFORE it. The distinction from a chain is what makes this a rule rather than an exception: a chained group has a head word and can be positioned; an orphan has no head, so position has nothing to work with.
8489
- Scope: the COMMA form only, deliberately. "Jong, Anke de" is unambiguous — the comma has already named the family. Without the comma the written shape is not settled: "Jong Anke de" may be a misformatted listing (arguably a missing comma under a declared family-first order) and "Jong de" may be a given name beside a particle. Those keep their positional reading and are not tracked as deviations.
8590
- The words-to-spare guard is load-bearing, not incidental. #379's own subject is "van", which is in the AMBIGUOUS half — so a rule keyed to never-given particles alone would not fix the issue it was filed for, while a rule with no guard breaks Vietnamese

docs/design/rules.md

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -245,24 +245,43 @@ P6. Rationale: a particle ending the name has nothing to link
245245
forward to, so it is not doing a particle's work there. A
246246
never-given particle in that position cannot be a name at all
247247
and must belong to the family written beside it; an ambiguous
248-
particle could genuinely be the name (Vietnamese "Van"), which
249-
is what the words-to-spare test below is for, not an
250-
afterthought to it. Dutch and Flemish names are listed exactly
248+
particle could genuinely be the name (Vietnamese "Van"), and
249+
after a comma there is no signal that separates the two
250+
readings. Dutch and Flemish names are listed exactly
251251
this way ("Beethoven, Ludwig van"), the tussenvoegsel trailing
252252
the given name but belonging to the surname.
253253
Where a family comma has already named the family, a particle
254254
ending the name attaches to that family name and is written
255255
before it — provided at least one given word remains, so that a
256256
name whose only given word is the particle keeps it (the
257-
words-to-spare test S2 applies to ambiguous suffixes). Where
257+
words-to-spare test S2 applies to ambiguous suffixes). A
258+
post-nominal is written BEHIND the particle in this listing, so
259+
it does not end the name for this purpose: the run is found by
260+
looking past a trailing word that holds no name. Where
258261
the word is BOTH a particle and suffix vocabulary, this
259262
attachment outranks the suffix reading (S2): a trailing
260263
abbreviation after a family comma is the tussenvoegsel far more
261264
often than the decoration it collides with.
262-
"Jong, Anke de" → family="de Jong" deviates: #379 (today: family="Jong")
263-
"Beethoven, Ludwig van" → family="van Beethoven" deviates: #379 (today: family="Beethoven")
264-
"Berg, Jan vd" → family="vd Berg" deviates: #380 (today: family="Berg")
265+
"Jong, Anke de" → family="de Jong"
266+
"Beethoven, Ludwig van" → family="van Beethoven"
267+
"Berg, Jan vd" → family="vd Berg"
268+
"Berg, Jan van der" → family="van der Berg"
269+
"Vega, Juan de la" → family_particles="de la"
270+
"Beethoven, Ludwig van" → family_base="Beethoven"
271+
"Beethoven, Ludwig van" → family_particles="van"
265272
"Nguyen, Van" → given="Van" · boundary
273+
Accepted: an ambiguous particle attaches on the same terms as a
274+
never-given one, and a Vietnamese name written in this listing
275+
loses its given name: "Nguyen, Thi Van" reads a family of
276+
Van Nguyen, where Vân is the given name. The two spellings are
277+
identical and the parser has no signal to separate them, so one
278+
tradition has to lose this shape. What makes that acceptable is
279+
that each has a format that reads correctly: Dutch the comma
280+
listing above, Vietnamese the comma-LESS family-first spelling,
281+
which the scoping below leaves alone.
282+
"Nguyen, Thi Van" → family="Van Nguyen"
283+
"Nguyen Thi Van" family-first-given-last → family="Nguyen"
284+
"Nguyen Thi Van" family-first-given-last → given="Van"
266285
Accepted: without a family comma the name's written shape is not
267286
settled — "Jong Anke de" may be a misformatted listing, and a
268287
bare "Jong de" may be a given name beside a particle — so the
@@ -274,7 +293,7 @@ P6. Rationale: a particle ending the name has nothing to link
274293
vocabulary — today vd, do and mc. Only vd's reading was
275294
weighed; the other two inherit it, which is the shape's cost
276295
and is tracked with the other contested memberships.
277-
history: decisions.md#P6 · interacts: C1, P1, S2 · tracked: #379, #380
296+
history: decisions.md#P6 · interacts: C1, P1, S2 · implemented: nameparser/_pipeline/_post_rules.py
278297

279298
## Suffixes: generational & credentials (S)
280299

docs/release_log.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ Release Log
3232

3333
- Fix a name opening with a particle that is *never* a given name being split at the particle under a family-first name order -- ``Policy(name_order=FAMILY_FIRST)`` and ``Policy(name_order=FAMILY_FIRST_GIVEN_LAST)`` alike, and identically: ``"de Mesnil"`` read as family ``de``, given ``Mesnil``, and ``"de la Vega"`` as family ``de``, given ``la Vega``. Each is now the whole surname, as it has always been in the default order. The rule enforcing it asked for the particle by the ``GIVEN`` role, which under a family-first order belongs to the token *after* the particle, so the test read the wrong word and declined. It now also asks by position -- the piece that opens the name -- so both shapes of the same rule are caught: where such a particle stands alone as a piece, either opening the name or in the given position, the name is left with no given name at all, the given and the middles folding into the family. Standing *alone* is the whole of it, and the rule claims nothing wider: ``"Juan de la Vega"`` under ``FAMILY_FIRST`` still reports given ``de la Vega``, because there the particle chained onto the words after it rather than standing alone, and a bare ``"de"`` with nothing to fold into is still reported as the given name. The decision behind the fix: a word that can never be a given name leaves ``name_order`` nothing to decide, so declaring family-first is not a reason to make ``de`` a surname on its own. A leading particle that *may* be a given name is genuinely order-dependent and is untouched -- ``"van Gogh"`` still reads as family ``van``, given ``Gogh`` under both family-first orders. This is also what gives ``Lexicon.particles_ambiguous`` an effect outside the default order: taking a word out of it now changes the parsed fields under a family-first order, where before it moved only the ambiguity report. Seven of the 751 differential corpus names move, the same seven under each family-first order; default-order output is byte-identical over all 751, at the 1.4.0, 2.0.0 and 2.1.0 differential baselines alike (closes #359)
3434

35+
- Fix a tussenvoegsel after a family comma being parsed as a middle name. Dutch and Belgian alphabetized listings move the particle behind the given name -- ``"Beethoven, Ludwig van"`` is how ``"Ludwig van Beethoven"`` is filed -- and the trailing particle run was read as a middle name rather than as part of the surname: ``"Beethoven, Ludwig van"`` gave middle ``van``, last ``Beethoven``, and ``"Berg, Jan van der"`` gave middle ``van der``. The run now attaches to the family the comma has already named and renders before it, so those read family ``van Beethoven`` and ``van der Berg`` with the given name unchanged. The derived views move with the parse, so ``family_particles`` is ``van`` and ``family_base`` is ``Beethoven`` where they were empty and ``Beethoven`` before. `#130 <https://github.com/derek73/python-nameparser/issues/130>`_ asked for those views and shipped them in 1.3.0; what was wrong until now was the values they reported for this listing. Both halves of the particle vocabulary attach -- never-given ``de`` and may-be-given ``van`` alike -- because after a comma the family is already named and the particle has no other role to take. Two guards bound it. A name whose only given word is the particle keeps it, so ``"Nguyen, Van"`` still reads given ``Van``: the attachment needs a given word to spare. And where the word is BOTH particle and suffix vocabulary the attachment outranks the post-nominal reading, so ``"Berg, Jan vd"`` reads family ``vd Berg`` where 1.4.0 and 2.1 alike gave suffix ``vd`` -- a trailing abbreviation after a family comma is the tussenvoegsel far more often than the decoration it collides with, and the same shape sweeps in ``mc``, which 2.1 also read as a suffix. ``do`` is in ``SUFFIX_ACRONYMS_AMBIGUOUS`` and 2.1 already read a trailing one as a name word, so it attaches by the plain rule rather than by the override (closes #379, closes #380). Names without the comma are untouched: ``"Ludwig van Beethoven"`` already read family ``van Beethoven`` and is byte-identical. One of the 751 differential corpus names moves, ``"Vega, Juan de la"``, at the 1.4.0, 2.0.0 and 2.1.0 baselines alike
36+
3537
- Add ``abd`` to :data:`~nameparser.config.bound_given_names.BOUND_GIVEN_NAMES`, so the spellings that write the article as its own word join like the others do: ``"abd Allah Smith"`` was given ``abd``, middle ``Allah`` and is now given ``abd Allah``. ``abdul``, ``abdel`` and ``abdal`` were already there, and the Arabic-script ``عبد`` has covered the same word since 2.0, so only the Latin spelling was short. The word is also the postnominal ABD ("All But Dissertation") and stays in ``SUFFIX_ACRONYMS``: position tells the two readings apart, so ``"Jane Smith ABD"``, ``"Jane Smith, ABD"`` and ``"Jane Smith A.B.D."`` all still read the credential as a suffix. Making that work needed one fix beyond the wordlist. The join reserves enough words to leave a family name behind, and the reserve counted every piece except titles and suffixes -- so a word in BOTH vocabularies did not count as a name word even when the rule had already claimed it as one, and the join declined silently on the common three-word shape. The claimed piece now counts, which is what the reserve means: it asks whether enough OTHER words are left to spare. No shipped word other than ``abd`` is in both sets, and for any word that is not, the changed expression reduces to the old one -- so nothing else can move, by arithmetic rather than by sampling. The differential corpus agrees at all 751 names, but it holds no name carrying a dual-membership word, so that agreement tests the corpus rather than the change (#400)
3638

3739
- Change how far a leading never-given particle takes the surname when a family-first ``name_order`` is declared. ``Policy(name_order=FAMILY_FIRST)`` read ``"de Mesnil Juan"`` as family ``de Mesnil Juan`` -- the whole name -- and now reads family ``de Mesnil``, given ``Juan``. Declaring a family-first order asserts that what follows the family is not more surname, and where the surname run ends is exactly that question, so the declaration settles it. The default order is unchanged, deliberately: with no order declared nothing marks where the surname ends, and a particle followed by several words really can be all surname -- ``von Bergen Wessels`` is one such name. Nothing in ``"de Mesnil Juan"`` distinguishes it from that reading except a declared order or a comma. A caller who means family ``de la Vega`` plus given ``Juan`` in the default order writes the comma, which already parses that way. The run takes one name WORD rather than one token: a conjunction-joined run and a bound given-name pair each count once, so the stop cannot land inside one -- ``"de la Vega y Santos Juan"`` reads family ``de la Vega y Santos``, and ``"ibn Awf abdul Rahman"`` reads given ``abdul Rahman``. Where two or more words are left over the two family-first orders differ from each other for the first time: ``"de la Cruz Juan Carlos"`` reads given ``Juan``, middle ``Carlos`` under ``FAMILY_FIRST`` and middle ``Juan``, given ``Carlos`` under ``FAMILY_FIRST_GIVEN_LAST``. An ambiguous leading particle is untouched in every order -- ``"van Gogh Jan Pieter"`` still reads family ``van`` under both family-first orders -- and so is a family comma, where the comma has already fixed the surname (``"Smith, de Mesnil"`` keeps family ``Smith de Mesnil``). One of the 751 differential corpus names moves, ``"de Mesnil Garcia"`` to family ``de Mesnil``, given ``Garcia``, under each family-first order; default-order output is byte-identical over all 751. This reverses the answer #364 was closed on, and the reasoning is recorded at ``docs/design/decisions.md#P1`` (closes #395)

0 commit comments

Comments
 (0)