Skip to content

range-parent falsely fires when a .lift-ranges mixes NFC and NFD between @id and @parent #14

Description

@imnasnainaec

Problem

The range-integrity check in iter_problems() compares a range-element's parent
attribute against its siblings' id values as raw strings
(src/sil_lift/_validate.py:422-431):

for range_ in all_ranges.values():
    element_ids = {element.id for element in range_.elements}
    for element in range_.elements:
        if element.parent and element.parent not in element_ids:
            yield Problem("error", "range-parent", ...)

Real FLEx exports mix Unicode normalization forms within a single .lift-ranges
file
range-element/@id in NFD, @parent in NFC — so a correctly parented
hierarchy is reported as a range-parent error.

Observed in a real SIL.FLEx 8.3.12.43172 export (Sena, from sillsdev/TheCombine
Backend.Tests/Assets/Sena.zip): 4 errors, all spurious.

range 'grammatical-info': range-element 'Associativo' has parent 'Preposição' which is not a sibling id
range 'grammatical-info': range-element 'Prepositional phrase' has parent 'Preposição' which is not a sibling id
range 'from-part-of-speech': range-element 'Associativo' has parent 'Preposição' which is not a sibling id
range 'from-part-of-speech': range-element 'Prepositional phrase' has parent 'Preposição' which is not a sibling id

The range does define the parent. The defining element's id is NFD while the
referring @parent is NFC:

id     = 'Preposição'   (NFD: c + U+0327, a + U+0303)
parent = 'Preposição'     (NFC: U+00E7, U+00E3)

All four resolve once both sides are NFC-normalized. Because range-parent is
error-level, this fails validate outright — --strict is not needed to trip it.

Inconsistent with the module's own stated policy

The undefined-range-value check ~40 lines below normalizes for exactly this
reason, with a comment naming the quirk:

# Comparison is NFC-normalized: FLEx writes the .lift in NFC but the
# companion .lift-ranges in NFD within the same export.
def nfc(value: str) -> str:
    return unicodedata.normalize("NFC", value)

The range-integrity check above it never got the same treatment. Sena also widens
the quirk's description: the mismatch is not only across the .lift/.lift-ranges
pair, it occurs between two attributes inside one .lift-ranges file.

Proposed fix

Normalize both sides of the comparison. RangeElement.parent is read only at
_reader.py:608, written only at _writer.py:548, and consumed only at
_validate.py:425, so the change is confined to the check — no model change, and
nothing about what gets serialized moves.

element_ids = {nfc(element.id) for element in range_.elements}
for element in range_.elements:
    if element.parent and nfc(element.parent) not in element_ids:

This requires hoisting nfc() above the range-integrity block — it is currently a
closure defined further down the same function.

Test

A hand-authored fixture under tests/corpus/negative/: one .lift +
.lift-ranges pair whose range-element ids are NFD and whose @parent values are
NFC, asserting zero range-parent findings, plus one genuinely dangling @parent
in the same file to prove the check still fires. No real-world corpus import
needed.

The fixture should also assert byte-identical save: normalization belongs in the
comparison only, and the original NFD/NFC bytes must survive the round trip per the
fidelity contract.

Note on scope

Only range-parent is affected. Sweeping the other identity comparisons in
_validate.py for the same defect is worth doing while here — duplicate-guid
compares guids (ASCII, unaffected) and dangling-ref compares entry ids/guids,
where an NFD/NFC split between a relation/@ref and the target entry/@id is
plausible in the same way but was not observed in the exports checked.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions