Skip to content

UFAL/Subject keywords autocomplete stores an unsplit ';'-separated list as a single value #1381

Description

@milanmajchrak

Original issue: ufal/clarin-dspace#1387

Problem

Some items store an entire keyword list in a single dc.subject metadata value (one long ;-delimited string, ~18 keywords) instead of one value per keyword. Confirmed on the DH Data collection (example item 615ee655-974e-467b-a36e-cc8a9a505430, hdl 20.500.12800/1-6204, accessioned 2026-07-14 — a fresh UI submission; already fixed manually).

Root cause (verified in code + production)

  • The DH Data collection (20.500.12800/3) maps to the clariahSubmissions process (item-submission.xml:22). In clariahSubmissionsPageThree, dc.subject is not the tag input — it is <input-type autocomplete-custom="solr-subject_ac">autocomplete</input-type> (submission-forms.xml:1748; its hint at 1749-1757 promises separator splitting and documents hierarchical People::John Doe values). The Teaching materials collection (11234/5012teachingSubmissionsPageThree, :2018+) has the identical field.
  • DsDynamicAutocompleteComponent (src/app/shared/form/builder/ds-dynamic-form-ui/models/autocomplete/ds-dynamic-autocomplete.component.ts) has no splitting at all:
    • onBlur (135-138) dispatches the entire raw input string as ONE value.
    • onSelectItem/updateModel (126-129 / 144-154) dispatch the picked suggestion verbatim — and Solr *_ac suggestions are built from stored values verbatim, so existing unsplit values breed new ones (feedback loop).
    • Bonus defect: ngOnInit (102-120) does not call super.ngOnInit(), so chips/hasAuthority stay undefined → pressing Enter/comma runs the inherited addTagsToChips into a silent TypeError and the typed value is lost.
  • Users accustomed to clarin-dspace v5 (one box, ;-separated, server-side split via REPEATABLE_SPLIT_REGEX = ",|;") paste the whole list → one metadata value. The missing semicolon inside the example value ("Digital Humanities First Intermediate Period") confirms a hand-assembled paste.
  • No server-side splitting exists in v7 (the only submission-path split(";") is DescribeStep.java:331 — local.sponsor complex handling).
  • Secondary defects in the tag input (used by the traditional form's dc.subject, :284): onKeyUp (dynamic-tag.component.ts:186-193) handles only keyCodes 13/188 — ; is keyCode 186 and does not trigger the split despite the code comment and the form hint; the typeahead offers the raw text as a selectable entry (92-103) and onSelectItem (164-174) adds it as ONE chip without splitting; onBlur with an open popup discards the value.
  • Other repeatable autocomplete fields have the same exposure: dc.publisher (:145/1633/1936), dc.contributor.author (:193), dc.language.iso (:254/269/1717/1732/2003), dc.relation* (specialFields).

Reproduction (local docker stack)

  1. Either map a collection to clariahSubmissions (a fresh DB has no handle 20.500.12800/3 — create a collection and map its handle; note the docker compose uses a prebuilt image, so config changes need a mount/custom build), or zero-config: use dc.publisher / dc.language.iso on the default form — same component.
  2. Paste kw1; kw2; kw3 into the field, click elsewhere (blur), save.
  3. One dc.subject row with the literal value kw1; kw2; kw3.
  4. Tag-input path (traditional dc.subject): paste a list → click the typeahead suggestion → one chip with the whole list; Enter with the popup open / blur can silently lose the value.

Suggested fix

Frontend (this repo):

  1. DsDynamicAutocompleteComponent: split the input on ; (, — pending customer decision) in all three dispatch paths (blur, Enter, suggestion select — a suggestion itself can be an unsplit string from *_ac). Trim tokens, drop empties. Never split on : (hierarchical People::John Doe).
    Mandatory guard: DsDynamicSponsorAutocompleteComponent inherits onBlur, and its composed value legitimately contains ; (AUTOCOMPLETE_COMPLEX_PREFIX) — the split must be scoped (per-field config / whitelist / subclass); sponsor and complex-prefix values must be excluded, or local.sponsor gets corrupted.
  2. Cleanup in the same PR: fix the inherited onKeyUp/undefined-chips TypeError (override or guard); add keyCode 186 to the tag component's onKeyUp; split in the tag onSelectItem path; drop the stale twobox comment (submission-forms.xml:1745).
  3. Tests: pasting a; b; c + blur yields three values; Enter and ; work without TypeError; a suggestion containing ; is split; the sponsor composed value is NOT split; :: values are NOT split; non-repeatable fields unchanged.

Backend (companion issue dataquest-dev/DSpace#1373): server-side normalization in DescribeStep.doPatchProcessing (the original issue says "split on ingest"), QA detector, and the one-off curation of existing data (split preserving place order, copy text_lang, mandatory human review).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions