UFAL/local.sponsor OpenAIRE suggestion stores displayed value (EU) instead of stored value (euFunds)

[milanmajchrak]

Original issue: ufal/clarin-dspace#1388

Problem

local.sponsor (metadata_field_id 136 in production, verified via /api/core/metadatafields/136) sometimes stores the displayed value-pair value EU;… instead of the stored value euFunds;… (metashare_funding value-pairs). As of 2026-07 production has 13 rows starting EU; vs 155 correct rows starting euFunds;, and corruption is ongoing (bad rows created as late as 2026-07-01, e.g. item 11234/1-6202).

Root cause (verified end-to-end in code; all sampled production rows match)

There are two write paths for the funding type — one correct, one broken. That is why good and bad values coexist:

1. Manual path (CORRECT): selecting "EU" in the funding-type dropdown dispatches the VocabularyEntry whose .value is the stored value (dynamic-scrollable-dropdown.component.ts onSelect → dynamic-vocabulary.component.ts:117-120). The vocabulary REST correctly returns display: EU / value: euFunds.
2. OpenAIRE suggestion path (BROKEN): picking an OpenAIRE suggestion in the "Grant no. / funding project code" autocomplete:
   • src/app/shared/form/builder/ds-dynamic-form-ui/models/sponsor-autocomplete/ds-dynamic-sponsor-autocomplete.component.ts → updateModel() (lines 59-69) → composeSponsorInput() (152-178) rebuilds the ENTIRE metadata value and takes the funding type from the i18n translation: loadEUFundingType() (184-206) reads the key autocomplete.suggestion.sponsor.eu = "EU" in every locale (en.json5:6236, cs/de/it the same). The non-EU fallback loadNoneSponsorFundingType() reads sponsor.empty, which under the Czech locale is "nepoužitelné" (cs.json5:9403) — stored values would depend on the UI language.
   • The composed string autocomplete_in_complex_input;EU;<code>;<org>;<name>;info:eu-repo/... replaces the whole complex value verbatim (ds-dynamic-complex.model.ts:72-81), is PATCHed via section-form-operations.service.ts:229-234 (parent.value), and the backend accepts it silently (MetadataValidation.java:277-296 validates only regex + authority, never value-pairs membership).
   • The constant DEFAULT_EU_FUNDING_TYPES = ['EU', 'euFunds'] (ds-dynamic-sponsor-autocomplete.model.ts:10-12) is an existing workaround acknowledging both variants.

Timeline: the bug is latent since the CLARIN v7 port (commit eef127f0, "Upgrade CLARIN-DSpace to 7.5"; the component was last touched 2024-10-17 only cosmetically). It was NOT introduced by PR #897 — that PR unblocked the manual (correct-storing) EU path (ufal/clarin-dspace#1234). The 2025-10-22 onset of bad rows is usage/deployment-driven (EU-funded deposits are rare, OpenAIRE reachability from production is intermittent — ufal/clarin-dspace#1329).

"Is it just the funding value-pair?" — No. The same bug class (display instead of stored dispatched from an autocomplete) previously hit dc.language.iso; the code is already fixed on dtq-dev (PR #1275, port of ufal#124) but production data is still dirty (at least: English; Czech on 11234/1-6153, English on 20.500.12800/1-6040, English + Egyptian Hieroglyphic Transliteration on 20.500.12800/1-6204). Data cleanup is tracked in the companion DSpace issue.

Reproduction (local docker stack)

1. Run dspace-angular (dtq-dev) + DSpace (dtq-dev); any collection whose form contains local.sponsor (default → traditional is enough). Prerequisite: OpenAIRE API reachable.
2. In the submission form, select funding type EU in the dropdown, type ≥3 chars into "Grant no. or funding project code" and click an OpenAIRE suggestion.
3. Save → the stored local.sponsor value starts with EU; and ends with info:eu-repo/....
4. Control: select "National" and fill the fields manually → stores nationalFunds;… correctly.

Suggested fix (single PR, all three parts required)

1. ds-dynamic-sponsor-autocomplete.component.ts
   • composeSponsorInput(): derive the funding type from constants (DEFAULT_EU_STORAGE_VALUE = euFunds; add a storage constant for N/A), never from i18n. Remove/limit loadEUFundingType/loadNoneSponsorFundingType to display-only use.
   • replaceSeparatorValue (258-260): strip ALL ; occurrences (global regex) and guard against undefined fundingOrganization (OpenAIRE entries without project.funder.name).
   • Keep isEUSponsor() tolerant (DEFAULT_EU_FUNDING_TYPES) so existing bad rows stay editable.
2. Mandatory companion change — dynamic-sponsor-scrollable-dropdown.component.ts: shouldCleanInputs (134-153, wipe branch at 143) and cleanSponsorInputs (94-123) compare the funding type only against the display value 'EU'; once we store euFunds, the cleaning logic would wipe the freshly composed sub-fields. Compare against DEFAULT_EU_FUNDING_TYPES, and fix fundingTypeIsNotApplicable (158-160), which compares against a translated label (broken under cs locale).
3. Defense-in-depth — ds-dynamic-complex.model.ts get value() (72-81): normalize the first segment of a consumed suggestion value through the display→stored map (EU→euFunds, Own funds→ownFunds, National→nationalFunds, other→Other).
4. Tests (ds-dynamic-sponsor-autocomplete.component.spec.ts): OpenAIRE suggestion dispatches euFunds segment; ; inside funder/project names and missing funder name do not corrupt segment structure; cs locale does not influence the stored value; dropdown cleaning does not wipe inputs after the fix.

Notes

• Priority: corruption is ongoing — please consider a priority cherry-pick to clarin-v7 instead of waiting for the next routine lindat-merge.
• Backend hardening (value-pairs validation in MetadataValidation), the QA detector and the one-off data curation (including the EU;→euFunds; rewrite and the non-EC info:eu-repo carve-out) are tracked in UFAL/Metadata QA detectors, backend hardening and data curation for issues 1388/1387/169 DSpace#1373.