[Port to dtq-dev] Issue dspace-customers#903: stop ItemConverter writing to the DB on every item GET - #1411
Conversation
ItemConverter.convert() -- invoked during read-only REST serialization -- called ClarinItemService.updateItemDatesMetadata(), which runs clearMetadata/addMetadata. That triggered Hibernate dirty-checking and DB writes on every item GET (rolled back by DSpaceRequestContextFilter), plus per-request log noise. Split derivation from persistence: - ClarinItemService/Impl: extract deriveDateIssuedFromApproximateDate(Item) (pure, returns the derived value or null); updateItemDatesMetadata() delegates to it and only writes when the value actually changes. dtq-dev's log.debug for the empty approximate-date case is kept (the WARN->DEBUG change already landed via 84e9f3a), which is why the source cherry-pick conflicted here. - ItemServiceImpl.update(): derive + persist inside the existing isMetadataModified guard, so the write happens on PATCH/PUT, not on read. - ItemConverter.convert(): drop the write-on-read call; instead override dc.date.issued on the REST DTO from the derived value (display-only, honouring metadataExposureService.isHidden) so stale DB values still display correctly. Port of dataquest-dev/dspace-customers#903 (item 1). Source: customer/zcu-data 24f31c3 (adapted; cherry-pick conflicted on the WARN->DEBUG divergence). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR ports dspace-customers#903 to dtq-dev to eliminate unintended database writes during REST GET /items/{id} serialization by removing entity mutation from the read/DTO conversion path and relocating persistence to the item update path.
Changes:
- Removed the
ItemConverter.convert()write-on-read call toupdateItemDatesMetadata()and replaced it with a DTO-only override ofdc.date.issuedderived fromlocal.approximateDate.issued. - Added a pure derivation method
deriveDateIssuedFromApproximateDate(Item)toClarinItemService/ClarinItemServiceImpl, and updatedupdateItemDatesMetadata()to only write when the stored value differs. - Updated
ItemServiceImpl.update()to persist the derived date only when item metadata is modified (PATCH/PUT).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| dspace-server-webapp/src/main/java/org/dspace/app/rest/converter/ItemConverter.java | Removes write-on-read and derives dc.date.issued for display on the REST DTO only. |
| dspace-api/src/main/java/org/dspace/content/service/clarin/ClarinItemService.java | Adds a pure derivation API to support read-only DTO overrides. |
| dspace-api/src/main/java/org/dspace/content/ItemServiceImpl.java | Persists derived date only during update flows when metadata changes. |
| dspace-api/src/main/java/org/dspace/content/clarin/ClarinItemServiceImpl.java | Extracts derivation logic and avoids unnecessary metadata writes when already up-to-date. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- ClarinItemServiceImpl.updateItemDatesMetadata(): skip the dc.date.issued write only when the field holds exactly one value equal to the derived value. A multi-valued field is now still normalized to the single derived value instead of being skipped on a first-value match (Copilot review #2). - ItemConverter: set the derived dc.date.issued DTO value via MetadataRest#put instead of getMap().put(Collections.singletonList(...)), so place is normalized and list semantics stay consistent with other fields; drops the now-unused Collections import and manual setPlace(0) (Copilot review #1). - Add ClarinItemServiceImplTest: pure Mockito unit tests covering deriveDateIssuedFromApproximateDate (empty/blank/non-numeric/sequence) and the skip/normalize guard (single-equal skip, multi-value normalize, differ). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Addressed both Copilot review comments in
Added |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Suppressed comments (1)
dspace-server-webapp/src/main/java/org/dspace/app/rest/converter/ItemConverter.java:97
- Avoid the magic number
-1for metadata confidence. DSpace definesChoices.CF_UNSETfor the default/unset confidence (andMetadataValuedefaults confidence to -1), so using the constant improves readability and reduces the chance of inconsistency if the value ever changes.
MetadataValueRest dateRest = new MetadataValueRest(derivedValue);
dateRest.setConfidence(-1);
// MetadataRest#put normalizes place and keeps Arrays.asList semantics consistent with other fields
target.getMetadata().put("dc.date.issued", dateRest);
Problem
Every REST
GETof an item triggered database writes.ItemConverter.convert()— invoked during read-only serialization — calledClarinItemService.updateItemDatesMetadata(), which doesclearMetadata/addMetadata. This caused Hibernate dirty-checking and DB writes on read (rolled back byDSpaceRequestContextFilter.abort()), plus per-request log noise, on every item GET.Port of dataquest-dev/dspace-customers#903 (item 1 of a 6-item umbrella). Source:
customer/zcu-data24f31c330d— cherry-pick conflicted (see below), applied as an adapted port.Root cause
Date-derivation (turn
local.approximateDate.issuedintodc.date.issued) is a write that was being run from a read path so that stale DB values would still display correctly. The write belongs inupdate(); the display correction belongs on the DTO.Change set
Split derivation (pure) from persistence (write):
ClarinItemService(interface) — addderiveDateIssuedFromApproximateDate(Item).ClarinItemServiceImpl— extract the parsing intoderiveDateIssuedFromApproximateDate()(returns the derived year,"0000", ornull);updateItemDatesMetadata()delegates to it and writes only when the stored value differs — skipping the write only whendc.date.issuedalready holds exactly one value equal to the derived one (multi-valued fields are still normalized to the single derived value). Kept dtq-dev'slog.debugfor the empty-approximate-date case (the WARN→DEBUG change already landed on dtq-dev via84e9f3a2c2; the source commit still hadlog.warn, which is why the cherry-pick conflicted here).ItemServiceImpl.update()— callclarinItemService.updateItemDatesMetadata()inside the existingisMetadataModified()guard, so the date is derived and persisted only when metadata actually changes (PATCH/PUT).ItemConverter.convert()— remove the write-on-read call; instead overridedc.date.issuedon theItemRestDTO from the derived value viaMetadataRest#put(display-only, honouringmetadataExposureService.isHidden). No entity mutation, no DB write.Review updates (
b238a0b397)Both Copilot review comments addressed and unit coverage added:
ClarinItemServiceImpl) — skip the write only for a single already-derived value; a multi-valueddc.date.issuedis still collapsed to the single derived value (was skipping on a first-value match).ItemConverter) — build the derived value viaMetadataRest#put(place normalization +Arrays.asListsemantics, consistent with other fields) instead ofgetMap().put(Collections.singletonList(...)); dropped the manualsetPlace(0)and the now-unusedCollectionsimport.ClarinItemServiceImplTest(9 pure-Mockito unit tests) covering derivation (empty / blank / non-numeric / sequence / single) and the skip/normalize guard.Validation
The no-write property is covered structurally (the converter no longer calls
updateItemDatesMetadata) plus the unit-level skip/normalize guard. Full-context runtime (docker / DSpace test-kernel) is not bootstrappable in this environment. CI exercises the approximate-date display path viaItemRestRepositoryIT— note those ITs assert the displayed value; they are not a dedicated "no DB write on GET" assertion.Risk & rollback
Touches core CLARIN item serialization and
ItemServiceImpl.update(). For the standard read/display path, behaviour is preserved: items withlocal.approximateDate.issuedstill show the deriveddc.date.issuedin REST responses, and the value is now persisted on metadata change (previously the read-path write was always rolled back, so nothing was ever persisted). Regression focus: approximate-date handling. Revert = single commit.Behaviour notes
local.approximateDate.issuedpresent it is authoritative: any metadata-modifyingupdate()(PATCH/PUT) derives and overwritesdc.date.issued, so a client-supplieddc.date.issuedis replaced. This matches what the REST API already displayed."0000".Notes / assumptions
Conflict resolution kept dtq-dev-side changes (the
log.debugdowngrade). The four-file split matches the source commit's intent; the display-only override reuses the shared derivation rather than duplicating parsing.