DSpace9/copy the assetstore during import - #1452
Merged
Merged
Conversation
The dump restores the bitstream rows (internal_id, size_bytes, checksum) but not the files, so /dspace/assetstore stayed empty on the v9 stack and every bitstream download returned 500 - including tiny licence files, so it was not route-specific. It also broke OAI silently. SpecialItemService.getUploadedMetadata() reads the authored CMDI from the METADATA bundle, and its catch block returns an empty Document, so an unreadable file degrades to the generated CMDI profile rather than failing. That surfaced only as six mismatching OAI records in the REST test suite and took days to attribute. dev-5 already does this copy in the import-db composite action; the v9 stack had no equivalent step. Two deliberate differences from dev-5's version: - The copy runs before `dspace oai import -c`. Copying the files and even restarting the container is not enough - the records keep serving the generated substitute until that cache is cleared. - No `chown -R dspace:dspace`. The v9 runtime stage is FROM eclipse-temurin with no USER directive, so there is no `dspace` user (the one in the Dockerfile exists only in the build stage) and the webapp runs as root, which can both read and write root-owned files. dev-5's command fails outright here with "chown: invalid user: 'dspace:dspace'". The step verifies the result inside the container and fails if the assetstore is still empty, so a silent recurrence is not possible. ASSETSTORE_PATH can be set to an empty string to import without bitstream files. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Kasinhou
changed the title
Drop the extra guards (missing-path / basename / still-empty checks) and the long comment, keeping just the empty-check + docker cp that dtq-dev's import-db action uses for v7. Also shorten the ASSETSTORE_PATH input description. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the deploy.yml GitHub Actions workflow to copy the DSpace 8603 assetstore from the dev machine into the running DSpace container during the import-8603 job, aiming to ensure bitstream files exist after restoring a SQL dump (so downloads and OAI-derived metadata match production).
Changes:
- Add a new
ASSETSTORE_PATHinput for bothworkflow_callandworkflow_dispatch(defaulting to/opt/dspace-envs/8603/assetstore). - Export
ASSETSTORE_PATHinto theimport-8603job environment. - Insert a new “copy assetstore” step after restarting the DSpace container and before creating the administrator.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The v9 import restores the SQL dump, which brings the bitstream rows but not the files. Without copying the assetstore,
/dspace/assetstorestays empty — every bitstream download returns 500, and OAI silently serves generated CMDI instead of the authored files.This adds an
ASSETSTORE_PATHinput and acopy assetstorestep toimport-8603, doing the samedocker cpthat dtq-dev'simport-dbaction does for v7.Notes
chown(unlike v7): the v9 runtime image has nodspaceuser and runs as root, so v7'schown -R dspace:dspacefails withinvalid user./opt/dspace-envs/8603/assetstoremust exist on dev-6, otherwisedocker cpfails. LeaveASSETSTORE_PATHempty to import without bitstream files.