[schmitch]

Fixes #642

An import found what to copy through Nexus 3's REST API, so it could only import from a Nexus. It now reads the source through the format's own protocol, using the format plugins' upstream conversations, the same ones proxy repositories hold. A hosted repository can therefore import from Nexus, Artifactory, ProGet, BaGet, Harbor, a distribution registry or another Forge.

NuGet: a v3 feed

• IDs: the feed's search, with an empty query, prerelease and SemVer 2.0.0. A new UpstreamOp::Packages { skip, take } walks it. The proxy's Search stays capped at nuget.org's offset of 3,000; the walk is not.
• Walk end: the walk stops at an empty page, not at totalHits. Against a real Nexus 3.70, totalHits is the number of hits in the returned page (take=1 → 1; take=100 → all 5). If the search ignores skip and returns the same page again, the import stops with an error.
• source_total: the preflight reads the first page of 100. The scan then raises the total to as many packages as it walked.
• Versions and download: from PackageBaseAddress, which lists unlisted versions too. The .nupkg is downloaded from the same resource.
• Publish time: published from the registration. A year ≤ 1900 (nuget.org's 1900-01-01) is no publish time.
• Checksums: a version is verified against the registration's inline packageHash (SHA-512) where one is listed. Forge and nuget.org list one; Nexus does not. A re-run skips a version Forge holds with that hash without downloading it; without a hash, the version is downloaded and compared. New: VersionMetadata.sha512 and HostedWrite::package_sha512.

Docker: a registry's catalog

• Images: from /v2/_catalog, read through a new UpstreamOp::ImageCatalog. The docker format now shares one paged listing for tags and the catalog.
• Tags: from the tag lists.
• Referrers: after a tag is copied, the referrers API lists its manifest's referrers, and they become manifest items. A registry without the API keeps referrers under tags (cosign's sha256-<d>.sig, the referrers tag schema's sha256-<d>); those are copied like any other tag. This replaces the scan of Nexus' asset listing.

Named imports

names lists the package IDs or images to copy, for a feed without a search, packages Nexus' search misses, or a registry without a catalog. A named ID the source lacks becomes a failed item that says so.

What the protocols do not carry

• An imported version with no last download counts as downloaded at its import, so not_downloaded_for_days does not delete it right after.
• A tag is created at its import.

Neutral names

• crate::nexus_import is now crate::import, and the Nexus REST client is gone.
• Migration 0007 renames nexus_import / nexus_import_item (with their constraints, indexes and sequences) to repository_import / repository_import_item.
• Column changes: base_url → url, nexus_version → source_server. New columns names and source_total. assets_seen and bytes_seen are dropped. scan_listing takes search, catalog or names.
• The password's AEAD binding keeps the table name nexus_import, so a password sealed before the upgrade still opens.
• An import stored before the upgrade keeps its Nexus url and source_repository. The worker reads <url>/repository/<name>/index.json, or the registry, through the protocol. If it was scanning, its scan starts over.
• The previous request form (url + repository [+ registry_url]) is still accepted and stored as the derived source URL.
• The admin view and the web UI drop the Nexus wording: Import, a feed or registry URL, an optional list of package IDs or images.

Docs

• docs/import.md is the guide.
• docs/nexus-import.md stays as a short Nexus page, linked from it: URLs, the 10,000-package search cap and its page-sized totalHits, privileges, and imports stored before 2.2. The released 2.0 changelog links that path, and a change never edits CHANGELOG.md.
• Changelog fragments: 642.feature.md and 642.upgrade.md.

Tests

• crates/server/tests/import.rs: rewritten against a mock source. It serves a NuGet v3 feed and a registry under Nexus' paths, with Basic auth, and can switch off the search, the catalog or the referrers API, or report page-sized totals like Nexus. 12 tests:
  • bytes, dates and the 1900 rule;
  • the second run, which downloads only versions the feed lists no hash for;
  • the legacy request form;
  • retention after an import;
  • kill-and-resume through Nexus-style paging;
  • conflicts and the dry run, which downloads nothing;
  • SHA-512 mismatch and retry;
  • cancel, resume and delete;
  • preflight refusals;
  • named packages and images without search or catalog;
  • Docker: catalog paging, an index with children, a cosign tag, an API referrer, and a stray manifest left out;
  • a referrer through the referrers tag schema;
  • a legacy row that lists and resumes;
  • the password sealed at rest.
• Mutation check: without the "downloaded at import" rule, the retention test fails (deleted: 1).
• crates/db/tests/migrations.rs: migrates to 0006, stores imports in the old nexus_import shape with an item and a password sealed as before, then migrates on. Rows, items, listing, cursor and the password come through; one active import per repository is still enforced; nothing keeps a nexus name.
• e2e (crates/e2e/tests/import.rs), run locally:
  • real Nexus 3.70.1: NuGet through /repository/nuget-hosted/index.json, with publish dates equal to the registration's. Docker through the connector's catalog with 105 images, a cosign signature, and an oras attach-style referrer through the tag schema. A second run in the legacy request form.
  • registry:2: catalog, signature and tag-schema referrer; a second run is all identical.
  • Forge to Forge: a second run downloads nothing, measured by the source's download counts.
• NuGet unit test: the import walk reads past offset 3,000.
• Also run:
  • cargo clippy --workspace --all-targets --all-features -D warnings (offline);
  • server --lib, registry, plugin_contract, proxy_search, group_proxy;
  • the docker, npm, nuget and plugin-api crates;
  • cargo xtask changelog --check and cargo xtask docs --check;
  • web tsc, all 524 vitest tests, and build.

Notes

• Migration number: 0007 leaves room for Auth realms: accept client passwords over HTTP Basic per realm #708's 0006. They touch different tables and apply in either order.
• Not covered: an unlisted NuGet version is imported listed, as before; the old Nexus import did not carry the listing state either. Whether an import from a Nexus should also read its REST API for checksums, last downloads and uploaders is Decide whether an import from a Nexus also reads Nexus' REST API for checksums, last downloads and uploaders #645.

---

Generated by Claude Code