gvl.write hardlinks variants.arrow into the dataset with no integrity guard

[d-laub]

Found while designing dataset concatenation (#334).

_write_from_pgen publishes the genoray variant index into the dataset via _link_or_copy (python/genvarloader/_dataset/_write.py:608), which hardlinks index.arrow → genotypes/variants.arrow and only falls back to a copy on EXDEV. The dataset therefore aliases an inode it does not own, with no recorded fingerprint.

Why this is (mostly) safe today

Worth stating, because it argues for keeping the hardlink rather than removing it:

• gvl never mmaps variants.arrow — _haps.py:121 and _haps.py:185 both pass memory_map=False. So there is no SIGBUS exposure from this file; the realistic failure is a torn/inconsistent read.
• genoray rewrites both the PGEN index (_pgen.py:1125) and the VCF index (_vcf.py:1095) through atomic_write_path — temp file + os.replace. A rename does not touch the old inode, so the dataset's hardlink keeps pointing at the intact old index even when genoray regenerates it.

The actual gap

A third-party tool that truncate-rewrites the index in place (open(path, "wb")), or any out-of-band edit, silently changes the dataset's variants.arrow underneath it. variant_idxs then resolve against a different variant table and reads are wrong with no error.

This is exactly the problem svar_link / svar2_link already solve for the .svar/.svar2 stores: reference a large external artifact, record a cheap fingerprint, verify at open.

Proposal

Record a bounded fingerprint for variants.arrow in metadata.json and verify it in Dataset.open, raising a clear ValueError on mismatch (naming expected vs observed, as the svar path does).

Reuse the existing idiom rather than inventing one — _fasta_cache.Fingerprint is blake2b over a bounded FINGERPRINT_WINDOW (1 MiB) plus size_bytes, so the check stays O(1) on a multi-GB index.

Keep the hardlink. It is the correct zero-copy choice and matches the svar_link precedent; the fix is the missing guard, not the aliasing.

Notes

• Needs a format-version bump and a tolerant path for datasets written before the field exists (absent fingerprint → skip verification, as with the pre-0.25.0 svar_link migration).
• concat (Reuse variant index across cohorts with identical variant space and support merging non-overlapping genomic shards #334) will record this fingerprint from the outset.