feat(concat): add gvl.concat for merging datasets along regions or samples - #346

Merged

d-laub merged 18 commits into

mcvickerlab:mainfrom

d-laub:worktree-dataset-concat-spec

Aug 1, 2026

Commits on Jul 31, 2026

docs(spec): #334 dataset concatenation design

d-laub
and
claude
committed
b528951
docs(spec): #334 correct IO benchmark — memmap reads are slow on NFS too

d-laub
and
claude
committed
9b23c53
docs(plan): #334 gvl.concat implementation plan

d-laub
and
claude
committed
6727f93
feat(concat): provenance map and run coalescing for dataset merge
d-laub
committed
c1af2fd
test(concat): shard-builder fixtures for the concat oracle
d-laub
committed
8ee7894
feat(concat): preconditions and variants.arrow fingerprint
d-laub
committed
8688762
feat(concat): buffered streaming IO primitives
d-laub
committed
6533d32
test(concat): cover link_or_copy_buffered hardlink and EXDEV fallback
d-laub
committed
b840ed5
fix(concat): resolve has_dosages/fingerprint from the real store, enforce one variant source
d-laub
committed
74f598d
feat(concat): gvl.concat entry point for PGEN/VCF-backed datasets
d-laub
committed
11af7fc
test(concat): cover regions.npy byte-identity and sample-axis self-consistency
d-laub
committed
5f24df7
feat(concat): tracks, annot tracks, and svar/svar2 backends
d-laub
committed
c03cca2
fix(concat): verify annot-track data agrees before copying on sample axis
d-laub
committed
725b735
feat(concat): verify variants.arrow fingerprint at open
d-laub
committed
ed0626a
docs(concat): document gvl.concat and parent-PGEN cohort workflow
d-laub
committed
087b252
fix(concat): close tie-order/schema gaps and drop full-array copies

d-laub
and
claude
committed
19fbaa2
fix(concat): scope duplicate-coordinate guard to axis="regions"
d-laub
committed
c974c2a

Commits on Aug 1, 2026

Merge remote-tracking branch 'origin/main' into worktree-dataset-concat-spec
d-laub
committed
c63952b

Navigation Menu

Uh oh!

feat(concat): add gvl.concat for merging datasets along regions or samples - #346

feat(concat): add gvl.concat for merging datasets along regions or samples#346
d-laub merged 18 commits into
mcvickerlab:mainfrom
d-laub:worktree-dataset-concat-spec

Commits on Jul 31, 2026

docs(spec): #334 dataset concatenation design

docs(spec): #334 correct IO benchmark — memmap reads are slow on NFS too

docs(plan): #334 gvl.concat implementation plan

feat(concat): provenance map and run coalescing for dataset merge

test(concat): shard-builder fixtures for the concat oracle

feat(concat): preconditions and variants.arrow fingerprint

feat(concat): buffered streaming IO primitives

test(concat): cover link_or_copy_buffered hardlink and EXDEV fallback

fix(concat): resolve has_dosages/fingerprint from the real store, enforce one variant source

feat(concat): gvl.concat entry point for PGEN/VCF-backed datasets

test(concat): cover regions.npy byte-identity and sample-axis self-consistency

feat(concat): tracks, annot tracks, and svar/svar2 backends

fix(concat): verify annot-track data agrees before copying on sample axis

feat(concat): verify variants.arrow fingerprint at open

docs(concat): document gvl.concat and parent-PGEN cohort workflow

fix(concat): close tie-order/schema gaps and drop full-array copies

fix(concat): scope duplicate-coordinate guard to axis="regions"

Commits on Aug 1, 2026

Merge remote-tracking branch 'origin/main' into worktree-dataset-concat-spec