[XiaoHongbo-Hope]

Purpose

A batch whose windows reference one video uses only one decoder, leaving the shared decode pool idle. Split those windows across independent cached decoders to improve LeRobot throughput for single-video datasets such as PushT.

• Reuse the shared worker pool and existing cache limits; each decoder owns its stream.
• Keep sample order, exact decoding and output isolation. Group equivalent windows and automatically reduce the decoder count when splitting would add more than 50% duplicate frame requests.
• Add read.video.decoders-per-payload (default: 4; 1 disables splitting). Multi-payload batches keep their existing scheduling. No public constructor changes.

Built on master after #10703; the implementation is a single commit.

PushT prototype results (three paired runs, median samples/s):

Storage	Paimon baseline	Paimon, 4 decoders	Official Lance
Local	5,727	6,722	6,203
OSSFS	5,513	6,699	5,942

Both OSSFS stores were remounted and kernel caches cleared before each run. Eight workers, batch 32, eight-frame windows, uint8, CPU, no transforms/model; 10 warmup and 500 timed batches. These are warmed throughput results, not cache-free OSS throughput. Lance: lerobot-lancedb commit 40bcb659a52d, pylance 13.0.0. Paimon used an integration branch with #10701/#10703/#10704 and startup/allocation optimizations; these are prototype measurements, not benchmarks of this PR head.

On OSSFS, four decoders raised CPU use from about 8.7 to 12.2 cores and PSS from 3.32 to 3.42 GiB. Fixed-workload time was unchanged versus baseline (14.05 → 14.08 s); two decoders were faster for that short task (13.33 s). Additional decoders can increase startup cost and repeated reads.

Follow-up OSSFS checks use this implementation on integrated base 5c23609a04 (three paired runs, 10 warmup + 100 timed batches; other settings unchanged):

Dataset	Paimon, 1 decoder	Paimon, 4 decoders	Official Lance
ALOHA	217.2	236.8	157.3
Koch	332.6	318.6	237.2

ALOHA exercised splitting in all workers: +9.0% throughput over baseline, CPU 24.7 → 34.8 cores, PSS 9.72 → 9.91 GiB. Koch never exercised splitting because its batches contain multiple payloads; measured throughput was 4.2% lower and total time essentially unchanged (24.80 → 24.87 s), so no incremental benefit is claimed there. Both candidate throughput results exceeded Lance in all three paired runs. Candidate/Lance total times: ALOHA 28.77/36.79 s; Koch 24.87/28.49 s. Koch first batch remained slightly slower (13.82/13.55 s).

Tests

• 93 tests and 115 subtests passed; 29 optional-dependency tests skipped.
• Adaptive splitting keeps equivalent windows on one decoder and bounds overlapping-window frame requests.
• Real PyAV single-video output matches serial decoding. Covers duplicate/unsorted windows, uint8/float output, shared-memory isolation, decoder reuse, cache limits/eviction, pickle and existing fork/spawn cleanup.
• Local/OSSFS comparisons passed sample-order and separate full-tensor checks, with native reads verified in every Paimon worker. ALOHA/Koch follow-up: 18 timed runs and 6 full-tensor validation runs passed.
• Flake8, Python 3.6 compilation and git diff --check passed.

[JingsongLi]

+1