Repository navigation
Are the segments of a block ordered in time? #1937
Description
I was working under the assumption that the segments of a block are ordered in time, but looking at the documentation I can't find anything that says so. The closest is the Segment docstring, which says a segment holds data "sharing a common clock (time basis)" but nothing about how two segments relate.
In SpikeInterface, however, we seem to assume that they follow each other. Sorters that do not handle several segments ask the user to call si.concatenate_recordings, which joins the segments end to end in index order and drops their times:
https://github.com/SpikeInterface/spikeinterface/blob/93c612f51cca09ecafe2c23a283132cf0ad57869/src/spikeinterface/sorters/basesorter.py#L141-L145
I have also been relying on this in NeuroConv, where the units table joins the spike times of each unit across all segments into one array:
https://github.com/catalystneuro/neuroconv/blob/128578d4a78de3e115500cc8d349a6be44dcd4a7/src/neuroconv/tools/spikeinterface/spikeinterface.py#L2871-L2883
The purpose of this issue is that we agree on this. It can be closed once we make a decision and fix the documentation.
This is relevant because of Blackrock. The Blackrock clock can be reset in the middle of a recording, and right now BlackrockRawIO returns the part after the reset as a second segment that overlaps the first one in time. This is blackrock/segment/ResetCorrect from ephy_testing_data:
from neo.rawio import BlackrockRawIO
reader = BlackrockRawIO(filename="ephy_testing_data/blackrock/segment/ResetCorrect/reset", nsx_to_load=2)
reader.parse_header()
for seg_index in range(reader.segment_count(0)):
t_start = reader.segment_t_start(0, seg_index)
t_stop = reader.segment_t_stop(0, seg_index)
print(f"{seg_index=}, {t_start=:.4f}, {t_stop=:.4f}")seg_index=0, t_start=0.0000, t_stop=4.0200
seg_index=1, t_start=0.0032, t_stop=3.9842
I am wondering how to handle this better. I see two options:
- If segments are meant to be ordered in time: a reset starts a new block, and the documentation says that the segments of a block follow each other without overlapping.
- If they are not: a reset stays a new segment, the documentation says that segments can overlap, and code downstream (like the NeuroConv case above) has to stop assuming that they follow each other.
What do you think @alejoe91, @samuelgarcia and @zm711 ?
Activity
cboulay commentedon Oct 28, 2026
One data point from Blackrock spec 3.0 (PTP) files written by Central. Not every step back in timestamps is a clock reset, and treating them as resets is what produces most of the overlapping segments.
We found packets whose timestamps are correct but whose place in the file is wrong, most likely UDP datagrams reassembled out of order on the recording PC:
.ns6, one sample written 51 positions early;.ns3, one sample written 2 positions early;.nev, spikes written up to a few ms after later ones.Sorted by timestamp, they give a perfect sampling grid, with no duplicates and nothing missing. Today neo reads each of these as a gap or a reset. In a small test file this produced 4 segments instead of 2, and 7,681 of 31,752 spikes were lost. Comments stamped ahead of the spikes around them (#1924) are another source of false resets.
So I'd argue that before we decide what a reset means for blocks and segments, the reader should only call something a reset when it can't be explained as packets out of order. We have a change that reads these in timestamp order within a 10 ms window; anything larger, or overlapping, stays a reset. I'll open it as a PR stacked on #1924. With false resets out of the way, a true reset like
ResetCorrectis rare and unambiguous. My preference is that it starts a new block, so the segments of a block stay ordered and never overlap, which matches what SpikeInterface and NeuroConv already assume.h-mayorquin commentedon Oct 28, 2026
I agree that we need some tolerance and that not everything out of order is a reset (as you found for the nev in #1924). Initially I was thinking about adding yet another parameter so users could specify this but on a second thought I think it is better to handle it internally.
But I think we should be very principled about it, otherwise we could make a release that leaves users without recourse. In that vein, why did you choose 10 ms as the backward tolerance? What would be a good principle to move forward? You know the format and the acquisition better than us, so if you are convinced that it is a good value I can settle for that.
cboulay commentedon Oct 28, 2026
In recent weeks, I have told my colleagues multiple times to "never sort Blackrock data! The order is correct; if anything, the timestamp is wrong due to a clock adjustment."
Then I found that Trellis data where we'd occasionally get a subset of channels feeding into the data stream with a bit of a delay. "Fine, I guess their hardware works differently," I thought.
Now, I'm seeing out-of-order packets in the data streams saved by Central. I still can't explain it. It's the first time I've seen it, and it's only coming from one Gemini Hub at one user's site. I can't reproduce it. Claude is telling me it's because the user's network card is spreading the incoming UDP packets over several CPU queues and can reassemble out-of-order. I didn't know that was a thing. Sounds like a hallucination.
And the out-of-order packets in the .ns3 stream happen every 1.111 seconds. Surely that's got to be a bug in the code, either the hub or Central, right?
Anyway, putting my existential crisis aside for a moment, these data exist and we should be able to load them. Supporting these data certainly complicates loading the files by stream. Unfortunately, I don't have much of a choice there because I have other data where I have a single .nev file that is 90 GB and several in the 40 GB range, and I'm curating these data in the cloud on job runners with only 5 GB RAM.
cboulay commentedon Oct 28, 2026
To come back to your question, I chose 10 msec because it had to be larger than 1 msec as that's the average duration of data that fits into a datagram when running at 30 kHz, and I don't think we'd see this if running slower than 30 kHz. The largest out-of-order jump I've seen so far has been in spiking data at ~ 3 ms. Claude suggested 7 ms. I rounded up.
cboulay commentedon Oct 28, 2026
One more thing worth making clear: the 10 msec window isn't the criterion, it's just a search bounds. The criteria are that the out of order samples must fit into a timestamp gap, and after reordering we restore the timestamp grid.