[ivonastojanovic]

Tachyon's binary format currently stores all samples in a single compressed stream. Each record contains its thread and interpreter IDs, timestamp, status, and stack change, so values such as the thread and interpreter IDs are written repeatedly across records. The existing RLE encoding avoids some of this when a stack repeats, and compression reduces it further, but the format still mixes fields with very different patterns in the same stream.

More importantly, this means replay has to decompress the complete sample stream before it can process the first sample. The temporary buffer therefore grows with the length of the capture, even though replay only needs a small part of the sample data at a time.

This PR changes the binary format to split samples into bounded, independently decodable blocks. Replay can decompress and process one block at a time, then reuse the same buffers for the next block. Within each block, sample fields are also separated into columns, allowing repeated values to be removed before compression and each type of data to use an encoding that better matches its structure.

This change focuses on profiles exported to regular files. Live streaming remains separate work in #145464. Regular files can keep shared string and frame tables at the end of the file, while a live stream cannot refer to tables that have not been sent yet. For streaming, we may instead need to include the required string and frame data in each block so that it can be decoded as soon as it arrives.

Implementation

The new file layout is:

header | sample block | sample block | ... | string table | frame table | stats | footer

Each sample block has its own header followed by six columns:

block header
    | stream table
    | stream IDs
    | timestamp residuals
    | status bitmaps
    | stack instructions
    | frame IDs

Samples keep their original order across these columns, but no longer need a complete record containing every field. Values shared across a block move into its header or tables, while the per-sample columns only store what changes. The block header contains the sample count, base timestamp, frame-ID width, status masks, and the size and encoding of each column.

Column	What it stores	What this avoids
Stream table	Each (thread_id, interpreter_id) pair used by the block, once	Repeating both full IDs in every record
Stream IDs	A small stream-table index for each sample	Carrying the full thread and interpreter IDs with each sample
Timestamp residuals	Difference from the expected timestamp for that stream	Storing larger timestamp differences alongside other fields
Status bitmaps	One bitmap for each status bit that varies	Writing one complete status byte per sample; common bits are stored once in the header
Stack instructions	How to repeat, change, or reuse a stack	Writing frames that are unchanged or already available in the block
Frame IDs	Referenced frame-table indices at the smallest width needed by the block	Using a larger representation for every frame ID

The writer first reserves space for the file header. It then collects samples into a block and starts a new one when the decoded column data reaches 1 MiB or the block reaches 4,096 unique (thread_id, interpreter_id) pairs. Once sampling finishes, it writes the shared string and frame tables, statistics, and footer, then seeks back to fill in the file header.

Strings and frames remain shared across the whole file, preserving deduplication across the complete capture. These tables still grow with the number of unique strings and frames, but temporary decoded sample data is now bounded by the size of one block rather than the total number of samples.

Timestamp and stack state are tracked independently for each stream. Timestamps are stored as residuals from the expected timestamp, so samples arriving at the normal interval usually store zero. Status bits shared by all samples in a block are stored once in the block header, while only varying bits need per-sample bitmaps.

Stacks use three instructions:

Instruction	Operands	Meaning
REPEAT	count	Reuse this stream's previous stack for count samples
REPLACE_SUFFIX	keep_count, new_count	Keep the common outer frames and append new_count frame IDs
REUSE	offset, count	Copy a complete stack already stored in the block's frame-ID column

REPEAT is tracked per stream, so samples from other streams can appear between repetitions. The other instructions either write only the frames that changed or reuse a complete stack already stored in the block. Frame IDs use between one and four bytes and are stored in byte planes to improve compression.

Each column is stored as raw bytes, a single repeated byte, or zstd level 3, whichever is smallest. The file-wide string and frame tables similarly choose between raw and zstd.

After discussing this with @pablogsal, we decided not to keep support for version 1 profiles since this change will be shipped in 3.16.

The public BinaryWriter, BinaryReader, and collector APIs remain unchanged, except that the get_stats() dictionaries now report reuse_records instead of pop_push_records, since the pop-push encoding no longer exists.

• Issue: Tachyon: use bounded columnar blocks for binary profiles #159207

[ivonastojanovic]

cc @maurycy

[ivonastojanovic]

I did some benchmarking while working on this, so leaving the results here for reference.

For the main comparison, I profiled test_list, test_dict, test_tokenize, test_exceptions, test_syntax, and test_threading with non-debug builds on an Apple Silicon Mac. The number of samples was slightly different between runs, so I compared bytes per sample instead of the final file size.

Compression	main	This PR	With compressed tables
None	2.4121	1.6643 (-31.0%)	N/A
zstd	0.3186	0.2787 (-12.5%)	0.2572 (-19.3%)

I also experimented with compressing the file-wide string and frame tables, so I included those numbers in the last column. It helps a bit more with zstd, but I left it out of this PR since the change is already pretty large. We can always add that separately.

I also tried two smaller programs with zstd to see how the format behaves with different workloads. One does JSON and zlib work across eight threads, so repeated stacks are interleaved across several streams. The other runs SQLite queries and processes the results in Python, so it mostly stays on one stream and moves between a small set of stacks.

Workload	main	This PR	With compressed tables
JSON/zlib threads	0.4659	0.2508 (-46.2%)	0.2652 (-43.1%)
SQLite queries	1.3747	1.0991 (-20.0%)	1.0896 (-20.7%)

The small differences between the two block-format results are just normal run-to-run variation. These are also bytes per sample.

I also checked a few synthetic stack patterns with zstd: one stable stack, stacks where only the inner frames change, complete stacks shared across threads, and a bank of randomly selected stacks.

Workload	main	This PR	With compressed tables
Stable stacks	1.6 KiB	1.4 KiB (-13.9%)	1.2 KiB (-28.6%)
Common outer frames	323.1 KiB	129.1 KiB (-60.0%)	108.7 KiB (-66.4%)
Shared complete stacks	518.2 KiB	125.2 KiB (-75.8%)	50.8 KiB (-90.2%)
Random stack bank	734.6 KiB	3,506.1 KiB (+377.3%)	3,385.9 KiB (+360.9%)

The random stack case is the obvious regression. The frame IDs do not compress particularly well, and repeated stacks can end up in different blocks, so we lose the ability to reuse them across block boundaries.

The main thing I wanted to improve with this change was replay memory, though. With the previous format, peak memory grew from 31.0 MiB at 100,000 samples to 59.4 MiB at one million samples. With the block reader, it stayed at around 26.3 MiB.