diff --git a/InternalDocs/profiling_binary_format.md b/InternalDocs/profiling_binary_format.md index a3d86b1a4ab5693..b467514ca228e3d 100644 --- a/InternalDocs/profiling_binary_format.md +++ b/InternalDocs/profiling_binary_format.md @@ -41,7 +41,7 @@ The file consists of five required sections and one optional extension: | Header | 64 bytes (fixed) +------------------+ Offset 64 | | -| Sample Data | Variable size (optionally compressed) +| Sample Data | Blocks of columns (optionally compressed) | | +------------------+ string_table_offset | String Table | Variable size @@ -60,8 +60,8 @@ so these tables must be built incrementally and written at the end. The header comes first so readers can quickly validate the file and locate the metadata tables. The sample data follows immediately, allowing the writer -to stream samples directly to disk (or through a compression stream) without -buffering the entire dataset in memory. +to write each block of samples as soon as it is full without buffering the +entire dataset in memory. The string and frame tables are placed after sample data because they grow as new unique entries are discovered during profiling. By deferring their @@ -120,86 +120,141 @@ network streams. ## Sample Data Sample data begins at offset 64 and extends to `string_table_offset`. Samples -use delta compression to minimize redundancy when consecutive samples from the -same thread have identical or similar call stacks. +are grouped into blocks of at most 1 MiB of decoded data. Within a block, +samples use delta compression to minimize redundancy when consecutive samples +from the same thread have identical or similar call stacks. Each block can be +decoded on its own once the string and frame tables are loaded, so a reader +only needs to decode one block at a time. + +### Block Format + +A block stores values of the same kind together, in six columns: + +``` ++------------------------+ +| Block header | 21 to 88 bytes ++------------------------+ +| STREAM_TABLE | (thread_id, interpreter_id) pairs ++------------------------+ +| SAMPLE_STREAM_IDS | One stream index per sample ++------------------------+ +| TIMESTAMP_RESIDUALS | One timestamp residual per sample ++------------------------+ +| STATUS_BITMAPS | One bitmap per varying status flag ++------------------------+ +| STACK_INSTRUCTIONS | Stack encodings ++------------------------+ +| FRAME_IDS | Frame table indices ++------------------------+ +``` + +**Block header:** +``` ++---------------------+-----------+----------------------------------------+ +| magic | 4 bytes | "TBLK" | +| sample_count | varint | Number of samples in this block | +| base_timestamp | varint | Timestamp of the first sample | +| frame_id_width | 1 byte | Bytes per frame index (1 to 4) | +| common_status_flags | 1 byte | Status bits set in every sample | +| status_bitmap_flags | 1 byte | Status bits stored as bitmaps | +| columns | varies | One column descriptor per column | ++---------------------+-----------+----------------------------------------+ +``` + +**Column descriptor:** +``` ++---------------------+-----------+----------------------------------------+ +| decoded_size | varint | Size of the decoded column | +| encoding | 1 byte | 0=raw, 1=zstd, 2=constant | +| stored_size | varint | Compressed size (zstd only) | ++---------------------+-----------+----------------------------------------+ +``` + +A raw column is stored as is, and a constant column is a single byte repeated +`decoded_size` times, so neither needs a stored size. The number of streams +and frame indices in a block follows from the decoded sizes of +`STREAM_TABLE` and `FRAME_IDS`. + +A block holds at most 4,096 streams. There is no separate limit on the number +of samples: the stream index and timestamp columns need at least one byte per +sample, so the 1 MiB limit already bounds it. ### Stack Encoding Types -Each sample record begins with thread identification, then an encoding byte: +Each thread's stack is reconstructed from the previous stack of the same +thread in the block. The first sample of a thread in a block starts from an +empty stack. `STACK_INSTRUCTIONS` holds one of these encodings for every +sample not covered by a `REPEAT`: | Code | Name | Description | |------|------|-------------| | 0x00 | REPEAT | RLE: identical stack repeated N times | -| 0x01 | FULL | Complete stack (first sample or no match) | -| 0x02 | SUFFIX | Shares N frames from bottom of previous stack | -| 0x03 | POP_PUSH | Remove M frames from top, add N new frames | +| 0x01 | REPLACE_SUFFIX | Keeps N outer frames of the previous stack, adds M new frames | +| 0x02 | REUSE | Complete stack stored earlier in the block | + +Stacks hold at most 256 frames. The writer keeps the innermost 256 frames of +deeper stacks. -### Record Formats +### Stack Encoding Formats + +All fields are varints. Stacks are written outermost frame first. **REPEAT (0x00) - Run-Length Encoded Identical Stacks:** ``` +-----------------+-----------+----------------------------------------+ -| thread_id | 8 bytes | Thread identifier (uint64, fixed) | -| interpreter_id | 4 bytes | Interpreter ID (uint32, fixed) | -| encoding | 1 byte | 0x00 (REPEAT) | +| encoding | varint | 0x00 (REPEAT) | | count | varint | Number of samples in this RLE group | -| samples | varies | Interleaved: [delta: varint, status: 1]| -| | | repeated count times | +-----------------+-----------+----------------------------------------+ ``` -The stack is inherited from this thread's previous sample. Each sample in the -group gets its own timestamp delta and status byte, stored as interleaved pairs -(delta1, status1, delta2, status2, ...) rather than separate arrays. +The stack is inherited from this thread's previous sample, for this sample +and the thread's next `count - 1` samples. Samples from other threads can +appear in between: the reader keeps a remaining count for each thread and +reads no encoding for that thread until the count reaches zero. Each sample +in the group keeps its own timestamp and status in the other columns. -**FULL (0x01) - Complete Stack:** +**REPLACE_SUFFIX (0x01) - Shared Outer Frames:** ``` +-----------------+-----------+----------------------------------------+ -| thread_id | 8 bytes | Thread identifier (uint64, fixed) | -| interpreter_id | 4 bytes | Interpreter ID (uint32, fixed) | -| encoding | 1 byte | 0x01 (FULL) | -| timestamp_delta | varint | Microseconds since thread's last sample| -| status | 1 byte | Thread state flags | -| stack_depth | varint | Number of frames in call stack | -| frame_indices | varint[] | Array of frame table indices | +| encoding | varint | 0x01 (REPLACE_SUFFIX) | +| keep_count | varint | Outer frames kept from previous stack | +| new_count | varint | Inner frames read from FRAME_IDS | +-----------------+-----------+----------------------------------------+ ``` -Used for the first sample from a thread, or when delta encoding would not -provide savings. +Used when a call or return changed the inner part of the stack: the outer +frames are kept, and `new_count` new frames are read from `FRAME_IDS`. A +complete stack, such as the first sample of a thread, uses `keep_count` 0. -**SUFFIX (0x02) - Shared Suffix Match:** +**REUSE (0x02) - Repeated Complete Stack:** ``` +-----------------+-----------+----------------------------------------+ -| thread_id | 8 bytes | Thread identifier (uint64, fixed) | -| interpreter_id | 4 bytes | Interpreter ID (uint32, fixed) | -| encoding | 1 byte | 0x02 (SUFFIX) | -| timestamp_delta | varint | Microseconds since thread's last sample| -| status | 1 byte | Thread state flags | -| shared_count | varint | Frames shared from bottom of prev stack| -| new_count | varint | New frames at top of stack | -| new_frames | varint[] | Array of new_count frame indices | +| encoding | varint | 0x02 (REUSE) | +| offset | varint | Index of the first frame in FRAME_IDS | +| count | varint | Number of frames in the stack | +-----------------+-----------+----------------------------------------+ ``` -Used when a function call added frames to the top of the stack. The shared -frames from the previous stack are kept, and new frames are prepended. +Used when a complete stack was already written earlier in the same block, +for example by another thread running the same code. The frames are copied +from `FRAME_IDS` instead of being stored again. -**POP_PUSH (0x03) - Pop and Push:** -``` -+-----------------+-----------+----------------------------------------+ -| thread_id | 8 bytes | Thread identifier (uint64, fixed) | -| interpreter_id | 4 bytes | Interpreter ID (uint32, fixed) | -| encoding | 1 byte | 0x03 (POP_PUSH) | -| timestamp_delta | varint | Microseconds since thread's last sample| -| status | 1 byte | Thread state flags | -| pop_count | varint | Frames to remove from top of prev stack| -| push_count | varint | New frames to add at top | -| new_frames | varint[] | Array of push_count frame indices | -+-----------------+-----------+----------------------------------------+ -``` -Used when the code path changed: some frames were popped (function returns) -and new frames were pushed (different function calls). +For example, two threads produce four samples: + +| Sample | Thread | Stack | Encoding | +|--------|--------|----------------|----------------------| +| 0 | A | `[4, 21, 37]` | `REPLACE_SUFFIX 0 3` | +| 1 | B | `[4, 21, 37]` | `REUSE 0 3` | +| 2 | A | `[4, 21, 52]` | `REPLACE_SUFFIX 2 1` | +| 3 | A | `[4, 21, 52]` | `REPEAT 1` | + +`FRAME_IDS` holds only `[4, 21, 37, 52]`. ### Thread and Interpreter Identification +Each `(thread_id, interpreter_id)` pair used in a block is stored once in +`STREAM_TABLE`, as an 8-byte thread ID followed by a 4-byte interpreter ID. +`SAMPLE_STREAM_IDS` then holds one varint per sample: the index of its pair +in the table. Samples are kept in arrival order, so samples from different +threads are interleaved. + Thread IDs are 64-bit values that can be large (memory addresses on some platforms) and vary unpredictably. Using a fixed 8-byte encoding avoids the overhead of varint encoding for large values and simplifies parsing @@ -226,23 +281,30 @@ Multiple flags can be set simultaneously (e.g., a thread can hold the GIL while also running on CPU). Analysis tools use these to filter samples or visualize thread states over time. +Most flags rarely change within a block, so the status bytes are stored as +bit planes. Bits set in every sample are recorded once in +`common_status_flags`. Each bit in `status_bitmap_flags` has a bitmap of +`ceil(sample_count / 8)` bytes in `STATUS_BITMAPS`, in increasing bit order, +with sample `i` in bit `i % 8` of byte `i / 8`. Bits that never occur take no +space. The reader rebuilds each status byte from `common_status_flags` and +the sample's bit in each bitmap. + ### Timestamp Delta Encoding Timestamps use delta encoding rather than absolute values. Absolute timestamps in microseconds require 8 bytes each, but consecutive samples from the same thread are typically separated by the sampling interval -(e.g., 1000 microseconds), so the delta between them is small and fits -in 1-2 varint bytes. The writer tracks the previous timestamp for each -thread separately. The first sample from a thread encodes its delta from -the profiling start time; subsequent samples encode the delta from that -thread's previous sample. This per-thread tracking is necessary because -samples are interleaved across threads in arrival order, not grouped by -thread. - -For REPEAT (RLE) records, timestamp deltas and status bytes are stored as -interleaved pairs (delta, status, delta, status, ...) - one pair per -repeated sample - allowing efficient batching while preserving the exact -timing and state of each sample. +(e.g., 1000 microseconds), so each timestamp is expected to be the thread's +previous timestamp plus the interval, and only the difference from that +expected timestamp is stored. The writer tracks the previous timestamp for +each thread separately. The expected timestamp for a thread's first sample +in a block is the block's `base_timestamp`. This per-thread tracking is +necessary because samples are interleaved across threads in arrival order, +not grouped by thread. + +Each residual is a signed 64-bit zigzag varint. A sample taken exactly on +schedule has a residual of 0 and takes one byte, while timestamps are still +preserved exactly. ### Frame Indexing @@ -251,15 +313,25 @@ rather than inline data. This provides massive space savings because call stacks are highly repetitive: the same function appears in many samples (hot functions), call stacks often share common prefixes (main -> app -> handler -> ...), and recursive functions create repeated frame sequences. -A frame index is typically 1-2 varint bytes. Inline frame data would be -20-200+ bytes (two strings plus a line number). For a profile with 100,000 -samples averaging 30 frames each, this reduces frame data from potentially +A frame index is typically 1-2 bytes. Inline frame data would be 20-200+ +bytes (two strings plus a line number). For a profile with 100,000 samples +averaging 30 frames each, this reduces frame data from potentially gigabytes to tens of megabytes. -Frame indices are written innermost-first (the currently executing frame -has index 0 in the array). This ordering works well with delta compression: -function calls typically add frames at the top (index 0), while shared -frames remain at the bottom. +`FRAME_IDS` stores the indices read by `REPLACE_SUFFIX`, using the smallest +byte width that fits the largest index in the block. The bytes are split into +planes, low bytes first, so similar bytes sit together for compression: + +``` +indices: 0x1234 0x5678 0x9abc +low-byte plane: 0x34 0x78 0xbc +high-byte plane: 0x12 0x56 0x9a +``` + +Frame indices are written outermost-first (the outermost frame has index 0 +in the array). This ordering works well with delta compression: function +calls typically change frames at the end of the array, while shared frames +remain at the start. ## String Table @@ -453,29 +525,34 @@ bytes. Zigzag encoding interleaves positive and negative values: ``` This ensures small-magnitude values (whether positive or negative) encode -in few bytes. +in few bytes. Frame table deltas use 32-bit zigzag values and timestamp +residuals use 64-bit zigzag values. ## Compression -When compression is enabled, the sample data region contains a zstd stream. +The header's compression type controls whether zstd may be used. Each column +of each block is encoded separately, using whichever representation is +smallest: + +- **constant**, when every byte of the column is the same, such as the stream + IDs of a single-threaded block or residuals of on-time samples; +- **zstd**, when compression is enabled and the compressed column plus its + stored-size varint is smaller than the raw column; +- **raw** otherwise. + The string table, frame table, and footer remain uncompressed so readers can access metadata without decompressing the entire file. A tool that only needs to report "this file contains 50,000 samples of 3 threads" can read the header -and footer without touching the compressed sample data. This also simplifies -the format: the header's offset fields point directly to the tables rather -than to positions within a decompressed stream. +and footer without touching the compressed sample data. Zstd provides an excellent balance of compression ratio and speed. Profiling data compresses very well (often 5-10x) due to repetitive patterns: the same -small set of frame indices appears repeatedly, and delta-encoded timestamps -cluster around the sampling interval. Zstd's streaming API allows compression -without buffering the entire dataset. The writer feeds sample data through -the compressor incrementally, flushing compressed chunks to disk as they -become available. +small set of frame indices appears repeatedly, and timestamp residuals +cluster around zero. -Level 5 compression is used as a default. Lower levels (1-3) are faster but -compress less; higher levels (6+) compress more but slow down writing. Level -5 provides good compression with minimal impact on profiling overhead. +Level 3 compression is used as a default. It gives good compression without +slowing down each block flush. Compressing columns separately keeps blocks +independent, at the cost of restarting compression history at every block. ## Reading and Writing @@ -486,9 +563,9 @@ compress less; higher levels (6+) compress more but slow down writing. Level 3. For each sample: - Intern any new strings, assigning sequential indices - Intern any new frames, assigning sequential indices - - Encode the sample record and write to the buffer - - Flush the buffer through compression (if enabled) when full -4. Flush remaining buffered data and finalize compression + - Append the sample to the current block's columns + - Flush the block when it reaches its size or stream limit +4. Flush the last block 5. Write the string table (length-prefixed strings in index order) 6. Write the frame table (varint-encoded entries in index order) 7. Write measured profile statistics, when available @@ -497,7 +574,13 @@ compress less; higher levels (6+) compress more but slow down writing. Level The writer maintains two dictionaries: one mapping strings to indices, one mapping (filename_idx, funcname_idx, lineno) tuples to frame indices. These -enable O(1) lookup during interning. +enable O(1) lookup during interning. To find `REUSE` candidates, it also keeps +a small per-block cache of complete stacks, confirmed by a full comparison. + +When a block is flushed, the status and frame ID columns are transposed into +bit and byte planes, the `REPEAT` counts are inserted into the instruction +column, and each column is encoded as described in +[Compression](#compression). ### Reading @@ -510,13 +593,32 @@ enable O(1) lookup during interning. 6. Parse the string table, populating the array 7. Allocate the frame array 8. Parse the frame table, populating the array -9. If compressed, decompress the sample data region -10. Iterate through samples, resolving indices to strings/frames - (byte-swapping thread_id and interpreter_id if needed) +9. For each block, decode its columns into a reusable buffer and replay its + samples in order, resolving indices to strings/frames (byte-swapping the + stream table if needed) The reader builds lookup arrays rather than dictionaries since it only needs index-to-value mapping, not value-to-index. +Consecutive samples of one stream with the same stack and status are passed +to the collector in a single call, with up to 8,192 timestamps. + +The block header and columns are validated before any sample of the block is +replayed. Stack instructions are checked as they are replayed, so a block +with a bad instruction may already have passed earlier samples to the +collector before the error is raised. The reader rejects a block when: + +- a size, encoding, or stream or frame reference is out of range, or a + column has trailing data; +- the stream table lists a thread twice or has a stream with no samples; +- a stream's first instruction in the block is `REPEAT`, or a `REPEAT` + count is zero or runs past the block; +- a stack would be deeper than 256 frames, or `REUSE` refers to frame IDs + that have not been read yet; +- not every frame ID is used; +- status bitmap padding bits are not zero; +- it uses zstd in an uncompressed file. + ## Platform Considerations ### Byte Ordering and Cross-Platform Portability @@ -540,7 +642,7 @@ zeros instead of the actual value. The reader tracks whether byte-swapping is needed via a `needs_swap` flag set during header parsing. All fixed-width fields in the header, footer, and -sample data are conditionally byte-swapped using Python's internal byte-swap +stream tables are conditionally byte-swapped using Python's internal byte-swap functions (`_Py_bswap32`, `_Py_bswap64` from `pycore_bitutils.h`). Variable-length integers (varints) are byte-order independent since they @@ -568,10 +670,8 @@ On Windows, the implementation falls back to standard file I/O with full file buffering. Profiling data files are typically small enough (tens to hundreds of megabytes) that this is acceptable. -The writer uses a 512 KB buffer to batch small writes. Each sample record -is typically tens of bytes; writing these individually would incur excessive -syscall overhead. The buffer accumulates data until full, then flushes in -one write() call (or feeds through the compression stream). +The writer buffers samples in the current block and writes each completed +block with a single call, so individual samples never cause a write. ## Future Considerations diff --git a/Lib/profiling/sampling/binary_collector.py b/Lib/profiling/sampling/binary_collector.py index 3d6d988077cfa2a..21f58daf76f3dd4 100644 --- a/Lib/profiling/sampling/binary_collector.py +++ b/Lib/profiling/sampling/binary_collector.py @@ -146,7 +146,7 @@ def get_stats(self): """Get encoding statistics. Returns: - Dict with encoding statistics including repeat/full/suffix/pop-push + Dict with encoding statistics including repeat/full/suffix/reuse record counts, frames written/saved, and compression ratio. """ return self._writer.get_stats() diff --git a/Lib/profiling/sampling/sample.py b/Lib/profiling/sampling/sample.py index 1dff6529ade0408..e50af2528e93bf0 100644 --- a/Lib/profiling/sampling/sample.py +++ b/Lib/profiling/sampling/sample.py @@ -411,22 +411,22 @@ def _print_binary_stats(self, collector): repeat_samples = stats.get('repeat_samples', 0) full_records = stats.get('full_records', 0) suffix_records = stats.get('suffix_records', 0) - pop_push_records = stats.get('pop_push_records', 0) + reuse_records = stats.get('reuse_records', 0) total_records = stats.get('total_records', 0) if total_records > 0: repeat_pct = repeat_records / total_records * 100 full_pct = full_records / total_records * 100 suffix_pct = suffix_records / total_records * 100 - pop_push_pct = pop_push_records / total_records * 100 + reuse_pct = reuse_records / total_records * 100 else: - repeat_pct = full_pct = suffix_pct = pop_push_pct = 0 + repeat_pct = full_pct = suffix_pct = reuse_pct = 0 print(f" Records: {total_records:,}") print(f" RLE repeat: {repeat_records:,} ({ANSIColors.GREEN}{repeat_pct:.1f}%{ANSIColors.RESET}) [{repeat_samples:,} samples]") print(f" Full stack: {full_records:,} ({full_pct:.1f}%)") print(f" Suffix match: {suffix_records:,} ({suffix_pct:.1f}%)") - print(f" Pop-push: {pop_push_records:,} ({pop_push_pct:.1f}%)") + print(f" Stack reuse: {reuse_records:,} ({reuse_pct:.1f}%)") frames_written = stats.get('total_frames_written', 0) frames_saved = stats.get('frames_saved', 0) diff --git a/Lib/test/test_profiling/test_sampling_profiler/test_binary_format.py b/Lib/test/test_profiling/test_sampling_profiler/test_binary_format.py index 66a9c68a61c47dc..700d36012d6c318 100644 --- a/Lib/test/test_profiling/test_sampling_profiler/test_binary_format.py +++ b/Lib/test/test_profiling/test_sampling_profiler/test_binary_format.py @@ -10,7 +10,7 @@ from unittest import mock from collections import defaultdict -from test.support import captured_stderr +from test.support import captured_stderr, import_helper try: import _remote_debugging @@ -127,6 +127,28 @@ def export(self, filename): pass +def encode_varint(value): + """Encode an unsigned varint.""" + out = bytearray() + while value >= 0x80: + out.append((value & 0x7F) | 0x80) + value >>= 7 + out.append(value) + return bytes(out) + + +def decode_varint(data, offset): + """Decode an unsigned varint, returning (value, next offset).""" + value = shift = 0 + while True: + byte = data[offset] + offset += 1 + value |= (byte & 0x7F) << shift + shift += 7 + if byte < 0x80: + return value, offset + + def samples_to_by_thread(samples): """Convert input samples to by-thread format for comparison.""" by_thread = defaultdict(list) @@ -147,6 +169,20 @@ def samples_to_by_thread(samples): class BinaryFormatTestBase(unittest.TestCase): """Base class with common setup/teardown for binary format tests.""" + HDR_OFF_STR_TABLE = 40 + FILE_HEADER_PLACEHOLDER_SIZE = 64 + + # Sample block layout (see binary_io.h). + BLOCK_MAGIC = b"TBLK" + COLUMN_COUNT = 6 + COLUMN_STREAM_IDS = 1 + COLUMN_TIMESTAMP_RESIDUALS = 2 + COLUMN_STATUS_BITMAPS = 3 + COLUMN_STACK_INSTRUCTIONS = 4 + COLUMN_FRAME_IDS = 5 + COLUMN_ZSTD = 1 + COLUMN_CONSTANT = 2 + def setUp(self): self.temp_files = [] @@ -155,6 +191,13 @@ def tearDown(self): if os.path.exists(f): os.unlink(f) + def temp_filename(self): + """Return a new temporary file name that is removed on cleanup.""" + with tempfile.NamedTemporaryFile(suffix=".bin", delete=False) as f: + filename = f.name + self.temp_files.append(filename) + return filename + def create_binary_file(self, samples, interval=1000, compression="none", mode=None, capture_config=None): """Create a test binary file and track it for cleanup.""" @@ -179,6 +222,45 @@ def write_binary_file(self, samples, interval=1000, compression="none", collector.export(None) return filename, collector + def read_block_columns(self, filename): + """Return the header flags and (decoded size, encoding) per column. + + Each block is a dict with frame_width, common_status_flags, + status_bitmap_flags and columns. + """ + with open(filename, "rb") as raw: + data = raw.read() + sample_end = struct.unpack_from("=Q", data, self.HDR_OFF_STR_TABLE)[0] + offset = self.FILE_HEADER_PLACEHOLDER_SIZE + blocks = [] + while offset < sample_end: + magic_end = offset + len(self.BLOCK_MAGIC) + self.assertEqual(data[offset:magic_end], self.BLOCK_MAGIC) + _, offset = decode_varint(data, magic_end) # sample count + _, offset = decode_varint(data, offset) # base timestamp + block = {"frame_width": data[offset], + "common_status_flags": data[offset + 1], + "status_bitmap_flags": data[offset + 2]} + offset += 3 + columns = [] + stored_total = 0 + for _ in range(self.COLUMN_COUNT): + decoded_size, offset = decode_varint(data, offset) + encoding = data[offset] + offset += 1 + if encoding == self.COLUMN_ZSTD: + stored_size, offset = decode_varint(data, offset) + elif encoding == self.COLUMN_CONSTANT: + stored_size = 1 + else: + stored_size = decoded_size + columns.append((decoded_size, encoding)) + stored_total += stored_size + offset += stored_total + block["columns"] = columns + blocks.append(block) + return blocks + def roundtrip(self, samples, interval=1000, compression="none"): """Write samples to binary and read back.""" filename = self.create_binary_file(samples, interval, compression) @@ -897,6 +979,320 @@ def test_rejects_non_pathlike(self): with self.assertRaises(TypeError): _remote_debugging.BinaryReader(123) + def test_writer_rejects_invalid_compression(self): + """BinaryWriter rejects an unknown compression type.""" + with self.assertRaisesRegex(ValueError, "compression"): + _remote_debugging.BinaryWriter( + self.temp_filename(), 1000, 0, compression=2 + ) + + def test_writer_rejects_status_outside_byte_range(self): + """A status above 255 raises OverflowError and drops that sample.""" + filename = self.temp_filename() + writer = _remote_debugging.BinaryWriter(filename, 1000, 0) + valid = [make_interpreter(0, [make_thread(1, [], status=1)])] + invalid = [make_interpreter(0, [make_thread(1, [], status=256)])] + writer.write_sample(valid, 1000) + with self.assertRaisesRegex(OverflowError, "thread status"): + writer.write_sample(invalid, 2000) + self.assertTrue(writer.limit_reached) + writer.write_sample(valid, 3000) + writer.finalize() + self.assertEqual(writer.total_samples, 2) + + with BinaryReader(filename) as reader: + self.assertEqual(reader.replay_samples(RawCollector()), 2) + + def test_writer_reports_stack_reuse(self): + """A stack seen earlier in the block is stored as STACK_REUSE.""" + filename = self.temp_filename() + frames = [make_frame("shared.py", line, "shared") for line in range(3)] + sample = [make_interpreter(0, [ + make_thread(1, frames), + make_thread(2, frames), + ])] + writer = _remote_debugging.BinaryWriter(filename, 1000, 0) + writer.write_sample(sample, 1000) + stats = writer.get_stats() + self.assertEqual(stats["reuse_records"], 1) + self.assertEqual(stats["total_records"], 2) + self.assertEqual(stats["total_samples"], 2) + writer.close() + + def test_writer_reuses_colliding_stacks(self): + """Stacks sharing a cache bucket are both stored as STACK_REUSE.""" + buckets = {} + for frame_id in range(2, 4099): + bucket = hash(struct.pack("=3I", 0, 1, frame_id)) % 4096 + if bucket in buckets: + first_id = buckets[bucket] + second_id = frame_id + break + buckets[bucket] = frame_id + + frames = [make_frame("a.py", line, "f") + for line in range(second_id + 1)] + first = [frames[0], frames[1], frames[first_id]] + second = [frames[0], frames[1], frames[second_id]] + + filename = self.temp_filename() + writer = _remote_debugging.BinaryWriter(filename, 1000, 0) + timestamp = 1000 + # Assign frame IDs without exceeding MAX_STACK_DEPTH. + for start in range(0, len(frames), 256): + stack = frames[start:start + 256] + writer.write_sample([make_interpreter(0, [make_thread(2, stack)])], + timestamp) + timestamp += 1000 + for stack in (first, second, first, second): + writer.write_sample([make_interpreter(0, [make_thread(1, stack)])], + timestamp) + timestamp += 1000 + self.assertEqual(writer.get_stats()["reuse_records"], 2) + writer.close() + + def test_writer_buffered_bytes_match_flushed_bytes(self): + """bytes_written before finalize() matches the flushed size.""" + filename = self.temp_filename() + frames = [make_frame("a.py", line, "f") for line in range(300)] + writer = _remote_debugging.BinaryWriter(filename, 1000, 0) + for i in range(100): + sample = [make_interpreter(0, [ + make_thread(1, frames[i:i + 5], status=i % 3), + make_thread(2, frames[:3]), + ])] + writer.write_sample(sample, 1000 * (i + 1)) + buffered = writer.get_stats()["bytes_written"] + writer.finalize() + + [block] = self.read_block_columns(filename) + self.assertEqual(buffered, + sum(size for size, _ in block["columns"])) + + def _write_block_spanning_file(self): + # A block accepts at most 4,096 streams. + frame = make_frame("many_threads.py", 1, "work") + samples = [ + [make_interpreter(0, [make_thread(i + 1, [frame])])] + for i in range(4097) + ] + return samples, self.create_binary_file(samples) + + def test_samples_span_multiple_blocks(self): + """Samples filling more than one block roundtrip.""" + samples, filename = self._write_block_spanning_file() + self.assertGreaterEqual(len(self.read_block_columns(filename)), 2) + + collector = RawCollector() + with BinaryReader(filename) as reader: + self.assertEqual(reader.replay_samples(collector), len(samples)) + self.assert_samples_equal(samples, collector) + + def test_block_size_limit_starts_new_block(self): + """A thread whose samples exceed 1 MiB is split across blocks.""" + # Each sample writes a new 256-frame stack, about 1 KiB while + # buffered, so 1,100 samples need more than one block. + frames = [make_frame("deep.py", line, "f") for line in range(1356)] + samples = [ + [make_interpreter(0, [make_thread(1, frames[i:i + 256])])] + for i in range(1100) + ] + filename = self.create_binary_file(samples) + blocks = self.read_block_columns(filename) + self.assertGreaterEqual(len(blocks), 2) + for block in blocks: + self.assertLessEqual(sum(size for size, _ in block["columns"]), + 1024 * 1024) + with BinaryReader(filename) as reader: + self.assertEqual(reader.replay_samples(RawCollector()), + len(samples)) + + def test_writer_uses_constant_columns_and_narrow_frame_ids(self): + """Repeated bytes are stored once and frame IDs use the least width.""" + frame = make_frame("steady.py", 1, "steady") + sample = [make_interpreter(0, [make_thread(1, [frame])])] + filename = self.temp_filename() + writer = _remote_debugging.BinaryWriter(filename, 1000, 0) + for i in range(100): + writer.write_sample(sample, 1000 * (i + 1)) + writer.finalize() + # One stream and on-time samples: every stream ID and residual is 0 + [block] = self.read_block_columns(filename) + self.assertEqual(block["frame_width"], 1) + self.assertEqual(block["columns"][self.COLUMN_STREAM_IDS], + (100, self.COLUMN_CONSTANT)) + self.assertEqual(block["columns"][self.COLUMN_TIMESTAMP_RESIDUALS], + (100, self.COLUMN_CONSTANT)) + + # 300 distinct frames need two bytes per frame ID + frames = [make_frame("wide.py", line, "f") for line in range(300)] + samples = [ + [make_interpreter(0, [make_thread(1, frames[:150])])], + [make_interpreter(0, [make_thread(1, frames[150:])])], + ] + filename = self.create_binary_file(samples) + [block] = self.read_block_columns(filename) + self.assertEqual(block["frame_width"], 2) + self.assertEqual(block["columns"][self.COLUMN_FRAME_IDS][0], 300 * 2) + + def test_writer_stores_common_status_bits_once(self): + """Status bits set in every sample go in the header, not a bitmap.""" + frame = make_frame("status.py", 1, "f") + filename = self.temp_filename() + writer = _remote_debugging.BinaryWriter(filename, 1000, 0) + for i in range(20): + # HAS_GIL in every sample; ON_CPU in every other one + status = THREAD_STATUS_HAS_GIL | (THREAD_STATUS_ON_CPU * (i % 2)) + sample = [make_interpreter(0, [make_thread(1, [frame], status)])] + writer.write_sample(sample, 1000 * (i + 1)) + writer.finalize() + [block] = self.read_block_columns(filename) + self.assertEqual(block["common_status_flags"], THREAD_STATUS_HAS_GIL) + self.assertEqual(block["status_bitmap_flags"], THREAD_STATUS_ON_CPU) + # One bitmap of ceil(20 / 8) bytes, for ON_CPU only + self.assertEqual(block["columns"][self.COLUMN_STATUS_BITMAPS][0], 3) + + @unittest.skipUnless(ZSTD_AVAILABLE, "zstd compression not available") + def test_writer_compresses_repetitive_columns(self): + """Compressed files store repetitive columns with zstd.""" + root = [make_frame("root.py", line, "root") for line in range(20)] + leaves = [make_frame("leaf.py", 1, "even"), + make_frame("leaf.py", 2, "odd")] + samples = [ + [make_interpreter(0, [make_thread(1, [leaves[i % 2]] + root)])] + for i in range(2000) + ] + filename = self.create_binary_file(samples, compression="zstd") + [block] = self.read_block_columns(filename) + columns = block["columns"] + self.assertEqual(columns[self.COLUMN_STACK_INSTRUCTIONS][1], + self.COLUMN_ZSTD) + self.assertEqual(columns[self.COLUMN_FRAME_IDS][1], self.COLUMN_ZSTD) + + def test_replay_byte_swapped_file(self): + """A file written with the other byte order replays the same.""" + samples = [[make_interpreter(0, [ + make_thread(1, [make_frame("a.py", 1, "f")]), + make_thread(2, [make_frame("b.py", 2, "g")]), + ])] for _ in range(3)] + filename = self.temp_filename() + writer = _remote_debugging.BinaryWriter(filename, 1000, 0) + for i, sample in enumerate(samples): + writer.write_sample(sample, 1000 * (i + 1)) + writer.finalize() + + data = bytearray(pathlib.Path(filename).read_bytes()) + # Header fields: magic, version, start time, interval, sample + # count, thread count, table offsets, compression and config + fields = [(0, 4), (4, 4), (12, 8), (20, 8), (28, 8), (36, 4), + (40, 8), (48, 8), (56, 4), (60, 4)] + # Footer string and frame counts + fields += [(len(data) - 32, 4), (len(data) - 28, 4)] + # The block's stream table entries follow its header + offset = self.FILE_HEADER_PLACEHOLDER_SIZE + len(self.BLOCK_MAGIC) + _, offset = decode_varint(data, offset) # sample count + _, offset = decode_varint(data, offset) # base timestamp + offset += 3 + for _ in range(self.COLUMN_COUNT): + _, offset = decode_varint(data, offset) + offset += 1 # raw encoding, no stored size + for entry in range(2): + fields += [(offset + entry * 12, 8), (offset + entry * 12 + 8, 4)] + for start, size in fields: + data[start:start + size] = data[start:start + size][::-1] + pathlib.Path(filename).write_bytes(data) + + collector = RawCollector() + with BinaryReader(filename) as reader: + self.assertEqual(reader.replay_samples(collector), 6) + self.assert_samples_equal(samples, collector) + + def test_progress_updates_across_blocks(self): + """Progress is reported at fixed intervals across block boundaries.""" + samples, filename = self._write_block_spanning_file() + calls = [] + with BinaryReader(filename) as reader: + reader.replay_samples( + RawCollector(), + lambda current, total: calls.append((current, total))) + total = len(samples) + self.assertEqual( + calls, + [(0, total), (1000, total), (2000, total), (3000, total), + (4000, total), (total, total)], + ) + + def test_reader_replays_twice(self): + """Replaying twice gives the same samples and statistics.""" + class RetainingCollector(RawCollector): + def __init__(self): + super().__init__() + self.frame_lists = [] + + def collect(self, stack_frames, timestamps_us): + self.frame_lists.extend( + thread.frame_info + for interp in stack_frames + for thread in interp.threads + ) + super().collect(stack_frames, timestamps_us) + + samples = [ + [make_interpreter(0, [ + make_thread(1, [make_frame("a.py", i % 3, "f")]), + make_thread(2, [make_frame("b.py", 1, "g")]), + ])] + for i in range(10) + ] + filename = self.create_binary_file(samples) + with BinaryReader(filename) as reader: + first = RetainingCollector() + second = RetainingCollector() + self.assertEqual(reader.replay_samples(first), 20) + self.assertEqual(reader.replay_samples(second), 20) + self.assertEqual(reader.get_stats()["total_samples"], 20) + self.assertEqual(first.by_thread, second.by_thread) + for first_frames, second_frames in zip( + first.frame_lists, second.frame_lists, strict=True + ): + self.assertIsNot(first_frames, second_frames) + self.assertEqual(len(first_frames), len(second_frames)) + for first_frame, second_frame in zip( + first_frames, second_frames, strict=True + ): + self.assertIs(first_frame, second_frame) + + def test_writer_repeat_run_spans_interleaved_streams(self): + """A thread's repeat run continues across other threads' samples.""" + filename = self.temp_filename() + idle = [make_frame("idle.py", 1, "wait")] + busy = [[make_frame("busy.py", 1, "even")], + [make_frame("busy.py", 2, "odd")]] + writer = _remote_debugging.BinaryWriter(filename, 1000, 0) + for i in range(10): + sample = [make_interpreter(0, [ + make_thread(1, idle), + make_thread(2, busy[i % 2]), + ])] + writer.write_sample(sample, 1000 * (i + 1)) + stats = writer.get_stats() + self.assertEqual(stats["repeat_records"], 1) + self.assertEqual(stats["repeat_samples"], 9) + writer.finalize() + + collector = RawCollector() + with BinaryReader(filename) as reader: + self.assertEqual(reader.replay_samples(collector), 20) + self.assertEqual(reader.get_stats()["repeat_samples"], 9) + idle_frames = [frame_to_dict(frame) for frame in idle] + self.assertEqual( + [sample["frames"] for sample in collector.by_thread[(0, 1)]], + [idle_frames] * 10) + self.assertEqual( + [sample["frames"][0]["funcname"] + for sample in collector.by_thread[(0, 2)]], + ["even", "odd"] * 5) + def test_invalid_path_error_preserves_pathlib(self): """Missing path: OSError carries the original path object, not a string.""" missing = pathlib.Path("/i/do/not/exist") @@ -907,13 +1303,9 @@ def test_invalid_path_error_preserves_pathlib(self): def test_writer_handles_empty_stack_first_sample(self): """BinaryWriter.write_sample tolerates an empty stack on a fresh thread. - Regression test for the C-level RLE bug in process_thread_sample: a - freshly-created ThreadEntry has prev_stack_depth == 0, so an empty - curr_stack compares as STACK_REPEAT against the zero-initialized - previous stack. Before the fix, this fell through the - `&& !is_new_thread` guard into write_sample_with_encoding, which had - no handler for STACK_REPEAT and raised - RuntimeError("Invalid stack encoding type"). Goes through + Regression test for gh-149342: an empty first stack equals the empty + stack a new thread starts from, but it must still be written as a + stack (STACK_REPLACE_SUFFIX 0 0) rather than a repeat. Goes through BinaryWriter.write_sample directly so the test cannot be masked by any Python-level filtering. """ @@ -931,8 +1323,7 @@ def test_writer_handles_empty_stack_first_sample(self): # scenario that exposes the bug. writer.write_sample(empty_sample, 1000) writer.write_sample(empty_sample, 2000) - # Mix in a real sample to exercise the transition out of the - # empty-stack RLE buffer. + # Mix in a real sample after the repeated empty stack. real_sample = [ make_interpreter(0, [make_thread(1, [make_frame("a.py", 1, "f")])]) ] @@ -942,8 +1333,8 @@ def test_writer_handles_empty_stack_first_sample(self): reader_collector = RawCollector() with BinaryReader(filename) as reader: count = reader.replay_samples(reader_collector) - # Empty-stack samples are recorded as STACK_REPEAT records with - # depth-0 stacks; the file must replay all three samples. + # The empty stack is written once and then repeated; the file must + # replay all three samples. self.assertEqual(count, 3) def test_writer_handles_mixed_empty_and_real_first_sample(self): @@ -977,11 +1368,9 @@ def test_writer_handles_mixed_empty_and_real_first_sample(self): def test_writer_total_samples_after_finalize_matches_reader(self): """BinaryWriter.total_samples after finalize() matches the reader's count.""" - # Five IDENTICAL samples force every sample beyond the first into the - # per-thread RLE buffer. Regression for the cached_total_samples - # ordering bug: capturing the cache BEFORE binary_writer_finalize() - # missed the buffered samples that flush_pending_rle() counts. Keep - # the samples identical to preserve coverage. See gh-149342. + # Five IDENTICAL samples stay in the open block, as one repeat, until + # finalize() flushes it; total_samples must still count all of them. + # Keep the samples identical to preserve coverage. See gh-149342. samples = [ [make_interpreter(0, [make_thread(1, [make_frame("a.py", 1, "f")])])] ] * 5 @@ -992,8 +1381,8 @@ def test_writer_total_samples_after_finalize_matches_reader(self): self.assertEqual(writer_collector.total_samples, len(samples)) self.assertEqual(writer_collector.total_samples, replayed) - def test_rle_buffer_flushes_before_finalize(self): - """RLE buffer flushes before the writer is destroyed during finalize().""" + def test_open_block_flushes_on_finalize(self): + """finalize() writes the open block before the writer is destroyed.""" with tempfile.NamedTemporaryFile(suffix=".bin", delete=False) as f: filename = f.name self.temp_files.append(filename) @@ -1194,11 +1583,10 @@ def test_finalize_failure_breaks_writer(self): class TestBinaryFormatValidation(BinaryFormatTestBase): """Tests for malformed binary files.""" + HDR_OFF_VERSION = 4 HDR_OFF_SAMPLES = 28 HDR_OFF_THREADS = 36 - HDR_OFF_STR_TABLE = 40 HDR_OFF_FRAME_TABLE = 48 - FILE_HEADER_PLACEHOLDER_SIZE = 64 FILE_FOOTER_SIZE = 32 FTR_OFF_STRINGS = 0 FTR_OFF_FRAMES = 4 @@ -1261,7 +1649,8 @@ def test_replay_rejects_trailing_partial_sample_header(self): with BinaryReader(filename) as reader: with self.assertRaises(ValueError) as cm: reader.replay_samples(RawCollector()) - self.assertEqual(str(cm.exception), "Truncated sample data: 1 trailing bytes") + self.assertEqual(str(cm.exception), + "Invalid sample block: truncated header") # Minimum on-disk size of one table entry (see binary_io.h). MIN_STRING_ENTRY_SIZE = 1 @@ -1345,6 +1734,21 @@ def test_open_accepts_frame_count_at_capacity_boundary(self): f"possible {max_frames}", ) + def test_open_rejects_older_format_version(self): + """Open rejects version 1 files and reports version 2.""" + filename = self.create_binary_file([], compression="none") + with BinaryReader(filename) as reader: + self.assertEqual(reader.get_info()["version"], 2) + with open(filename, "r+b") as raw: + raw.seek(self.HDR_OFF_VERSION) + raw.write(struct.pack("=I", 1)) + with self.assertRaisesRegex( + ValueError, + r"Unsupported format version 1 \(this reader supports " + r"version 2\)"): + with BinaryReader(filename): + pass + def test_sample_count_reads_full_64_bits(self): """sample_count values requiring the upper 32 bits decode correctly.""" filename = self.create_binary_file([], compression="none") @@ -1370,12 +1774,255 @@ def test_sample_count_boundary_values(self): with BinaryReader(filename) as reader: self.assertEqual(reader.get_info()["sample_count"], value) + # Block column indices and encodings (see binary_io.h). + MAX_BLOCK_SIZE = 1024 * 1024 + + def _write_single_block(self, num_samples=1, instructions=b"\x01\x00\x00", + frame_ids=b"", *, header_samples=None, + file_samples=None, stream_count=1, + stream_table=None, stream_ids=None, + residuals=None, statuses=b"", status_flags=(0, 0), + frame_width=1, descriptors=None, magic=b"TBLK", + truncate=None, frame_count=1): + """Write a file whose only block is built from the given parts. + + By default every sample belongs to stream 0, has status 0 and is + exactly one sample interval after the previous one, and the only + stack is empty (STACK_REPLACE_SUFFIX 0 0). The frame table holds + *frame_count* frames: frame 0 is in "a.py" and frame N in "N.py". + *descriptors* maps a column index to (encoding, decoded size) to + replace its raw descriptor. + """ + frames = [make_frame("a.py", 1, "f")] + frames += [make_frame(f"{index}.py", 1, "f") + for index in range(1, frame_count)] + samples = [[make_interpreter(0, [make_thread(1, frames[i:i + 256])])] + for i in range(0, frame_count, 256)] + filename = self.create_binary_file(samples, compression="none") + data = bytearray(pathlib.Path(filename).read_bytes()) + sample_end = struct.unpack_from("=Q", data, self.HDR_OFF_STR_TABLE)[0] + if stream_table is None: + stream_table = b"".join(struct.pack("=QI", thread, 0) + for thread in range(1, stream_count + 1)) + columns = ( + stream_table, + b"\x00" * num_samples if stream_ids is None else stream_ids, + b"\x00" * num_samples if residuals is None else residuals, + statuses, + instructions, + frame_ids, + ) + if header_samples is None: + header_samples = num_samples + block = (magic + encode_varint(header_samples) + + b"\x01" # base timestamp + + bytes([frame_width, *status_flags])) + payload = b"" + for index, column in enumerate(columns): + encoding, decoded_size = (descriptors or {}).get( + index, (0, len(column))) + block += encode_varint(decoded_size) + bytes([encoding]) + if encoding == self.COLUMN_ZSTD: + block += encode_varint(len(column)) + if encoding == self.COLUMN_CONSTANT: + column = column[:1] + payload += column + block = (block + payload)[:truncate] + data[self.FILE_HEADER_PLACEHOLDER_SIZE:sample_end] = block + shift = len(block) - (sample_end - self.FILE_HEADER_PLACEHOLDER_SIZE) + if file_samples is None: + file_samples = num_samples + struct.pack_into("=Q", data, self.HDR_OFF_SAMPLES, file_samples) + struct.pack_into("=I", data, self.HDR_OFF_THREADS, + max(len(stream_table) // 12, 1)) + for offset in (self.HDR_OFF_STR_TABLE, self.HDR_OFF_FRAME_TABLE): + old_offset = struct.unpack_from("=Q", data, offset)[0] + struct.pack_into("=Q", data, offset, old_offset + shift) + struct.pack_into("=Q", data, len(data) - 24, len(data)) + pathlib.Path(filename).write_bytes(data) + return filename + + def assert_block_rejected(self, filename, message): + with BinaryReader(filename) as reader: + with self.assertRaises(ValueError) as cm: + reader.replay_samples(RawCollector()) + self.assertEqual(str(cm.exception), f"Invalid sample block: {message}") + + def test_replay_accepts_hand_built_block(self): + """Replay accepts a block built by hand from its columns.""" + # STACK_REPLACE_SUFFIX 0 0, then STACK_REPEAT 2. + filename = self._write_single_block(3, b"\x01\x00\x00\x00\x02") + collector = RawCollector() + with BinaryReader(filename) as reader: + self.assertEqual(reader.replay_samples(collector), 3) + self.assertEqual(len(collector.by_thread[(0, 1)]), 3) + + def test_replay_decodes_every_frame_id_width(self): + """Frame IDs decode correctly at every byte width.""" + # STACK_REPLACE_SUFFIX 0 1 reads one frame ID, stored in one byte + # plane per byte of the width, low byte first. + for width, frame_id in ((1, 43), (2, 299), (3, 299), (4, 299)): + with self.subTest(width=width): + filename = self._write_single_block( + 1, b"\x01\x00\x01", frame_id.to_bytes(width, "little"), + frame_width=width, frame_count=300) + collector = RawCollector() + with BinaryReader(filename) as reader: + self.assertEqual(reader.replay_samples(collector), 1) + frames = collector.by_thread[(0, 1)][0]["frames"] + self.assertEqual([frame["filename"] for frame in frames], + [f"{frame_id}.py"]) + + def test_replay_rejects_reentrant_replay(self): + """Replaying from inside the same reader's collector raises.""" + filename = self._write_single_block( + 2, b"\x01\x00\x00\x00\x01") # REPEAT 1 + + with BinaryReader(filename) as reader: + class ReentrantCollector: + def collect(self, stack_frames, timestamps_us): + reader.replay_samples(RawCollector()) + + with self.assertRaisesRegex(RuntimeError, "already replaying"): + reader.replay_samples(ReentrantCollector()) + self.assertEqual(reader.replay_samples(RawCollector()), 2) + + def test_replay_rejects_invalid_block(self): + """Replay rejects every kind of malformed block.""" + frames = self.COLUMN_FRAME_IDS + cases = [ + ("wrong magic", dict(magic=b"XBLK")), + # Magic, sample count and base timestamp, then 1 of 3 flag bytes + ("truncated header", dict(truncate=7)), + ("no samples", dict(header_samples=0)), + ("frame ID width out of range", dict(frame_width=0)), + ("frame ID width out of range", dict(frame_width=5)), + ("common and varying status flags overlap", + dict(status_flags=(1, 1))), + # Magic, sample count, base timestamp, 3 flag bytes and the + # first decoded size + ("truncated column descriptor", dict(truncate=10)), + ("unknown column encoding", dict(descriptors={frames: (3, 0)})), + ("zstd column in an uncompressed file", + dict(descriptors={self.COLUMN_STACK_INSTRUCTIONS: + (self.COLUMN_ZSTD, 3)})), + ("encoded column is empty", + dict(descriptors={frames: (self.COLUMN_CONSTANT, 0)})), + ("decoded columns exceed the block limit", + dict(frame_ids=b"\x00", + descriptors={frames: (self.COLUMN_CONSTANT, + self.MAX_BLOCK_SIZE)})), + # Rejected before anything is allocated for the samples + ("more samples than stored stream indices", + dict(header_samples=0xFFFFFFFF)), + ("column extends past the sample section", dict(truncate=-1)), + ("stream table has a partial entry", dict(stream_table=b"0" * 11)), + ("stream count out of range", dict(stream_table=b"")), + ("frame ID column has a partial value", + dict(frame_width=2, frame_ids=b"\x00")), + ("status bitmaps have the wrong size", dict(status_flags=(0, 1))), + ("status bitmap padding is not zero", + dict(status_flags=(0, 1), statuses=b"\x02")), + ("stream table repeats a thread", + dict(stream_table=struct.pack("=QI", 1, 0) * 2)), + ("sample refers to an unknown stream", dict(stream_ids=b"\x01")), + ("stream index column has trailing data", + dict(num_samples=2, header_samples=1)), + ("timestamp column has trailing data", + dict(residuals=b"\x00\x00")), + ("frame index out of range", + dict(instructions=b"\x01\x00\x01", frame_ids=b"\x05")), + # Frame IDs 65536 and 2**24 only have a byte in the third or + # fourth plane + ("frame index out of range", + dict(instructions=b"\x01\x00\x01", frame_width=3, + frame_ids=(1 << 16).to_bytes(3, "little"), + frame_count=300)), + ("frame index out of range", + dict(instructions=b"\x01\x00\x01", frame_width=4, + frame_ids=(1 << 24).to_bytes(4, "little"), + frame_count=300)), + ("unknown stack instruction", dict(instructions=b"\x03")), + ("first stack of a stream cannot repeat", + dict(instructions=b"\x00\x01")), + ("repeat count out of range", + dict(num_samples=2, instructions=b"\x01\x00\x00\x00\x00")), + ("stack prefix out of range", dict(instructions=b"\x01\x01\x00")), + ("stack uses more frame IDs than stored", + dict(instructions=b"\x01\x00\x02", frame_ids=b"\x00")), + # STACK_REUSE 0 257 + ("stack is too deep", dict(instructions=b"\x02\x00\x81\x02")), + ("reused stack is not stored earlier", + dict(instructions=b"\x02\x00\x01", frame_ids=b"\x00")), + ("stack instruction column has trailing data", + dict(instructions=b"\x01\x00\x00\x01")), + ("frame ID column has unused values", dict(frame_ids=b"\x00")), + ("stream table has an unused entry", dict(stream_count=2)), + ("repeat count extends past the block", + dict(num_samples=2, instructions=b"\x01\x00\x00\x00\x02")), + ("more samples than the file header declares", + dict(file_samples=0)), + ] + for message, kwargs in cases: + with self.subTest(message, **kwargs): + filename = self._write_single_block(**kwargs) + self.assert_block_rejected(filename, message) + + @unittest.skipUnless(ZSTD_AVAILABLE, "zstd compression not available") + def test_replay_rejects_invalid_zstd_column(self): + """Replay rejects corrupt and mismatched zstd columns.""" + zstd = import_helper.import_module("compression.zstd") + instructions = b"\x01\x00\x00" + frame = zstd.compress( + instructions, + options={zstd.CompressionParameter.checksum_flag: 1}) + bad_checksum = frame[:-1] + bytes([frame[-1] ^ 1]) + cases = [ + ("Corrupted zstd frame header", instructions, len(instructions)), + ("zstd frame size does not match the decoded size", + frame, len(instructions) + 1), + ("zstd decompression error: ", bad_checksum, len(instructions)), + ] + for message, stored, decoded_size in cases: + with self.subTest(message): + filename = self._write_single_block( + instructions=stored, + descriptors={self.COLUMN_STACK_INSTRUCTIONS: + (self.COLUMN_ZSTD, decoded_size)}) + with open(filename, "r+b") as raw: + raw.seek(56) # compression type + raw.write(struct.pack("=I", 1)) + with BinaryReader(filename) as reader: + with self.assertRaises(ValueError) as cm: + reader.replay_samples(RawCollector()) + self.assertStartsWith(str(cm.exception), message) + + def test_replay_propagates_callback_errors(self): + """Errors raised by the collector or progress callback propagate.""" + filename = self._write_single_block( + 2000, b"\x01\x00\x00\x00\xcf\x0f") # REPEAT 1999 + + class FailingCollector: + def collect(self, stack_frames, timestamps_us): + raise ZeroDivisionError + + def failing_progress(current, total): + if current: + raise ZeroDivisionError + + with BinaryReader(filename) as reader: + with self.assertRaises(ZeroDivisionError): + reader.replay_samples(FailingCollector()) + with self.assertRaises(ZeroDivisionError): + reader.replay_samples(RawCollector(), failing_progress) + self.assertEqual(reader.replay_samples(RawCollector()), 2000) + class TestBinaryEncodings(BinaryFormatTestBase): """Tests specifically targeting different stack encodings.""" - def test_stack_full_encoding(self): - """First sample uses STACK_FULL encoding and roundtrips.""" + def test_stack_first_sample_pattern(self): + """First sample stores its complete stack and roundtrips.""" frames = [make_frame(f"f{i}.py", i + 1, f"func{i}") for i in range(5)] samples = [[make_interpreter(0, [make_thread(1, frames)])]] collector, count = self.roundtrip(samples) @@ -1393,8 +2040,8 @@ def test_stack_repeat_encoding(self): self.assertEqual(count, 1000) self.assert_samples_equal(samples, collector) - def test_stack_suffix_encoding(self): - """Samples sharing suffix use STACK_SUFFIX and roundtrip.""" + def test_stack_shared_outer_frames_pattern(self): + """Samples keeping outer frames roundtrip (STACK_REPLACE_SUFFIX).""" samples = [] for i in range(10): frames = [make_frame(f"new{i}.py", i + 1, f"new{i}")] @@ -1409,8 +2056,8 @@ def test_stack_suffix_encoding(self): self.assertEqual(count, 10) self.assert_samples_equal(samples, collector) - def test_stack_pop_push_encoding(self): - """Samples with pop+push pattern roundtrip.""" + def test_stack_pop_push_pattern(self): + """Samples that pop frames and push new ones roundtrip.""" samples = [] base_frames = [make_frame("base.py", 10, "base")] @@ -1549,15 +2196,29 @@ def export(self, filename): def test_rle_long_run_splits_batches(self): - # Construct a single repeat record larger than the writer's buffer. + # Construct one block whose repeat run is longer than a replay batch. num_samples = 8193 filename = self.create_binary_file([], compression="none") data = bytearray(pathlib.Path(filename).read_bytes()) - record = (struct.pack("=QIB", 1, 0, 0) # STACK_REPEAT - + b"\x81\x40" # 8193 as a varint - + b"\x01\x00" * num_samples) # delta=1, status=0 + instructions = (b"\x01\x00\x00" # STACK_REPLACE_SUFFIX 0 0 + + b"\x00\x80\x40") # STACK_REPEAT 8192 + record = (b"TBLK" + + b"\x81\x40" # 8193 samples as a varint + + b"\x01" # base timestamp + + b"\x01\x00\x00" # frame width, status flags + + b"\x0c\x00" # stream table: 12 raw bytes + + b"\x81\x40\x02" # stream indices: constant + + b"\x81\x40\x02" # timestamp residuals: constant + + b"\x00\x00" # status bitmaps: empty + + bytes([len(instructions), 0]) # raw instructions + + b"\x00\x00" # frame IDs: empty + + struct.pack("=QI", 1, 0) + + b"\x00" # every sample is in stream 0 + + b"\x00" # every residual is zero + + instructions) data[64:64] = record struct.pack_into("=Q", data, 12, 0) # start timestamp + struct.pack_into("=Q", data, 20, 1) # sample interval struct.pack_into("=Q", data, 28, num_samples) struct.pack_into("=I", data, 36, 1) # thread count for offset in (40, 48): # string and frame table offsets @@ -1701,7 +2362,7 @@ def test_encoding_transitions_stress(self): # Repeat previous (will use RLE if identical) frames = base_frames[: random.randint(1, 5)] elif choice == 2: - # Add frames on top (suffix encoding) + # Add frames on top (keeps the outer frames) extra = random.randint(1, 3) frames = [ make_frame(f"top{i}_{j}.py", j, f"top{j}") @@ -1709,7 +2370,7 @@ def test_encoding_transitions_stress(self): ] frames.extend(base_frames[: random.randint(2, 4)]) else: - # Pop and push (pop-push encoding) + # Pop some frames and push new ones keep = random.randint(1, 3) push = random.randint(0, 2) frames = [ @@ -1893,6 +2554,25 @@ def test_replay_includes_persisted_stats(self): self.assertIn('"error_rate": 2.5', content) self.assertIn('"missed_samples": 1.5', content) + @unittest.skipUnless(ZSTD_AVAILABLE, "zstd compression not available") + def test_measured_stats_with_zstd(self): + """Measured statistics are read back when zstd is enabled.""" + frames = [make_frame(f"pkg/module_{i:03}.py", i + 1, f"func_{i:03}") + for i in range(200)] + sample = [make_interpreter(0, [make_thread(1, frames)])] + filename = self.temp_filename() + collector = BinaryCollector(filename, 2000, compression="zstd") + collector.collect(sample, timestamp_us=1000) + collector.set_stats(2000, 1.25, 4.0, error_rate=2.5, + missed_samples=1.5) + collector.export(None) + + with BinaryReader(filename) as reader: + info = reader.get_info() + self.assertEqual(info["duration_sec"], 1.25) + self.assertEqual(info["missed_samples"], 1.5) + self.assertEqual(reader.replay_samples(RawCollector()), 1) + def test_legacy_binary_has_no_measured_stats(self): frame = make_frame("hot.py", 99, "hot_func") bin_path = self.create_binary_file([ diff --git a/Lib/test/test_profiling/test_sampling_profiler/test_cli.py b/Lib/test/test_profiling/test_sampling_profiler/test_cli.py index e5dcd72c27ef5c2..789cc3408d5a856 100644 --- a/Lib/test/test_profiling/test_sampling_profiler/test_cli.py +++ b/Lib/test/test_profiling/test_sampling_profiler/test_cli.py @@ -954,7 +954,7 @@ def test_cli_replay_reader_errors_exit_cleanly(self): mock.patch("sys.argv", ["profiling.sampling.cli", "replay", profile]), mock.patch( "profiling.sampling.cli.BinaryReader", - side_effect=ValueError("Unsupported format version 2"), + side_effect=ValueError("Unsupported format version 3"), ), ): with self.assertRaises(SystemExit) as cm: @@ -962,7 +962,7 @@ def test_cli_replay_reader_errors_exit_cleanly(self): self.assertEqual( str(cm.exception), - "Error: Unsupported format version 2", + "Error: Unsupported format version 3", ) def test_cli_replay_propagates_recorded_mode(self): diff --git a/Makefile.pre.in b/Makefile.pre.in index cae9cae9d779bea..549c940664a9955 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -3452,7 +3452,7 @@ MODULE__CTYPES_MALLOC_CLOSURE=@MODULE__CTYPES_MALLOC_CLOSURE@ MODULE__ELEMENTTREE_DEPS=$(srcdir)/Modules/pyexpat.c @LIBEXPAT_INTERNAL@ MODULE__HASHLIB_DEPS=$(srcdir)/Modules/hashlib.h $(srcdir)/Modules/_openssl_mem.h MODULE__IO_DEPS=$(srcdir)/Modules/_io/_iomodule.h -MODULE__REMOTE_DEBUGGING_DEPS=$(srcdir)/Modules/_remote_debugging/_remote_debugging.h $(srcdir)/Modules/_remote_debugging/gc_stats.h +MODULE__REMOTE_DEBUGGING_DEPS=$(srcdir)/Modules/_remote_debugging/_remote_debugging.h $(srcdir)/Modules/_remote_debugging/binary_io.h $(srcdir)/Modules/_remote_debugging/gc_stats.h # HACL*-based cryptographic primitives MODULE__MD5_DEPS=$(srcdir)/Modules/hashlib.h $(LIBHACL_MD5_HEADERS) $(LIBHACL_MD5_LIB_@LIBHACL_LDEPS_LIBTYPE@) diff --git a/Misc/NEWS.d/next/Library/2026-10-20-12-49-32.gh-issue-159207.Lccezg.rst b/Misc/NEWS.d/next/Library/2026-10-20-12-49-32.gh-issue-159207.Lccezg.rst new file mode 100644 index 000000000000000..a96098fd211abc6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2026-10-20-12-49-32.gh-issue-159207.Lccezg.rst @@ -0,0 +1,3 @@ +The :mod:`profiling.sampling` binary output format now stores samples in +independent blocks. This is a breaking format change: Python 3.16 does not +support reading profiles written in the previous format. diff --git a/Modules/_remote_debugging/binary_io.h b/Modules/_remote_debugging/binary_io.h index 6a2c5b795823e15..bcc1ba65b4e5c34 100644 --- a/Modules/_remote_debugging/binary_io.h +++ b/Modules/_remote_debugging/binary_io.h @@ -2,7 +2,7 @@ * Python Remote Debugging Module - Binary I/O Header * * This header provides declarations for high-performance binary file I/O - * for profiling data with optional zstd streaming compression. + * for profiling data with optional zstd compression. ******************************************************************************/ #ifndef Py_BINARY_IO_H @@ -23,7 +23,7 @@ extern "C" { #define BINARY_FORMAT_MAGIC 0x54414348 /* "TACH" (Tachyon) in native byte order */ #define BINARY_FORMAT_MAGIC_SWAPPED 0x48434154 /* Byte-swapped magic for endianness detection */ -#define BINARY_FORMAT_VERSION 1 +#define BINARY_FORMAT_VERSION 2 /* Sentinel values for optional frame fields */ #define OPCODE_NONE 255 /* No opcode captured (u8 sentinel) */ @@ -67,17 +67,32 @@ extern "C" { static_assert(FILE_HEADER_SIZE <= FILE_HEADER_PLACEHOLDER_SIZE, "FILE_HEADER_SIZE exceeds FILE_HEADER_PLACEHOLDER_SIZE"); -/* Sample header field offsets and sizes */ -#define SMP_OFF_THREAD_ID 0 -#define SMP_SIZE_THREAD_ID sizeof(uint64_t) -#define SMP_OFF_INTERPRETER_ID (SMP_OFF_THREAD_ID + SMP_SIZE_THREAD_ID) -#define SMP_SIZE_INTERPRETER_ID sizeof(uint32_t) -#define SMP_OFF_ENCODING (SMP_OFF_INTERPRETER_ID + SMP_SIZE_INTERPRETER_ID) -#define SMP_SIZE_ENCODING sizeof(uint8_t) -#define SAMPLE_HEADER_FIXED_SIZE (SMP_OFF_ENCODING + SMP_SIZE_ENCODING) - -static_assert(SAMPLE_HEADER_FIXED_SIZE == 13, - "SAMPLE_HEADER_FIXED_SIZE must remain 13"); +/* Sample blocks */ +#define BLOCK_MAGIC "TBLK" +#define BLOCK_MAGIC_SIZE (sizeof(BLOCK_MAGIC) - 1) +#define MAX_BLOCK_SIZE (1024 * 1024) +#define MAX_BLOCK_STREAMS 4096 + +/* Block columns */ +enum { + COLUMN_STREAM_TABLE, + COLUMN_SAMPLE_STREAM_IDS, + COLUMN_TIMESTAMP_RESIDUALS, + COLUMN_STATUS_BITMAPS, + COLUMN_STACK_INSTRUCTIONS, + COLUMN_FRAME_IDS, + COLUMN_COUNT +}; + +/* Stream table entry offsets and sizes */ +#define STM_OFF_THREAD_ID 0 +#define STM_SIZE_THREAD_ID sizeof(uint64_t) +#define STM_OFF_INTERPRETER_ID (STM_OFF_THREAD_ID + STM_SIZE_THREAD_ID) +#define STM_SIZE_INTERPRETER_ID sizeof(uint32_t) +#define STREAM_ENTRY_SIZE (STM_OFF_INTERPRETER_ID + STM_SIZE_INTERPRETER_ID) + +static_assert(STREAM_ENTRY_SIZE == 12, + "STREAM_ENTRY_SIZE must remain 12"); /* Footer field offsets and sizes */ #define FTR_OFF_STRINGS 0 @@ -128,14 +143,10 @@ static_assert(PROFILE_STATS_SIZE == 56, #define MIN_STRING_ENTRY_SIZE 1 #define MIN_FRAME_ENTRY_SIZE 7 -/* Buffer sizes: 512KB balances syscall amortization against memory use, - * and aligns well with filesystem block sizes and zstd dictionary windows */ -#define WRITE_BUFFER_SIZE (512 * 1024) -#define COMPRESSED_BUFFER_SIZE (512 * 1024) - /* Compression types */ #define COMPRESSION_NONE 0 #define COMPRESSION_ZSTD 1 +#define COLUMN_CONSTANT 2 /* Column stored as one repeated byte */ /* Profiling configuration. The mode occupies the low three bits and is * stored plus one so zero remains compatible with files written before this @@ -152,17 +163,23 @@ static_assert(PROFILE_STATS_SIZE == 56, /* Stack encoding types for delta compression */ #define STACK_REPEAT 0x00 /* RLE: identical to previous, with count */ -#define STACK_FULL 0x01 /* Full stack (first sample or no match) */ -#define STACK_SUFFIX 0x02 /* Shares N frames from bottom */ -#define STACK_POP_PUSH 0x03 /* Remove M frames, add N frames */ +#define STACK_REPLACE_SUFFIX 0x01 /* Keep outer frames, append new ones */ +#define STACK_REUSE 0x02 /* Stack stored earlier in the block */ /* Maximum stack depth we'll buffer for delta encoding */ #define MAX_STACK_DEPTH 256 +/* Writer's stack cache size per block */ +#define STACK_CACHE_SIZE 4096 +#define STACK_CACHE_BUCKET_SIZE 2 +#define STACK_CACHE_BUCKET_COUNT \ + (STACK_CACHE_SIZE / STACK_CACHE_BUCKET_SIZE) + /* Initial capacities for dynamic arrays - sized to reduce reallocations */ #define INITIAL_STRING_CAPACITY 4096 #define INITIAL_FRAME_CAPACITY 4096 #define INITIAL_THREAD_CAPACITY 256 +#define INITIAL_BLOCK_CAPACITY 256 /* ============================================================================ * STATISTICS STRUCTURES @@ -174,7 +191,7 @@ typedef struct { uint64_t repeat_samples; /* Total samples encoded via RLE */ uint64_t full_records; /* Number of full stack records */ uint64_t suffix_records; /* Number of suffix match records */ - uint64_t pop_push_records; /* Number of pop-push records */ + uint64_t reuse_records; /* Number of stack reuse records */ uint64_t total_frames_written;/* Total frame indices written */ uint64_t frames_saved; /* Frames avoided due to delta encoding */ uint64_t bytes_written; /* Total bytes written (before compression) */ @@ -186,7 +203,7 @@ typedef struct { uint64_t repeat_samples; /* Samples decoded from RLE */ uint64_t full_records; /* Full stack records decoded */ uint64_t suffix_records; /* Suffix match records decoded */ - uint64_t pop_push_records; /* Pop-push records decoded */ + uint64_t reuse_records; /* Stack reuse records decoded */ uint64_t total_samples; /* Total samples reconstructed */ uint64_t stack_reconstructions; /* Number of stack array reconstructions */ } BinaryReaderStats; @@ -245,8 +262,6 @@ typedef struct { #else void *cctx; /* Placeholder */ #endif - uint8_t *compressed_buffer; - size_t compressed_buffer_size; } ZstdCompressor; /* Frame entry - combines all frame data for better cache locality. @@ -279,15 +294,13 @@ typedef struct { uint64_t prev_timestamp; uint32_t interpreter_id; + uint32_t block_stream_index; + uint32_t current_repeat; + /* Previous stack for delta encoding (frame indices, innermost first) */ uint32_t *prev_stack; size_t prev_stack_depth; size_t prev_stack_capacity; - - /* RLE pending buffer - samples waiting to be written as a repeat group */ - uint8_t *pending_rle; - size_t pending_rle_bytes; - size_t pending_rle_samples; } ThreadEntry; /* Limit errors occur before emitting an incomplete sample. Other write @@ -298,15 +311,63 @@ typedef enum { BINARY_WRITER_BROKEN, } BinaryWriterState; +/* Byte buffer for block data */ +typedef struct { + uint8_t *data; + size_t size; + size_t capacity; +} ByteBuffer; + +/* Column size and encoding */ +typedef struct { + uint32_t decoded_size; + uint32_t stored_size; + uint8_t encoding; +} ColumnDescriptor; + +/* Block header as stored in the file */ +typedef struct { + uint32_t sample_count; + uint64_t base_timestamp; + uint8_t frame_width; + uint8_t common_status_flags; + uint8_t status_bitmap_flags; + ColumnDescriptor columns[COLUMN_COUNT]; +} BlockHeader; + +/* Complete stack that later samples can REUSE */ +typedef struct { + Py_uhash_t hash; + uint32_t offset; + uint32_t depth; +} StackCacheEntry; + +/* REPEAT instruction position and count */ +typedef struct { + uint32_t offset; + uint32_t count; +} Repeat; + +/* Samples buffered until the next flush */ +typedef struct { + ByteBuffer columns[COLUMN_COUNT]; + ByteBuffer repeats; + ByteBuffer transformed; + ByteBuffer compressed; + ByteBuffer output; + StackCacheEntry cache[STACK_CACHE_BUCKET_COUNT][STACK_CACHE_BUCKET_SIZE]; + uint64_t base_timestamp; + uint32_t sample_count; + uint32_t stream_count; + uint32_t max_frame; + size_t size; +} SampleBlock; + /* Main binary writer structure */ typedef struct { FILE *fp; BinaryWriterState state; - - /* Write buffer for batched I/O */ - uint8_t *write_buffer; - size_t buffer_pos; - size_t buffer_size; + SampleBlock *block; /* Compression */ int compression_type; @@ -353,17 +414,44 @@ typedef struct { * BINARY READER STRUCTURES * ============================================================================ */ -/* Per-thread state for stack reconstruction during replay */ +/* Thread seen during replay */ typedef struct { uint64_t thread_id; uint32_t interpreter_id; - uint64_t prev_timestamp; + uint32_t block_stream_index; +} ReaderThreadState; + +typedef struct { + uint32_t frame_index; + PyObject *frame_info; +} ReaderFrameCacheEntry; - /* Reconstructed stack buffer (frame indices, innermost first) */ +/* Decoded sample block */ +typedef struct { + BlockHeader header; + uint32_t stream_count; + uint32_t frame_count; + ByteBuffer columns[COLUMN_COUNT]; + const uint32_t *stream_ids; + const uint64_t *timestamps; + const uint8_t *statuses; + const uint32_t *frames; + size_t instruction_offset; + uint32_t frame_offset; +} DecodedBlock; + +/* Per-stream replay state */ +typedef struct { + uint64_t thread_id; + uint32_t interpreter_id; + uint64_t prev_timestamp; uint32_t *current_stack; size_t current_stack_depth; - size_t current_stack_capacity; -} ReaderThreadState; + uint32_t pending_repeats; + int has_stack; + PyObject *batch; + uint8_t batch_status; +} ReaderBlockStream; /* Main binary reader structure */ typedef struct { @@ -378,9 +466,9 @@ typedef struct { /* Decompression state */ int compression_type; - /* Note: ZSTD_DCtx is not stored - created/freed during decompression */ - uint8_t *decompressed_data; - size_t decompressed_size; +#ifdef HAVE_ZSTD + ZSTD_DCtx *dctx; +#endif /* Header metadata */ uint8_t py_major; @@ -409,12 +497,24 @@ typedef struct { /* Parsed frame table: array of FrameEntry structures */ FrameEntry *frames; uint32_t frames_count; + ReaderFrameCacheEntry *frame_info_cache; + size_t frame_info_cache_size; /* Sample data region */ uint8_t *sample_data; size_t sample_data_size; - /* Per-thread state for stack reconstruction (used during replay) */ + ByteBuffer block_data; + ByteBuffer block_stream_ids; + ByteBuffer block_timestamps; + ByteBuffer block_statuses; + ByteBuffer block_frames; + ReaderBlockStream *block_streams; + uint32_t *block_stacks; + size_t block_stream_capacity; + int replaying; + + /* Threads seen during replay */ ReaderThreadState *thread_states; size_t thread_state_count; size_t thread_state_capacity; @@ -462,6 +562,14 @@ encode_varint_i32(uint8_t *buf, int32_t value) return encode_varint_u32(buf, zigzag); } +/* Encode signed 64-bit varint (zigzag encoding). Returns bytes written. */ +static inline size_t +encode_varint_i64(uint8_t *buf, int64_t value) +{ + uint64_t zigzag = ((uint64_t)value << 1) ^ (uint64_t)(value >> 63); + return encode_varint_u64(buf, zigzag); +} + /* Decode unsigned 64-bit varint (LEB128). Updates offset only on success. * On error (overflow or incomplete), offset is NOT updated, allowing callers * to detect errors via (offset == prev_offset) check. Sets PyErr on error. */ @@ -525,10 +633,64 @@ decode_varint_i32(const uint8_t *data, size_t *offset, size_t max_size) return (int32_t)((zigzag >> 1) ^ -(int32_t)(zigzag & 1)); } +/* Decode signed 64-bit varint (zigzag encoding). */ +static inline int64_t +decode_varint_i64(const uint8_t *data, size_t *offset, size_t max_size) +{ + size_t saved_offset = *offset; + uint64_t zigzag = decode_varint_u64(data, offset, max_size); + if (*offset == saved_offset) { + return 0; /* decode_varint_u64 already set PyErr */ + } + return (int64_t)((zigzag >> 1) ^ (0 - (zigzag & 1))); +} + /* ============================================================================ * SHARED UTILITY FUNCTIONS * ============================================================================ */ +/* Grow a byte buffer to hold at least size bytes */ +static inline int +buffer_reserve(ByteBuffer *buffer, size_t size) +{ + if (size <= buffer->capacity) { + return 0; + } + size_t capacity = buffer->capacity ? buffer->capacity + : INITIAL_BLOCK_CAPACITY; + while (capacity < size) { + if (capacity > SIZE_MAX / 2) { + PyErr_NoMemory(); + return -1; + } + capacity *= 2; + } + uint8_t *data = PyMem_Realloc(buffer->data, capacity); + if (data == NULL) { + PyErr_NoMemory(); + return -1; + } + buffer->data = data; + buffer->capacity = capacity; + return 0; +} + +static inline void +buffer_clear(ByteBuffer *buffer) +{ + PyMem_Free(buffer->data); + buffer->data = NULL; + buffer->size = 0; + buffer->capacity = 0; +} + +/* Bytes in one status bitmap */ +static inline size_t +status_bitmap_size(uint32_t sample_count) +{ + return ((size_t)sample_count + CHAR_BIT - 1) / CHAR_BIT; +} + /* Generic array growth - returns new pointer or NULL (sets PyErr_NoMemory) * Includes overflow checking for capacity doubling and allocation size. */ static inline void * @@ -716,6 +878,12 @@ void binary_reader_close(BinaryReader *reader); */ PyObject *binary_writer_get_stats(BinaryWriter *writer); +/* + * Decoded bytes of the samples buffered in the current block, not yet + * counted in the writer's bytes_written statistic. + */ +size_t binary_writer_buffered_size(BinaryWriter *writer); + /* * Get reader statistics as a Python dict. * diff --git a/Modules/_remote_debugging/binary_io_reader.c b/Modules/_remote_debugging/binary_io_reader.c index 80627db913ea21b..617b7356eaf9473 100644 --- a/Modules/_remote_debugging/binary_io_reader.c +++ b/Modules/_remote_debugging/binary_io_reader.c @@ -27,15 +27,14 @@ * CONSTANTS FOR BINARY FORMAT SIZES * ============================================================================ */ -/* File structure sizes */ -#define MIN_DECOMPRESS_BUFFER_SIZE (64 * 1024) /* Minimum decompression buffer */ - /* Progress callback frequency */ #define PROGRESS_CALLBACK_INTERVAL 1000 /* Cap per-batch RLE samples to bound the timestamp list (gh-151378) */ #define MAX_RLE_BATCH_SAMPLES 8192 +#define FRAME_INFO_CACHE_SIZE 1024 + /* ============================================================================ * BINARY READER IMPLEMENTATION * ============================================================================ */ @@ -230,119 +229,62 @@ reader_parse_profile_stats(BinaryReader *reader, const uint8_t *data, return 0; } -#ifdef HAVE_ZSTD -/* Maximum decompression buffer size to prevent memory exhaustion (1GB) */ -#define MAX_DECOMPRESS_SIZE (1ULL << 30) - -static inline int -reader_decompress_samples(BinaryReader *reader, const uint8_t *data) +static int +reader_decompress(BinaryReader *reader, const uint8_t *compressed_data, + size_t compressed_size, ByteBuffer *output, + size_t decoded_size) { - size_t compressed_size = reader->string_table_offset - FILE_HEADER_PLACEHOLDER_SIZE; - const uint8_t *compressed_data = data + FILE_HEADER_PLACEHOLDER_SIZE; - - /* Validate compressed data region */ - if (reader->string_table_offset < FILE_HEADER_PLACEHOLDER_SIZE) { - PyErr_SetString(PyExc_ValueError, "Invalid string table offset"); - return -1; - } - - ZSTD_DCtx *dctx = ZSTD_createDCtx(); - if (!dctx) { - PyErr_SetString(PyExc_MemoryError, "Failed to create zstd decompression context"); +#ifdef HAVE_ZSTD + /* The data must be exactly one zstd frame */ + size_t frame_size = ZSTD_findFrameCompressedSize(compressed_data, + compressed_size); + if (ZSTD_isError(frame_size) || frame_size != compressed_size) { + PyErr_SetString(PyExc_ValueError, "Corrupted zstd frame header"); return -1; } - /* Try to get exact decompressed size from frame header for optimal allocation */ + /* Check the decompressed size recorded in the frame header before + * allocating for it */ unsigned long long frame_content_size = ZSTD_getFrameContentSize(compressed_data, compressed_size); - size_t alloc_size; - if (frame_content_size == ZSTD_CONTENTSIZE_ERROR) { /* Corrupted frame header - fail early */ - ZSTD_freeDCtx(dctx); PyErr_SetString(PyExc_ValueError, "Corrupted zstd frame header"); return -1; - } else if (frame_content_size != ZSTD_CONTENTSIZE_UNKNOWN && - frame_content_size <= SIZE_MAX && - frame_content_size <= MAX_DECOMPRESS_SIZE) { - alloc_size = (size_t)frame_content_size; - } else { - alloc_size = ZSTD_DStreamOutSize() * 4; - if (alloc_size < MIN_DECOMPRESS_BUFFER_SIZE) { - alloc_size = MIN_DECOMPRESS_BUFFER_SIZE; - } } - - reader->decompressed_data = PyMem_Malloc(alloc_size); - if (!reader->decompressed_data) { - ZSTD_freeDCtx(dctx); - PyErr_NoMemory(); + if (frame_content_size != decoded_size) { + PyErr_SetString(PyExc_ValueError, + "zstd frame size does not match the decoded size"); return -1; } - ZSTD_inBuffer input = { compressed_data, compressed_size, 0 }; - size_t total_output = 0; - size_t last_result = 0; - - while (input.pos < input.size) { - if (total_output >= alloc_size) { - /* Check for overflow before doubling */ - if (alloc_size > SIZE_MAX / 2 || alloc_size * 2 > MAX_DECOMPRESS_SIZE) { - PyMem_Free(reader->decompressed_data); - reader->decompressed_data = NULL; - ZSTD_freeDCtx(dctx); - PyErr_SetString(PyExc_MemoryError, "Decompressed data exceeds maximum size"); - return -1; - } - size_t new_size = alloc_size * 2; - uint8_t *new_buf = PyMem_Realloc(reader->decompressed_data, new_size); - if (!new_buf) { - PyMem_Free(reader->decompressed_data); - reader->decompressed_data = NULL; - ZSTD_freeDCtx(dctx); - PyErr_NoMemory(); - return -1; - } - reader->decompressed_data = new_buf; - alloc_size = new_size; - } - - ZSTD_outBuffer output = { - reader->decompressed_data + total_output, - alloc_size - total_output, - 0 - }; - - last_result = ZSTD_decompressStream(dctx, &output, &input); - if (ZSTD_isError(last_result)) { - PyMem_Free(reader->decompressed_data); - reader->decompressed_data = NULL; - ZSTD_freeDCtx(dctx); - PyErr_Format(PyExc_ValueError, "zstd decompression error: %s", - ZSTD_getErrorName(last_result)); - return -1; - } - - _Py_MSAN_UNPOISON(output.dst, output.pos); - total_output += output.pos; + /* Append the decoded data to the output buffer */ + if (buffer_reserve(output, output->size + decoded_size) < 0) { + return -1; + } + uint8_t *decoded = output->data + output->size; + size_t result = ZSTD_decompressDCtx(reader->dctx, decoded, decoded_size, + compressed_data, compressed_size); + if (ZSTD_isError(result)) { + PyErr_Format(PyExc_ValueError, "zstd decompression error: %s", + ZSTD_getErrorName(result)); + return -1; } - /* Verify decompression is complete (last_result == 0 means frame is complete) */ - if (last_result != 0) { - PyMem_Free(reader->decompressed_data); - reader->decompressed_data = NULL; - ZSTD_freeDCtx(dctx); + /* Verify decompression is complete */ + if (result != decoded_size) { PyErr_SetString(PyExc_ValueError, "Incomplete zstd frame: data may be truncated"); return -1; } - ZSTD_freeDCtx(dctx); - reader->decompressed_size = total_output; - reader->sample_data = reader->decompressed_data; - reader->sample_data_size = reader->decompressed_size; - + _Py_MSAN_UNPOISON(decoded, decoded_size); + output->size += decoded_size; return 0; -} +#else + PyErr_SetString(PyExc_RuntimeError, + "File uses zstd compression but zstd support not compiled in"); + return -1; #endif +} /* Reject a table/run count whose entries cannot fit in the bytes still * available; a malicious file could otherwise drive a huge allocation. @@ -681,10 +623,11 @@ binary_reader_open(PyObject *path) goto error; } - /* Handle compressed data */ if (reader->compression_type == COMPRESSION_ZSTD) { #ifdef HAVE_ZSTD - if (reader_decompress_samples(reader, data) < 0) { + reader->dctx = ZSTD_createDCtx(); + if (!reader->dctx) { + PyErr_SetString(PyExc_MemoryError, "Failed to create zstd decompression context"); goto error; } #else @@ -692,10 +635,10 @@ binary_reader_open(PyObject *path) "File uses zstd compression but zstd support not compiled in"); goto error; #endif - } else { - reader->sample_data = data + FILE_HEADER_PLACEHOLDER_SIZE; - reader->sample_data_size = reader->string_table_offset - FILE_HEADER_PLACEHOLDER_SIZE; } + reader->sample_data = data + FILE_HEADER_PLACEHOLDER_SIZE; + reader->sample_data_size = + reader->string_table_offset - FILE_HEADER_PLACEHOLDER_SIZE; /* Parse string and frame tables */ if (reader_parse_string_table(reader, data, file_size) < 0) { @@ -754,170 +697,85 @@ reader_get_or_create_thread_state(BinaryReader *reader, uint64_t thread_id, memset(ts, 0, sizeof(ReaderThreadState)); ts->thread_id = thread_id; ts->interpreter_id = interpreter_id; - ts->prev_timestamp = reader->start_time_us; - ts->current_stack_capacity = MAX_STACK_DEPTH; - ts->current_stack = PyMem_Malloc(ts->current_stack_capacity * sizeof(uint32_t)); - if (!ts->current_stack) { - PyErr_NoMemory(); - return NULL; - } - // Increment count only after successful allocation to avoid - // leaving a half-initialized entry visible to future lookups reader->thread_state_count++; return ts; } -/* ============================================================================ - * STACK DECODING HELPERS - * ============================================================================ */ - -/* Validate that final_depth fits in the stack buffer. - * Uses uint64_t to prevent overflow on 32-bit platforms. */ -static inline int -validate_stack_depth(ReaderThreadState *ts, uint64_t final_depth) -{ - if (final_depth > ts->current_stack_capacity) { - PyErr_Format(PyExc_ValueError, - "Final stack depth %llu exceeds capacity %zu", - (unsigned long long)final_depth, ts->current_stack_capacity); - return -1; - } - return 0; -} - -/* Decode a full stack from sample data. - * Updates ts->current_stack and ts->current_stack_depth. - * Returns 0 on success, -1 on error (bounds violation). */ -static inline int -decode_stack_full(ReaderThreadState *ts, const uint8_t *data, - size_t *offset, size_t max_size) +static PyObject * +reader_get_frame_info(RemoteDebuggingState *state, BinaryReader *reader, + uint32_t frame_index) { - size_t prev_offset = *offset; - uint32_t depth = decode_varint_u32(data, offset, max_size); - if (*offset == prev_offset) { - return -1; - } - - /* Validate depth against capacity to prevent buffer overflow */ - if (depth > ts->current_stack_capacity) { - PyErr_Format(PyExc_ValueError, - "Stack depth %u exceeds capacity %zu", depth, ts->current_stack_capacity); - return -1; + if (frame_index >= reader->frames_count) { + PyErr_Format(PyExc_ValueError, "Invalid frame index: %u", frame_index); + return NULL; } - - ts->current_stack_depth = depth; - for (uint32_t i = 0; i < depth; i++) { - size_t prev_offset = *offset; - ts->current_stack[i] = decode_varint_u32(data, offset, max_size); - if (*offset == prev_offset) { - return -1; + if (reader->frame_info_cache == NULL) { + size_t cache_size = Py_MIN(reader->frames_count, + FRAME_INFO_CACHE_SIZE); + reader->frame_info_cache = PyMem_Calloc( + cache_size, sizeof(*reader->frame_info_cache)); + if (reader->frame_info_cache == NULL) { + PyErr_NoMemory(); + return NULL; } + reader->frame_info_cache_size = cache_size; } - return 0; -} - -/* Decode a suffix-encoded stack from sample data. - * The suffix encoding shares frames from the bottom of the previous stack. - * Returns 0 on success, -1 on error (bounds violation). */ -static inline int -decode_stack_suffix(ReaderThreadState *ts, const uint8_t *data, - size_t *offset, size_t max_size) -{ - size_t prev_offset = *offset; - uint32_t shared = decode_varint_u32(data, offset, max_size); - if (*offset == prev_offset) { - return -1; - } - prev_offset = *offset; - uint32_t new_count = decode_varint_u32(data, offset, max_size); - if (*offset == prev_offset) { - return -1; + ReaderFrameCacheEntry *entry = &reader->frame_info_cache[ + frame_index % reader->frame_info_cache_size]; + if (entry->frame_info != NULL && entry->frame_index == frame_index) { + return Py_NewRef(entry->frame_info); } - /* Validate shared doesn't exceed current stack depth */ - if (shared > ts->current_stack_depth) { - PyErr_Format(PyExc_ValueError, - "Shared count %u exceeds current stack depth %zu", - shared, ts->current_stack_depth); - return -1; - } - - /* Use uint64_t to prevent overflow on 32-bit platforms */ - uint64_t final_depth = (uint64_t)shared + new_count; - if (validate_stack_depth(ts, final_depth) < 0) { - return -1; + FrameEntry *frame = &reader->frames[frame_index]; + if (frame->filename_idx >= reader->strings_count || + frame->funcname_idx >= reader->strings_count) { + PyErr_SetString(PyExc_ValueError, "Invalid string index in frame"); + return NULL; } - /* Move shared frames (from bottom of stack) to make room for new frames at the top */ - if (new_count > 0 && shared > 0) { - /* Defensive check: ensure subtraction won't underflow. - * This should already be guaranteed by the check above, but we add - * this assertion as defense-in-depth against stack corruption. */ - if (ts->current_stack_depth < shared) { - PyErr_SetString(PyExc_ValueError, - "Internal error: stack corruption detected in suffix decoding"); - return -1; - } - size_t prev_shared_start = ts->current_stack_depth - shared; - memmove(&ts->current_stack[new_count], - &ts->current_stack[prev_shared_start], - shared * sizeof(uint32_t)); + PyObject *frame_info = PyStructSequence_New(state->FrameInfo_Type); + if (!frame_info) { + return NULL; } - for (uint32_t i = 0; i < new_count; i++) { - size_t prev_offset = *offset; - ts->current_stack[i] = decode_varint_u32(data, offset, max_size); - if (*offset == prev_offset) { - return -1; + /* Build location tuple with full position info */ + PyObject *location; + if (frame->lineno != LOCATION_NOT_AVAILABLE) { + location = Py_BuildValue("(iiii)", + frame->lineno, + frame->end_lineno, + frame->column, + frame->end_column); + if (!location) { + Py_DECREF(frame_info); + return NULL; } } - ts->current_stack_depth = final_depth; - return 0; -} - -/* Decode a pop-push encoded stack from sample data. - * Pops frames from the top and pushes new frames. - * Returns 0 on success, -1 on error (bounds violation). */ -static inline int -decode_stack_pop_push(ReaderThreadState *ts, const uint8_t *data, - size_t *offset, size_t max_size) -{ - size_t prev_offset = *offset; - uint32_t pop = decode_varint_u32(data, offset, max_size); - if (*offset == prev_offset) { - return -1; - } - prev_offset = *offset; - uint32_t push = decode_varint_u32(data, offset, max_size); - if (*offset == prev_offset) { - return -1; + else { + location = Py_NewRef(Py_None); } - size_t keep = (ts->current_stack_depth > pop) ? ts->current_stack_depth - pop : 0; - /* Use uint64_t to prevent overflow on 32-bit platforms */ - uint64_t final_depth = (uint64_t)keep + push; - if (validate_stack_depth(ts, final_depth) < 0) { - return -1; + /* Build opcode object */ + PyObject *opcode_obj; + if (frame->opcode != OPCODE_NONE) { + opcode_obj = PyLong_FromLong(frame->opcode); + if (!opcode_obj) { + Py_DECREF(location); + Py_DECREF(frame_info); + return NULL; + } } - - /* Move kept frames (from bottom of stack) to make room for new frames at the top. - * Even when push == 0, we need to move kept frames to index 0 if pop > 0. */ - if (keep > 0) { - memmove(&ts->current_stack[push], - &ts->current_stack[pop], - keep * sizeof(uint32_t)); + else { + opcode_obj = Py_NewRef(Py_None); } - for (uint32_t i = 0; i < push; i++) { - size_t prev_offset = *offset; - ts->current_stack[i] = decode_varint_u32(data, offset, max_size); - /* If offset didn't advance, varint decoding failed */ - if (*offset == prev_offset) { - return -1; - } - } - ts->current_stack_depth = final_depth; - return 0; + PyStructSequence_SetItem(frame_info, 0, Py_NewRef(reader->strings[frame->filename_idx])); + PyStructSequence_SetItem(frame_info, 1, location); + PyStructSequence_SetItem(frame_info, 2, Py_NewRef(reader->strings[frame->funcname_idx])); + PyStructSequence_SetItem(frame_info, 3, opcode_obj); + Py_XSETREF(entry->frame_info, Py_NewRef(frame_info)); + entry->frame_index = frame_index; + return frame_info; } /* Build a Python list of FrameInfo objects from frame indices */ @@ -931,60 +789,11 @@ build_frame_list(RemoteDebuggingState *state, BinaryReader *reader, } for (size_t k = 0; k < stack_depth; k++) { - uint32_t frame_idx = frame_indices[k]; - if (frame_idx >= reader->frames_count) { - PyErr_Format(PyExc_ValueError, "Invalid frame index: %u", frame_idx); - goto error; - } - - FrameEntry *frame = &reader->frames[frame_idx]; - - if (frame->filename_idx >= reader->strings_count || - frame->funcname_idx >= reader->strings_count) { - PyErr_SetString(PyExc_ValueError, "Invalid string index in frame"); + PyObject *frame_info = reader_get_frame_info( + state, reader, frame_indices[k]); + if (frame_info == NULL) { goto error; } - - PyObject *frame_info = PyStructSequence_New(state->FrameInfo_Type); - if (!frame_info) { - goto error; - } - - /* Build location tuple with full position info */ - PyObject *location; - if (frame->lineno != LOCATION_NOT_AVAILABLE) { - location = Py_BuildValue("(iiii)", - frame->lineno, - frame->end_lineno, - frame->column, - frame->end_column); - if (!location) { - Py_DECREF(frame_info); - goto error; - } - } - else { - location = Py_NewRef(Py_None); - } - - /* Build opcode object */ - PyObject *opcode_obj; - if (frame->opcode != OPCODE_NONE) { - opcode_obj = PyLong_FromLong(frame->opcode); - if (!opcode_obj) { - Py_DECREF(location); - Py_DECREF(frame_info); - goto error; - } - } - else { - opcode_obj = Py_NewRef(Py_None); - } - - PyStructSequence_SetItem(frame_info, 0, Py_NewRef(reader->strings[frame->filename_idx])); - PyStructSequence_SetItem(frame_info, 1, location); - PyStructSequence_SetItem(frame_info, 2, Py_NewRef(reader->strings[frame->funcname_idx])); - PyStructSequence_SetItem(frame_info, 3, opcode_obj); PyList_SET_ITEM(frame_list, k, frame_info); } @@ -1101,252 +910,726 @@ invoke_progress_callback(PyObject *callback, Py_ssize_t current, uint64_t total) return 0; } -Py_ssize_t -binary_reader_replay(BinaryReader *reader, PyObject *collector, PyObject *progress_callback) +/* ============================================================================ + * SAMPLE BLOCKS + * ============================================================================ */ + +static int +block_error(const char *message) { - PyObject *collect_method; - int has_collect = PyObject_GetOptionalAttrString(collector, "collect", &collect_method); - if (has_collect < 0) { - return -1; + PyErr_Format(PyExc_ValueError, "Invalid sample block: %s", message); + return -1; +} + +static inline int +block_read_varint(const ByteBuffer *column, size_t *offset, uint64_t *value) +{ + size_t prev_offset = *offset; + *value = decode_varint_u64(column->data, offset, column->size); + return *offset == prev_offset ? -1 : 0; +} + +static int +block_read_varint_u32(const ByteBuffer *column, size_t *offset, + uint32_t *value) +{ + size_t prev_offset = *offset; + *value = decode_varint_u32(column->data, offset, column->size); + return *offset == prev_offset ? -1 : 0; +} + +static int +reader_parse_block_header(BinaryReader *reader, const ByteBuffer *section, + size_t *offset, BlockHeader *header) +{ + /* Block magic "TBLK" */ + if (section->size - *offset < BLOCK_MAGIC_SIZE) { + return block_error("truncated header"); } - if (has_collect == 0) { - PyErr_SetString(PyExc_TypeError, "Collector must have a collect() method"); - return -1; + if (memcmp(section->data + *offset, BLOCK_MAGIC, BLOCK_MAGIC_SIZE) != 0) { + return block_error("wrong magic"); } - Py_DECREF(collect_method); + *offset += BLOCK_MAGIC_SIZE; - /* Get module state for struct sequence types */ - PyObject *module = PyImport_ImportModule("_remote_debugging"); - if (!module) { + /* Sample count and base timestamp (varints) */ + if (block_read_varint_u32(section, offset, &header->sample_count) < 0 || + block_read_varint(section, offset, &header->base_timestamp) < 0) { return -1; } - RemoteDebuggingState *state = RemoteDebugging_GetState(module); - Py_DECREF(module); - if (!state) { - PyErr_SetString(PyExc_RuntimeError, "Failed to get module state"); + /* Frame ID width and the two status flag masks (1 byte each) */ + if (section->size - *offset < 3) { + return block_error("truncated header"); + } + header->frame_width = section->data[(*offset)++]; + header->common_status_flags = section->data[(*offset)++]; + header->status_bitmap_flags = section->data[(*offset)++]; + if (header->sample_count == 0) { + return block_error("no samples"); + } + if (header->frame_width < 1 || header->frame_width > sizeof(uint32_t)) { + return block_error("frame ID width out of range"); + } + if (header->common_status_flags & header->status_bitmap_flags) { + return block_error("common and varying status flags overlap"); + } + + size_t decoded_total = 0; + for (unsigned column = 0; column < COLUMN_COUNT; column++) { + ColumnDescriptor *descriptor = &header->columns[column]; + /* Decoded size (varint) and encoding (1 byte) */ + if (block_read_varint_u32(section, offset, + &descriptor->decoded_size) < 0) { + return -1; + } + if (*offset >= section->size) { + return block_error("truncated column descriptor"); + } + descriptor->encoding = section->data[(*offset)++]; + + /* Stored size: implied for raw and constant, a varint for zstd */ + switch (descriptor->encoding) { + case COMPRESSION_NONE: + descriptor->stored_size = descriptor->decoded_size; + break; + case COLUMN_CONSTANT: + descriptor->stored_size = 1; + break; + case COMPRESSION_ZSTD: + if (reader->compression_type != COMPRESSION_ZSTD) { + return block_error("zstd column in an uncompressed file"); + } + if (block_read_varint_u32(section, offset, + &descriptor->stored_size) < 0) { + return -1; + } + break; + default: + return block_error("unknown column encoding"); + } + + if (descriptor->encoding != COMPRESSION_NONE && + descriptor->decoded_size == 0) { + return block_error("encoded column is empty"); + } + if (descriptor->decoded_size > MAX_BLOCK_SIZE - decoded_total) { + return block_error("decoded columns exceed the block limit"); + } + decoded_total += descriptor->decoded_size; + } + + /* Each sample needs at least one byte of stream index and residual */ + if (header->sample_count > + header->columns[COLUMN_SAMPLE_STREAM_IDS].decoded_size || + header->sample_count > + header->columns[COLUMN_TIMESTAMP_RESIDUALS].decoded_size) { + return block_error("more samples than stored stream indices"); + } + return 0; +} + +static int +decode_block_columns(BinaryReader *reader, const ByteBuffer *section, + size_t *offset, DecodedBlock *block) +{ + const ColumnDescriptor *descriptors = block->header.columns; + /* Constant and zstd columns share one buffer, reused across blocks. + * It is reserved up front so decoded columns never move. */ + size_t buffered_total = 0; + for (unsigned column = 0; column < COLUMN_COUNT; column++) { + if (descriptors[column].encoding != COMPRESSION_NONE) { + buffered_total += descriptors[column].decoded_size; + } + } + ByteBuffer *buffer = &reader->block_data; + if (buffer_reserve(buffer, buffered_total) < 0) { return -1; } - size_t offset = 0; - Py_ssize_t replayed = 0; + buffer->size = 0; + for (unsigned column = 0; column < COLUMN_COUNT; column++) { + const ColumnDescriptor *descriptor = &descriptors[column]; + if (descriptor->stored_size > section->size - *offset) { + return block_error("column extends past the sample section"); + } + uint8_t *stored_data = section->data + *offset; + ByteBuffer *decoded = &block->columns[column]; + decoded->size = descriptor->decoded_size; + switch (descriptor->encoding) { + case COMPRESSION_NONE: + /* Raw columns point into the file data */ + decoded->data = stored_data; + break; + case COLUMN_CONSTANT: + decoded->data = buffer->data + buffer->size; + memset(decoded->data, stored_data[0], decoded->size); + buffer->size += decoded->size; + break; + case COMPRESSION_ZSTD: + decoded->data = buffer->data + buffer->size; + if (reader_decompress(reader, stored_data, + descriptor->stored_size, buffer, + decoded->size) < 0) { + return -1; + } + break; + default: + return block_error("unknown column encoding"); + } + *offset += descriptor->stored_size; + } + return 0; +} - /* Initial progress callback at 0% */ - if (invoke_progress_callback(progress_callback, 0, reader->sample_count) < 0) { +static int +validate_block_columns(DecodedBlock *block) +{ + size_t stream_table_size = block->columns[COLUMN_STREAM_TABLE].size; + if (stream_table_size % STREAM_ENTRY_SIZE != 0) { + return block_error("stream table has a partial entry"); + } + size_t stream_count = stream_table_size / STREAM_ENTRY_SIZE; + if (stream_count == 0 || stream_count > MAX_BLOCK_STREAMS) { + return block_error("stream count out of range"); + } + block->stream_count = (uint32_t)stream_count; + + size_t frame_ids_size = block->columns[COLUMN_FRAME_IDS].size; + if (frame_ids_size % block->header.frame_width != 0) { + return block_error("frame ID column has a partial value"); + } + block->frame_count = + (uint32_t)(frame_ids_size / block->header.frame_width); + + size_t plane_size = status_bitmap_size(block->header.sample_count); + size_t plane_count = + (size_t)_Py_popcount32(block->header.status_bitmap_flags); + const ByteBuffer *statuses = &block->columns[COLUMN_STATUS_BITMAPS]; + if (statuses->size != plane_size * plane_count) { + return block_error("status bitmaps have the wrong size"); + } + unsigned used_bits = block->header.sample_count % CHAR_BIT; + for (size_t plane = 0; used_bits && plane < plane_count; plane++) { + if (statuses->data[(plane + 1) * plane_size - 1] >> used_bits) { + return block_error("status bitmap padding is not zero"); + } + } + return 0; +} + +static int +reserve_block_streams(BinaryReader *reader, uint32_t stream_count) +{ + if (stream_count <= reader->block_stream_capacity) { + return 0; + } + ReaderBlockStream *streams = PyMem_Realloc( + reader->block_streams, stream_count * sizeof(ReaderBlockStream)); + if (streams == NULL) { + PyErr_NoMemory(); + return -1; + } + reader->block_streams = streams; + uint32_t *stacks = PyMem_Realloc( + reader->block_stacks, + stream_count * MAX_STACK_DEPTH * sizeof(uint32_t)); + if (stacks == NULL) { + PyErr_NoMemory(); return -1; } + reader->block_stacks = stacks; + reader->block_stream_capacity = stream_count; + return 0; +} - while (offset < reader->sample_data_size) { - /* Read thread_id (8 bytes) + interpreter_id (4 bytes) + encoding byte */ - if (reader->sample_data_size - offset < SAMPLE_HEADER_FIXED_SIZE) { - PyErr_Format(PyExc_ValueError, - "Truncated sample data: %zu trailing bytes", - reader->sample_data_size - offset); +static int +reader_parse_stream_table(BinaryReader *reader, const DecodedBlock *block) +{ + const uint8_t *stream_table = block->columns[COLUMN_STREAM_TABLE].data; + uint64_t initial_prev_timestamp = + block->header.base_timestamp - reader->sample_interval_us; + memset(reader->block_streams, 0, + block->stream_count * sizeof(ReaderBlockStream)); + for (uint32_t i = 0; i < block->stream_count; i++) { + /* Read thread_id (8 bytes) + interpreter_id (4 bytes) */ + const uint8_t *entry = stream_table + (size_t)i * STREAM_ENTRY_SIZE; + uint64_t thread_id; + uint32_t interpreter_id; + memcpy(&thread_id, entry + STM_OFF_THREAD_ID, STM_SIZE_THREAD_ID); + memcpy(&interpreter_id, entry + STM_OFF_INTERPRETER_ID, + STM_SIZE_INTERPRETER_ID); + ReaderThreadState *thread = reader_get_or_create_thread_state( + reader, SWAP64_IF(reader->needs_swap, thread_id), + SWAP32_IF(reader->needs_swap, interpreter_id)); + if (thread == NULL) { return -1; } + /* Each thread may appear only once in a block. An index left over + * from an earlier block points at a different thread here. */ + uint32_t previous_index = thread->block_stream_index; + if (previous_index < i && + reader->block_streams[previous_index].thread_id == + thread->thread_id && + reader->block_streams[previous_index].interpreter_id == + thread->interpreter_id) { + return block_error("stream table repeats a thread"); + } + thread->block_stream_index = i; - /* Use memcpy to avoid strict aliasing violations, then byte-swap if needed */ - uint64_t thread_id_raw; - uint32_t interpreter_id_raw; - memcpy(&thread_id_raw, &reader->sample_data[offset], SMP_SIZE_THREAD_ID); - offset += SMP_SIZE_THREAD_ID; + /* Each stream's first expected timestamp is the base timestamp */ + ReaderBlockStream *stream = &reader->block_streams[i]; + stream->thread_id = thread->thread_id; + stream->interpreter_id = thread->interpreter_id; + stream->prev_timestamp = initial_prev_timestamp; + stream->current_stack = + reader->block_stacks + (size_t)i * MAX_STACK_DEPTH; + } + return 0; +} - memcpy(&interpreter_id_raw, &reader->sample_data[offset], SMP_SIZE_INTERPRETER_ID); - offset += SMP_SIZE_INTERPRETER_ID; +static int +decode_samples(BinaryReader *reader, DecodedBlock *block) +{ + size_t sample_count = block->header.sample_count; + if (buffer_reserve(&reader->block_stream_ids, + sample_count * sizeof(uint32_t)) < 0 || + buffer_reserve(&reader->block_timestamps, + sample_count * sizeof(uint64_t)) < 0) { + return -1; + } + uint32_t *stream_ids = (uint32_t *)reader->block_stream_ids.data; + uint64_t *timestamps = (uint64_t *)reader->block_timestamps.data; + const ByteBuffer *stream_id_column = + &block->columns[COLUMN_SAMPLE_STREAM_IDS]; + const ByteBuffer *residual_column = + &block->columns[COLUMN_TIMESTAMP_RESIDUALS]; + size_t stream_id_offset = 0; + size_t residual_offset = 0; + for (uint32_t sample = 0; sample < block->header.sample_count; sample++) { + uint64_t stream_id; + if (block_read_varint(stream_id_column, &stream_id_offset, + &stream_id) < 0) { + return -1; + } + if (stream_id >= block->stream_count) { + return block_error("sample refers to an unknown stream"); + } + size_t prev_offset = residual_offset; + int64_t residual = decode_varint_i64(residual_column->data, + &residual_offset, + residual_column->size); + if (residual_offset == prev_offset) { + return -1; + } - uint64_t thread_id = SWAP64_IF(reader->needs_swap, thread_id_raw); - uint32_t interpreter_id = SWAP32_IF(reader->needs_swap, interpreter_id_raw); + /* Timestamp is the expected time plus the stored residual */ + ReaderBlockStream *stream = &reader->block_streams[stream_id]; + stream->prev_timestamp += + reader->sample_interval_us + (uint64_t)residual; - /* Get or create thread state for reconstruction */ - ReaderThreadState *ts = reader_get_or_create_thread_state(reader, thread_id, interpreter_id); - if (!ts) { - return -1; + stream_ids[sample] = (uint32_t)stream_id; + timestamps[sample] = stream->prev_timestamp; + } + if (stream_id_offset != stream_id_column->size) { + return block_error("stream index column has trailing data"); + } + if (residual_offset != residual_column->size) { + return block_error("timestamp column has trailing data"); + } + block->stream_ids = stream_ids; + block->timestamps = timestamps; + return 0; +} + +static int +decode_statuses(BinaryReader *reader, DecodedBlock *block) +{ + if (buffer_reserve(&reader->block_statuses, + block->header.sample_count) < 0) { + return -1; + } + /* Start from the common flags, then add each varying flag's bitmap */ + uint8_t *statuses = reader->block_statuses.data; + memset(statuses, block->header.common_status_flags, + block->header.sample_count); + const uint8_t *plane = block->columns[COLUMN_STATUS_BITMAPS].data; + size_t plane_size = status_bitmap_size(block->header.sample_count); + for (unsigned bit = 0; bit < CHAR_BIT; bit++) { + if (!(block->header.status_bitmap_flags & (1U << bit))) { + continue; } + for (uint32_t i = 0; i < block->header.sample_count; i++) { + statuses[i] |= ((plane[i / CHAR_BIT] >> (i % CHAR_BIT)) & 1U) + << bit; + } + plane += plane_size; + } + block->statuses = statuses; + return 0; +} - /* Read encoding byte */ - uint8_t encoding = reader->sample_data[offset++]; - - switch (encoding) { - case STACK_REPEAT: { - /* RLE repeat: [count: varint] [delta: varint, status: 1]... */ - size_t prev_offset = offset; - uint32_t count = decode_varint_u32(reader->sample_data, &offset, reader->sample_data_size); - /* Detect varint decode failure */ - if (offset == prev_offset) { - PyErr_SetString(PyExc_ValueError, "Malformed varint for RLE count"); - return -1; - } +static int +decode_frame_ids(BinaryReader *reader, DecodedBlock *block) +{ + size_t frame_count = block->frame_count; + size_t frames_size = frame_count * sizeof(uint32_t); + if (buffer_reserve(&reader->block_frames, frames_size) < 0) { + return -1; + } + uint32_t *frames = (uint32_t *)reader->block_frames.data; + block->frames = frames; + if (frame_count == 0) { + return 0; + } - /* Reject a count larger than the remaining bytes can hold; each - * RLE sample needs at least 2 bytes (1-byte min varint + status). */ - if (reader_validate_count("RLE", count, - reader->sample_data_size - offset, 2) < 0) { - return -1; - } - if ((uint64_t)count > (uint64_t)PY_SSIZE_T_MAX - (uint64_t)replayed) { - PyErr_SetString(PyExc_OverflowError, - "Sample count exceeds Py_ssize_t maximum"); - return -1; - } + /* Combine byte b of every frame index from plane b, low byte first */ + memset(frames, 0, frames_size); + const uint8_t *plane = block->columns[COLUMN_FRAME_IDS].data; + for (unsigned byte = 0; byte < block->header.frame_width; byte++) { + for (size_t i = 0; i < frame_count; i++) { + frames[i] |= (uint32_t)plane[i] << (byte * CHAR_BIT); + } + plane += frame_count; + } + for (size_t i = 0; i < frame_count; i++) { + if (frames[i] >= reader->frames_count) { + return block_error("frame index out of range"); + } + } + return 0; +} - reader->stats.repeat_records++; - reader->stats.repeat_samples += count; - - /* Process RLE samples, batching by status */ - PyObject *timestamps_list = NULL; - uint8_t batch_status = 0; - Py_ssize_t batch_idx = 0; - - for (uint32_t i = 0; i < count; i++) { - size_t delta_prev_offset = offset; - uint64_t delta = decode_varint_u64(reader->sample_data, &offset, reader->sample_data_size); - if (offset == delta_prev_offset) { - Py_XDECREF(timestamps_list); - PyErr_SetString(PyExc_ValueError, "Malformed varint in RLE sample data"); - return -1; - } - if (offset >= reader->sample_data_size) { - Py_XDECREF(timestamps_list); - PyErr_SetString(PyExc_ValueError, "Unexpected end of sample data in RLE"); - return -1; - } - uint8_t status = reader->sample_data[offset++]; - ts->prev_timestamp += delta; - - /* Start new batch on first sample or status change */ - if (i == 0 || status != batch_status - || batch_idx >= MAX_RLE_BATCH_SAMPLES) { - if (timestamps_list) { - int rc = emit_sample(state, collector, thread_id, interpreter_id, - batch_status, ts->current_stack, ts->current_stack_depth, - reader, timestamps_list); - Py_DECREF(timestamps_list); - if (rc < 0) { - return -1; - } - } - timestamps_list = PyList_New(0); - if (!timestamps_list) { - return -1; - } - batch_status = status; - batch_idx = 0; - } - - PyObject *ts_obj = PyLong_FromUnsignedLongLong(ts->prev_timestamp); - if (!ts_obj) { - Py_DECREF(timestamps_list); - return -1; - } - int append_rc = PyList_Append(timestamps_list, ts_obj); - Py_DECREF(ts_obj); - if (append_rc < 0) { - Py_DECREF(timestamps_list); - return -1; - } - batch_idx++; - } +static int +decode_block(BinaryReader *reader, const ByteBuffer *section, + size_t *offset, DecodedBlock *block) +{ + block->instruction_offset = 0; + block->frame_offset = 0; + if (reader_parse_block_header(reader, section, offset, + &block->header) < 0 || + decode_block_columns(reader, section, offset, block) < 0 || + validate_block_columns(block) < 0 || + reserve_block_streams(reader, block->stream_count) < 0 || + reader_parse_stream_table(reader, block) < 0 || + decode_samples(reader, block) < 0 || + decode_statuses(reader, block) < 0 || + decode_frame_ids(reader, block) < 0) { + return -1; + } + return 0; +} - /* Emit final batch */ - if (timestamps_list) { - int rc = emit_sample(state, collector, thread_id, interpreter_id, - batch_status, ts->current_stack, ts->current_stack_depth, - reader, timestamps_list); - Py_DECREF(timestamps_list); - if (rc < 0) { - return -1; - } - } +static int +flush_stream_batch(BinaryReader *reader, RemoteDebuggingState *state, + PyObject *collector, ReaderBlockStream *stream) +{ + if (stream->batch == NULL) { + return 0; + } + /* Stacks are kept outermost first; collectors expect innermost first */ + uint32_t stack[MAX_STACK_DEPTH]; + for (size_t i = 0; i < stream->current_stack_depth; i++) { + stack[i] = stream->current_stack[stream->current_stack_depth - 1 - i]; + } + int result = emit_sample(state, collector, stream->thread_id, + stream->interpreter_id, stream->batch_status, + stack, stream->current_stack_depth, reader, + stream->batch); + Py_CLEAR(stream->batch); + return result; +} - replayed += count; - reader->stats.total_samples += count; +static int +decode_stack_repeat(BinaryReader *reader, DecodedBlock *block, + ReaderBlockStream *stream) +{ + const ByteBuffer *instructions = + &block->columns[COLUMN_STACK_INSTRUCTIONS]; + size_t *offset = &block->instruction_offset; + uint64_t count; + if (block_read_varint(instructions, offset, &count) < 0) { + return -1; + } + if (!stream->has_stack) { + return block_error("first stack of a stream cannot repeat"); + } + if (count == 0 || count > block->header.sample_count) { + return block_error("repeat count out of range"); + } + stream->pending_repeats = (uint32_t)(count - 1); + reader->stats.repeat_records++; + reader->stats.repeat_samples++; + return 0; +} - /* Progress callback after batch */ - if (replayed % PROGRESS_CALLBACK_INTERVAL < count) { - if (invoke_progress_callback(progress_callback, replayed, reader->sample_count) < 0) { - return -1; - } - } - break; - } +static int +decode_stack_replace_suffix(BinaryReader *reader, DecodedBlock *block, + ReaderBlockStream *stream) +{ + const ByteBuffer *instructions = + &block->columns[COLUMN_STACK_INSTRUCTIONS]; + size_t *offset = &block->instruction_offset; + uint64_t keep, count; + if (block_read_varint(instructions, offset, &keep) < 0 || + block_read_varint(instructions, offset, &count) < 0) { + return -1; + } + if (keep > stream->current_stack_depth || count > MAX_STACK_DEPTH - keep) { + return block_error("stack prefix out of range"); + } + if (count > block->frame_count - block->frame_offset) { + return block_error("stack uses more frame IDs than stored"); + } + if (count > 0) { + memcpy(&stream->current_stack[keep], + &block->frames[block->frame_offset], count * sizeof(uint32_t)); + } + block->frame_offset += (uint32_t)count; + stream->current_stack_depth = (size_t)(keep + count); + if (keep == 0) { + reader->stats.full_records++; + } + else { + reader->stats.suffix_records++; + } + return 0; +} - case STACK_FULL: - case STACK_SUFFIX: - case STACK_POP_PUSH: { - /* All three encodings share: [delta: varint] [status: 1] ... */ - size_t prev_offset = offset; - uint64_t delta = decode_varint_u64(reader->sample_data, &offset, reader->sample_data_size); - /* Detect varint decode failure: offset unchanged means error */ - if (offset == prev_offset) { - PyErr_SetString(PyExc_ValueError, "Malformed varint in sample data"); - return -1; - } - if (offset >= reader->sample_data_size) { - PyErr_SetString(PyExc_ValueError, "Unexpected end of sample data"); - return -1; - } - uint8_t status = reader->sample_data[offset++]; - ts->prev_timestamp += delta; - - if (encoding == STACK_FULL) { - if (decode_stack_full(ts, reader->sample_data, &offset, reader->sample_data_size) < 0) { - return -1; - } - reader->stats.full_records++; - } else if (encoding == STACK_SUFFIX) { - if (decode_stack_suffix(ts, reader->sample_data, &offset, reader->sample_data_size) < 0) { - return -1; - } - reader->stats.suffix_records++; - } else { /* STACK_POP_PUSH */ - if (decode_stack_pop_push(ts, reader->sample_data, &offset, reader->sample_data_size) < 0) { - return -1; - } - reader->stats.pop_push_records++; - } - reader->stats.stack_reconstructions++; +static int +decode_stack_reuse(BinaryReader *reader, DecodedBlock *block, + ReaderBlockStream *stream) +{ + const ByteBuffer *instructions = + &block->columns[COLUMN_STACK_INSTRUCTIONS]; + size_t *offset = &block->instruction_offset; + uint64_t start, count; + if (block_read_varint(instructions, offset, &start) < 0 || + block_read_varint(instructions, offset, &count) < 0) { + return -1; + } + if (count > MAX_STACK_DEPTH) { + return block_error("stack is too deep"); + } + if (start > block->frame_offset || count > block->frame_offset - start) { + return block_error("reused stack is not stored earlier"); + } + if (count > 0) { + memcpy(stream->current_stack, &block->frames[start], + count * sizeof(uint32_t)); + } + stream->current_stack_depth = (size_t)count; + reader->stats.reuse_records++; + return 0; +} - /* Build single-element timestamp list */ - PyObject *ts_obj = PyLong_FromUnsignedLongLong(ts->prev_timestamp); - if (!ts_obj) { - return -1; - } - PyObject *timestamps_list = PyList_New(1); - if (!timestamps_list) { - Py_DECREF(ts_obj); - return -1; - } - PyList_SET_ITEM(timestamps_list, 0, ts_obj); +static int +decode_stack(BinaryReader *reader, RemoteDebuggingState *state, + PyObject *collector, DecodedBlock *block, + ReaderBlockStream *stream) +{ + /* An earlier REPEAT still covers this sample */ + if (stream->pending_repeats > 0) { + stream->pending_repeats--; + reader->stats.repeat_samples++; + return 0; + } + uint64_t encoding; + if (block_read_varint(&block->columns[COLUMN_STACK_INSTRUCTIONS], + &block->instruction_offset, &encoding) < 0) { + return -1; + } + if (encoding == STACK_REPEAT) { + return decode_stack_repeat(reader, block, stream); + } - if (emit_sample(state, collector, thread_id, interpreter_id, status, - ts->current_stack, ts->current_stack_depth, reader, - timestamps_list) < 0) { - Py_DECREF(timestamps_list); - return -1; - } - Py_DECREF(timestamps_list); - if (replayed == PY_SSIZE_T_MAX) { - PyErr_SetString(PyExc_OverflowError, - "Sample count exceeds Py_ssize_t maximum"); - return -1; - } - replayed++; - reader->stats.total_samples++; + /* The pending batch was collected with the stack being replaced. */ + if (flush_stream_batch(reader, state, collector, stream) < 0) { + return -1; + } + int result; + switch (encoding) { + case STACK_REPLACE_SUFFIX: + result = decode_stack_replace_suffix(reader, block, stream); + break; + case STACK_REUSE: + result = decode_stack_reuse(reader, block, stream); + break; + default: + return block_error("unknown stack instruction"); + } + if (result < 0) { + return -1; + } + stream->has_stack = 1; + reader->stats.stack_reconstructions++; + return 0; +} + +static int +replay_sample(BinaryReader *reader, RemoteDebuggingState *state, + PyObject *collector, DecodedBlock *block, uint32_t sample) +{ + ReaderBlockStream *stream = + &reader->block_streams[block->stream_ids[sample]]; + uint8_t status = block->statuses[sample]; + if (decode_stack(reader, state, collector, block, stream) < 0) { + return -1; + } + + /* Start a new batch when the status changes or the batch is full */ + if (stream->batch == NULL || status != stream->batch_status || + PyList_GET_SIZE(stream->batch) >= MAX_RLE_BATCH_SAMPLES) { + if (flush_stream_batch(reader, state, collector, stream) < 0) { + return -1; + } + stream->batch = PyList_New(0); + if (stream->batch == NULL) { + return -1; + } + stream->batch_status = status; + } + PyObject *timestamp = + PyLong_FromUnsignedLongLong(block->timestamps[sample]); + if (timestamp == NULL) { + return -1; + } + int result = PyList_Append(stream->batch, timestamp); + Py_DECREF(timestamp); + return result; +} + +static int +validate_block_consumed(BinaryReader *reader, const DecodedBlock *block) +{ + if (block->instruction_offset != + block->columns[COLUMN_STACK_INSTRUCTIONS].size) { + return block_error("stack instruction column has trailing data"); + } + if (block->frame_offset != block->frame_count) { + return block_error("frame ID column has unused values"); + } + for (uint32_t i = 0; i < block->stream_count; i++) { + if (!reader->block_streams[i].has_stack) { + return block_error("stream table has an unused entry"); + } + if (reader->block_streams[i].pending_repeats != 0) { + return block_error("repeat count extends past the block"); + } + } + return 0; +} + +static int +replay_block(BinaryReader *reader, RemoteDebuggingState *state, + PyObject *collector, PyObject *progress_callback, + DecodedBlock *block, Py_ssize_t *replayed) +{ + if ((uint64_t)block->header.sample_count > + reader->sample_count - (uint64_t)*replayed) { + return block_error("more samples than the file header declares"); + } + int result = 0; + for (uint32_t sample = 0; sample < block->header.sample_count; sample++) { + if (replay_sample(reader, state, collector, block, sample) < 0) { + result = -1; break; } + (*replayed)++; + reader->stats.total_samples++; + if (*replayed % PROGRESS_CALLBACK_INTERVAL == 0 && + invoke_progress_callback(progress_callback, *replayed, + reader->sample_count) < 0) { + result = -1; + break; + } + } + if (result == 0) { + result = validate_block_consumed(reader, block); + } - default: - PyErr_Format(PyExc_ValueError, "Unknown stack encoding: %u", encoding); - return -1; + /* Send the remaining batches, or drop them after an error. */ + for (uint32_t i = 0; i < block->stream_count; i++) { + ReaderBlockStream *stream = &reader->block_streams[i]; + if (result == 0 && + flush_stream_batch(reader, state, collector, stream) < 0) { + result = -1; } + Py_CLEAR(stream->batch); + } - /* Progress callback */ - if (replayed % PROGRESS_CALLBACK_INTERVAL == 0) { - if (invoke_progress_callback(progress_callback, replayed, reader->sample_count) < 0) { - return -1; - } + return result; +} + +static int +replay_blocks(BinaryReader *reader, RemoteDebuggingState *state, + PyObject *collector, PyObject *progress_callback, + Py_ssize_t *replayed) +{ + ByteBuffer section = {reader->sample_data, reader->sample_data_size, 0}; + size_t offset = 0; + while (offset < section.size) { + DecodedBlock block; + if (decode_block(reader, §ion, &offset, &block) < 0 || + replay_block(reader, state, collector, progress_callback, + &block, replayed) < 0) { + return -1; } } + return 0; +} + +Py_ssize_t +binary_reader_replay(BinaryReader *reader, PyObject *collector, PyObject *progress_callback) +{ + if (reader->replaying) { + PyErr_SetString(PyExc_RuntimeError, "Reader is already replaying"); + return -1; + } + PyObject *collect_method; + int has_collect = PyObject_GetOptionalAttrString(collector, "collect", &collect_method); + if (has_collect < 0) { + return -1; + } + if (has_collect == 0) { + PyErr_SetString(PyExc_TypeError, "Collector must have a collect() method"); + return -1; + } + Py_DECREF(collect_method); + + /* Get module state for struct sequence types */ + PyObject *module = PyImport_ImportModule("_remote_debugging"); + if (!module) { + return -1; + } + RemoteDebuggingState *state = RemoteDebugging_GetState(module); + Py_DECREF(module); + + if (!state) { + PyErr_SetString(PyExc_RuntimeError, "Failed to get module state"); + return -1; + } + if (reader->sample_count > (uint64_t)PY_SSIZE_T_MAX) { + PyErr_SetString(PyExc_OverflowError, + "Sample count exceeds Py_ssize_t maximum"); + return -1; + } + + /* Initial progress callback at 0% */ + if (invoke_progress_callback(progress_callback, 0, reader->sample_count) < 0) { + return -1; + } + + memset(&reader->stats, 0, sizeof(reader->stats)); + /* The collector and progress callback must not replay this reader again + * while its block state is in use */ + Py_ssize_t replayed = 0; + reader->replaying = 1; + int result = replay_blocks(reader, state, collector, progress_callback, + &replayed); + reader->replaying = 0; + if (result < 0) { + return -1; + } if ((uint64_t)replayed != reader->sample_count) { PyErr_Format(PyExc_ValueError, @@ -1453,8 +1736,9 @@ binary_writer_get_stats(BinaryWriter *writer) /* Calculate derived stats */ uint64_t total_records = s->repeat_records + s->full_records + - s->suffix_records + s->pop_push_records; + s->suffix_records + s->reuse_records; uint64_t total_samples = writer->total_samples; + uint64_t bytes_written = s->bytes_written + binary_writer_buffered_size(writer); uint64_t potential_frames = s->total_frames_written + s->frames_saved; double compression_ratio = (potential_frames > 0) ? (double)s->frames_saved / potential_frames * 100.0 : 0.0; @@ -1465,12 +1749,12 @@ binary_writer_get_stats(BinaryWriter *writer) "repeat_samples", s->repeat_samples, "full_records", s->full_records, "suffix_records", s->suffix_records, - "pop_push_records", s->pop_push_records, + "reuse_records", s->reuse_records, "total_records", total_records, "total_samples", total_samples, "total_frames_written", s->total_frames_written, "frames_saved", s->frames_saved, - "bytes_written", s->bytes_written, + "bytes_written", bytes_written, "frame_compression_pct", compression_ratio ); } @@ -1481,7 +1765,7 @@ binary_reader_get_stats(BinaryReader *reader) BinaryReaderStats *s = &reader->stats; uint64_t total_records = s->repeat_records + s->full_records + - s->suffix_records + s->pop_push_records; + s->suffix_records + s->reuse_records; return Py_BuildValue( "{s:K, s:K, s:K, s:K, s:K, s:K, s:K, s:K}", @@ -1489,7 +1773,7 @@ binary_reader_get_stats(BinaryReader *reader) "repeat_samples", s->repeat_samples, "full_records", s->full_records, "suffix_records", s->suffix_records, - "pop_push_records", s->pop_push_records, + "reuse_records", s->reuse_records, "total_records", total_records, "total_samples", s->total_samples, "stack_reconstructions", s->stack_reconstructions @@ -1524,7 +1808,23 @@ binary_reader_close(BinaryReader *reader) } #endif - PyMem_Free(reader->decompressed_data); +#ifdef HAVE_ZSTD + ZSTD_freeDCtx(reader->dctx); +#endif + buffer_clear(&reader->block_data); + buffer_clear(&reader->block_stream_ids); + buffer_clear(&reader->block_timestamps); + buffer_clear(&reader->block_statuses); + buffer_clear(&reader->block_frames); + PyMem_Free(reader->block_streams); + PyMem_Free(reader->block_stacks); + + if (reader->frame_info_cache != NULL) { + for (size_t i = 0; i < reader->frame_info_cache_size; i++) { + Py_XDECREF(reader->frame_info_cache[i].frame_info); + } + PyMem_Free(reader->frame_info_cache); + } if (reader->strings) { for (uint32_t i = 0; i < reader->strings_count; i++) { @@ -1535,12 +1835,7 @@ binary_reader_close(BinaryReader *reader) PyMem_Free(reader->frames); - if (reader->thread_states) { - for (size_t i = 0; i < reader->thread_state_count; i++) { - PyMem_Free(reader->thread_states[i].current_stack); - } - PyMem_Free(reader->thread_states); - } + PyMem_Free(reader->thread_states); PyMem_Free(reader); } diff --git a/Modules/_remote_debugging/binary_io_writer.c b/Modules/_remote_debugging/binary_io_writer.c index 1794017053da503..6db72080c27b89a 100644 --- a/Modules/_remote_debugging/binary_io_writer.c +++ b/Modules/_remote_debugging/binary_io_writer.c @@ -2,7 +2,7 @@ * Python Remote Debugging Module - Binary Writer Implementation * * High-performance binary file writer for profiling data with optional zstd - * streaming compression. + * compression. ******************************************************************************/ #ifndef Py_BUILD_CORE_MODULE @@ -11,6 +11,7 @@ #include "binary_io.h" #include "_remote_debugging.h" +#include "pycore_bitutils.h" // _Py_bit_length(), _Py_popcount32() #include "pycore_opcode_utils.h" // MAX_REAL_OPCODE #include #include @@ -27,16 +28,69 @@ * CONSTANTS FOR BINARY FORMAT SIZES * ============================================================================ */ -/* Sample header sizes */ -#define SAMPLE_HEADER_MAX_SIZE 26 /* fixed + max_varint(10) + status(1) + margin */ #define MAX_VARINT_SIZE 10 /* Maximum bytes for a varint64 */ #define MAX_VARINT_SIZE_U32 5 /* Maximum bytes for a varint32 */ -/* Frame buffer: depth varint (max 2 bytes for 256) + 256 frames * 5 bytes/varint + margin */ -#define MAX_FRAME_BUFFER_SIZE ((MAX_STACK_DEPTH * MAX_VARINT_SIZE_U32) + MAX_VARINT_SIZE_U32 + 16) -/* RLE pending buffer (per thread): one entry is a u64 delta varint + status byte */ -#define MAX_RLE_BUF_SIZE (16 * 1024) -#define MAX_RLE_ENTRY_SIZE (MAX_VARINT_SIZE + sizeof(uint8_t)) +#define MAX_BLOCK_HEADER_SIZE \ + (BLOCK_MAGIC_SIZE + MAX_VARINT_SIZE_U32 + MAX_VARINT_SIZE + \ + 3 * sizeof(uint8_t) + \ + COLUMN_COUNT * (2 * MAX_VARINT_SIZE_U32 + sizeof(uint8_t))) + +#define ZSTD_MIN_SIZE 32 +#define STACK_REUSE_MIN_FRAMES 3 + +#define MAX_SAMPLE_SIZE \ + (STREAM_ENTRY_SIZE + 5 * MAX_VARINT_SIZE + sizeof(uint8_t) + \ + MAX_STACK_DEPTH * sizeof(uint32_t) + sizeof(Repeat)) +#define MAX_BITMAP_PADDING (CHAR_BIT - 1) +#define BLOCK_FLUSH_SIZE \ + (MAX_BLOCK_SIZE - MAX_SAMPLE_SIZE - MAX_BITMAP_PADDING) + +static int +buffer_append(ByteBuffer *buffer, const void *data, size_t size) +{ + if (buffer->size > MAX_BLOCK_SIZE || + size > MAX_BLOCK_SIZE - buffer->size) { + PyErr_SetString(PyExc_RuntimeError, + "sample block buffer exceeds the writer safety limit"); + return -1; + } + if (buffer_reserve(buffer, buffer->size + size) < 0) { + return -1; + } + if (size != 0) { + memcpy(buffer->data + buffer->size, data, size); + } + buffer->size += size; + return 0; +} + +static int +buffer_append_varint(ByteBuffer *buffer, uint64_t value) +{ + uint8_t bytes[MAX_VARINT_SIZE]; + size_t size = encode_varint_u64(bytes, value); + return buffer_append(buffer, bytes, size); +} + +static int +column_write(SampleBlock *block, unsigned column, const void *data, + size_t size) +{ + if (buffer_append(&block->columns[column], data, size) < 0) { + return -1; + } + block->size += size; + return 0; +} + +static int +column_write_varint(SampleBlock *block, unsigned column, uint64_t value) +{ + uint8_t bytes[MAX_VARINT_SIZE]; + size_t size = encode_varint_u64(bytes, value); + return column_write(block, column, bytes, size); +} /* Helper macro: convert PyLong to int32, using default_val if conversion fails */ #define PYLONG_TO_INT32_OR_DEFAULT(obj, var, default_val) \ @@ -131,19 +185,6 @@ fwrite_checked_allow_threads(const void *data, size_t size, FILE *fp) return 0; } -/* Forward declaration for writer_write_bytes */ -static inline int writer_write_bytes(BinaryWriter *writer, const void *data, size_t size); - -/* Encode and write a varint u32 - returns 0 on success, -1 on error */ -static inline int -writer_write_varint_u32(BinaryWriter *writer, uint32_t value) -{ - uint8_t buf[MAX_VARINT_SIZE]; - size_t len = encode_varint_u32(buf, value); - return writer_write_bytes(writer, buf, len); -} - - /* ============================================================================ * UTILITY FUNCTIONS * ============================================================================ */ @@ -182,9 +223,9 @@ writer_init_zstd(BinaryWriter *writer) return -1; } - /* Compression level 5: better ratio for repetitive profiling data */ + /* Compression level 3: good ratio without slowing down each block flush */ size_t result = ZSTD_CCtx_setParameter(writer->zstd.cctx, - ZSTD_c_compressionLevel, 5); + ZSTD_c_compressionLevel, 3); if (ZSTD_isError(result)) { PyErr_Format(PyExc_RuntimeError, "Failed to set zstd compression level: %s", ZSTD_getErrorName(result)); @@ -193,16 +234,6 @@ writer_init_zstd(BinaryWriter *writer) return -1; } - /* Use large buffer (512KB) for fewer I/O syscalls */ - writer->zstd.compressed_buffer = PyMem_Malloc(COMPRESSED_BUFFER_SIZE); - if (!writer->zstd.compressed_buffer) { - ZSTD_freeCCtx(writer->zstd.cctx); - writer->zstd.cctx = NULL; - PyErr_NoMemory(); - return -1; - } - writer->zstd.compressed_buffer_size = COMPRESSED_BUFFER_SIZE; - return 0; #else PyErr_SetString(PyExc_RuntimeError, @@ -211,79 +242,6 @@ writer_init_zstd(BinaryWriter *writer) #endif } -static int -writer_flush_buffer(BinaryWriter *writer) -{ - if (writer->buffer_pos == 0) { - return 0; - } - -#ifdef HAVE_ZSTD - if (writer->compression_type == COMPRESSION_ZSTD) { - ZSTD_inBuffer input = { writer->write_buffer, writer->buffer_pos, 0 }; - - while (input.pos < input.size) { - ZSTD_outBuffer output = { - writer->zstd.compressed_buffer, - writer->zstd.compressed_buffer_size, - 0 - }; - - size_t result = ZSTD_compressStream2( - writer->zstd.cctx, &output, &input, ZSTD_e_continue - ); - - if (ZSTD_isError(result)) { - PyErr_Format(PyExc_IOError, "zstd compression error: %s", - ZSTD_getErrorName(result)); - return -1; - } - - _Py_MSAN_UNPOISON(writer->zstd.compressed_buffer, output.pos); - if (output.pos > 0) { - if (fwrite_checked_allow_threads(writer->zstd.compressed_buffer, output.pos, writer->fp) < 0) { - return -1; - } - } - } - } else -#endif - { - if (fwrite_checked_allow_threads(writer->write_buffer, writer->buffer_pos, writer->fp) < 0) { - return -1; - } - } - - writer->buffer_pos = 0; - return 0; -} - -static inline int -writer_write_bytes(BinaryWriter *writer, const void *data, size_t size) -{ - const uint8_t *src = (const uint8_t *)data; - size_t original_size = size; - - while (size > 0) { - size_t space = writer->buffer_size - writer->buffer_pos; - size_t to_copy = (size < space) ? size : space; - - memcpy(writer->write_buffer + writer->buffer_pos, src, to_copy); - writer->buffer_pos += to_copy; - src += to_copy; - size -= to_copy; - - if (writer->buffer_pos == writer->buffer_size) { - if (writer_flush_buffer(writer) < 0) { - return -1; - } - } - } - - writer->stats.bytes_written += original_size; - return 0; -} - /* ============================================================================ * HASH TABLE SUPPORT FUNCTIONS (using _Py_hashtable) * ============================================================================ */ @@ -523,7 +481,8 @@ writer_get_or_create_thread_entry(BinaryWriter *writer, uint64_t thread_id, memset(entry, 0, sizeof(ThreadEntry)); entry->thread_id = thread_id; entry->interpreter_id = interpreter_id; - entry->prev_timestamp = writer->start_time_us; + entry->block_stream_index = UINT32_MAX; + entry->current_repeat = UINT32_MAX; entry->prev_stack_capacity = MAX_STACK_DEPTH; entry->prev_stack = PyMem_Calloc(entry->prev_stack_capacity, sizeof(uint32_t)); @@ -539,216 +498,614 @@ writer_get_or_create_thread_entry(BinaryWriter *writer, uint64_t thread_id, return entry; } -/* Compare two stacks and return the encoding type and parameters. - * Sets: - * - shared_count: number of frames matching from bottom of stack - * - pop_count: frames to remove from prev stack - * - push_count: new frames to add - * - * Returns the best encoding type to use. */ static int -compare_stacks(const uint32_t *prev_stack, size_t prev_depth, - const uint32_t *curr_stack, size_t curr_depth, - size_t *shared_count, size_t *pop_count, size_t *push_count) +encode_column(BinaryWriter *writer, ByteBuffer *decoded, + ColumnDescriptor *descriptor) { - if (prev_depth == curr_depth) { - int identical = 1; - for (size_t i = 0; i < prev_depth; i++) { - if (prev_stack[i] != curr_stack[i]) { - identical = 0; - break; - } + SampleBlock *block = writer->block; + const uint8_t *data = decoded->data; + size_t size = decoded->size; + uint8_t encoding = COMPRESSION_NONE; + if (size > 1 && memcmp(data, data + 1, size - 1) == 0) { + encoding = COLUMN_CONSTANT; + size = 1; + } +#ifdef HAVE_ZSTD + if (encoding == COMPRESSION_NONE && size >= ZSTD_MIN_SIZE && + writer->compression_type == COMPRESSION_ZSTD) { + size_t compress_bound = ZSTD_compressBound(size); + if (buffer_reserve(&block->compressed, compress_bound) < 0) { + return -1; + } + size_t compressed_size = ZSTD_compress2( + writer->zstd.cctx, block->compressed.data, compress_bound, + data, size); + if (ZSTD_isError(compressed_size)) { + PyErr_Format(PyExc_IOError, "zstd compression error: %s", + ZSTD_getErrorName(compressed_size)); + return -1; } - if (identical) { - *shared_count = prev_depth; - *pop_count = 0; - *push_count = 0; - return STACK_REPEAT; + _Py_MSAN_UNPOISON(block->compressed.data, compressed_size); + uint8_t stored_size[MAX_VARINT_SIZE]; + if (compressed_size + encode_varint_u64(stored_size, compressed_size) + < size) { + data = block->compressed.data; + size = compressed_size; + encoding = COMPRESSION_ZSTD; } } +#endif + descriptor->decoded_size = (uint32_t)decoded->size; + descriptor->stored_size = (uint32_t)size; + descriptor->encoding = encoding; + if (buffer_reserve(&block->output, block->output.size + size) < 0) { + return -1; + } + if (size != 0) { + memcpy(block->output.data + block->output.size, data, size); + } + block->output.size += size; + return 0; +} - /* Find longest common suffix (frames at the bottom/outer part of stack). - * Stacks are stored innermost-first, so suffix is at the end. */ - size_t suffix_len = 0; - size_t min_depth = (prev_depth < curr_depth) ? prev_depth : curr_depth; - - for (size_t i = 0; i < min_depth; i++) { - size_t prev_idx = prev_depth - 1 - i; - size_t curr_idx = curr_depth - 1 - i; - if (prev_stack[prev_idx] == curr_stack[curr_idx]) { - suffix_len++; - } else { - break; +static int +transform_statuses(SampleBlock *block, uint8_t bitmap_flags, + ByteBuffer **decoded) +{ + size_t plane_size = status_bitmap_size(block->sample_count); + unsigned plane_count = (unsigned)_Py_popcount32(bitmap_flags); + if (buffer_reserve(&block->transformed, + plane_size * plane_count) < 0) { + return -1; + } + const uint8_t *statuses = + block->columns[COLUMN_STATUS_BITMAPS].data; + block->transformed.size = 0; + for (unsigned bit = 0; bit < CHAR_BIT; bit++) { + if (!(bitmap_flags & (1U << bit))) { + continue; } + uint8_t *plane = block->transformed.data + block->transformed.size; + memset(plane, 0, plane_size); + for (uint32_t sample = 0; sample < block->sample_count; sample++) { + plane[sample / CHAR_BIT] |= ((statuses[sample] >> bit) & 1U) + << (sample % CHAR_BIT); + } + block->transformed.size += plane_size; } + *decoded = &block->transformed; + return 0; +} - *shared_count = suffix_len; - *pop_count = prev_depth - suffix_len; - *push_count = curr_depth - suffix_len; - - /* Choose best encoding based on byte cost */ - /* STACK_FULL: 1 (type) + 1-2 (depth) + sum(frame varints) */ - /* STACK_SUFFIX: 1 (type) + 1-2 (shared) + 1-2 (new_count) + sum(new frame varints) */ - /* STACK_POP_PUSH: 1 (type) + 1-2 (pop) + 1-2 (push) + sum(new frame varints) */ +static int +transform_frame_ids(SampleBlock *block, uint32_t frame_count, + unsigned width, ByteBuffer **decoded) +{ + ByteBuffer *frames = &block->columns[COLUMN_FRAME_IDS]; + if (buffer_reserve(&block->transformed, + (size_t)frame_count * width) < 0) { + return -1; + } + const uint32_t *frame_ids = (const uint32_t *)frames->data; + for (unsigned byte = 0; frame_count && byte < width; byte++) { + uint8_t *plane = + block->transformed.data + (size_t)byte * frame_count; + for (uint32_t frame = 0; frame < frame_count; frame++) { + plane[frame] = + (uint8_t)(frame_ids[frame] >> (byte * CHAR_BIT)); + } + } + block->transformed.size = (size_t)frame_count * width; + *decoded = &block->transformed; + return 0; +} - /* If no common suffix, use full stack */ - if (suffix_len == 0) { - return STACK_FULL; +static int +transform_stack_instructions(SampleBlock *block, ByteBuffer **decoded) +{ + size_t repeat_count = block->repeats.size / sizeof(Repeat); + if (repeat_count == 0) { + return 0; } + ByteBuffer *instructions = + &block->columns[COLUMN_STACK_INSTRUCTIONS]; + const Repeat *repeats = (const Repeat *)block->repeats.data; + size_t copied_offset = 0; + block->transformed.size = 0; + for (size_t i = 0; i < repeat_count; i++) { + if (buffer_append(&block->transformed, + instructions->data + copied_offset, + repeats[i].offset - copied_offset) < 0 || + buffer_append_varint( + &block->transformed, STACK_REPEAT) < 0 || + buffer_append_varint( + &block->transformed, repeats[i].count) < 0) { + return -1; + } + copied_offset = repeats[i].offset; + } + if (buffer_append(&block->transformed, + instructions->data + copied_offset, + instructions->size - copied_offset) < 0) { + return -1; + } + *decoded = &block->transformed; + return 0; +} - /* If only adding frames (suffix == prev_depth), use SUFFIX */ - if (*pop_count == 0 && *push_count > 0) { - return STACK_SUFFIX; +static void +block_reset(BinaryWriter *writer) +{ + SampleBlock *block = writer->block; + for (unsigned column = 0; column < COLUMN_COUNT; column++) { + block->columns[column].size = 0; + } + block->sample_count = 0; + block->stream_count = 0; + block->repeats.size = 0; + block->max_frame = 0; + block->size = 0; + memset(block->cache, 0, sizeof(block->cache)); + for (size_t thread = 0; thread < writer->thread_count; thread++) { + writer->thread_entries[thread].block_stream_index = UINT32_MAX; + writer->thread_entries[thread].current_repeat = UINT32_MAX; } +} - /* If popping and/or pushing, use POP_PUSH if it saves bytes */ - /* Heuristic: POP_PUSH is better when we're modifying top frames */ - if (*pop_count > 0 || *push_count > 0) { - /* Use full stack if sharing less than half the frames */ - if (suffix_len < curr_depth / 2) { - return STACK_FULL; - } - return STACK_POP_PUSH; +static unsigned +block_frame_width(const SampleBlock *block) +{ + if (block->max_frame == 0) { + return 1; } + return (unsigned)(_Py_bit_length(block->max_frame) + CHAR_BIT - 1) + / CHAR_BIT; +} - return STACK_FULL; +static void +block_status_flags(const SampleBlock *block, uint8_t *common_flags, + uint8_t *bitmap_flags) +{ + uint8_t common_status = UINT8_MAX; + uint8_t any_status = 0; + const uint8_t *statuses = block->columns[COLUMN_STATUS_BITMAPS].data; + for (uint32_t sample = 0; sample < block->sample_count; sample++) { + common_status &= statuses[sample]; + any_status |= statuses[sample]; + } + *common_flags = common_status; + *bitmap_flags = any_status ^ common_status; } -/* Write common sample header: thread_id(8) + interpreter_id(4) + encoding(1). - * Returns 0 on success, -1 on failure. */ -static inline int -write_sample_header(BinaryWriter *writer, ThreadEntry *entry, uint8_t encoding) +static size_t +write_block_header(const BlockHeader *header, uint8_t *bytes) { - uint8_t header[SAMPLE_HEADER_FIXED_SIZE]; - memcpy(header + SMP_OFF_THREAD_ID, &entry->thread_id, SMP_SIZE_THREAD_ID); - memcpy(header + SMP_OFF_INTERPRETER_ID, &entry->interpreter_id, SMP_SIZE_INTERPRETER_ID); - header[SMP_OFF_ENCODING] = encoding; - return writer_write_bytes(writer, header, SAMPLE_HEADER_FIXED_SIZE); + /* Magic "TBLK", sample count and base timestamp (varints), then frame + * ID width and the two status flag masks (1 byte each) */ + size_t size = BLOCK_MAGIC_SIZE; + memcpy(bytes, BLOCK_MAGIC, BLOCK_MAGIC_SIZE); + size += encode_varint_u32(bytes + size, header->sample_count); + size += encode_varint_u64(bytes + size, header->base_timestamp); + bytes[size++] = header->frame_width; + bytes[size++] = header->common_status_flags; + bytes[size++] = header->status_bitmap_flags; + + /* Each column: decoded size (varint), encoding (1 byte), and the stored + * size (varint) for zstd only */ + for (unsigned column = 0; column < COLUMN_COUNT; column++) { + const ColumnDescriptor *descriptor = &header->columns[column]; + size += encode_varint_u32(bytes + size, descriptor->decoded_size); + bytes[size++] = descriptor->encoding; + if (descriptor->encoding == COMPRESSION_ZSTD) { + size += encode_varint_u32(bytes + size, descriptor->stored_size); + } + } + return size; } -/* Flush pending RLE samples for a thread. - * Writes the RLE record to the output buffer. - * Returns 0 on success, -1 on failure. */ static int -flush_pending_rle(BinaryWriter *writer, ThreadEntry *entry) +block_flush(BinaryWriter *writer) { - if (entry->pending_rle_samples == 0) { + SampleBlock *block = writer->block; + if (block->sample_count == 0) { return 0; } - - /* Write RLE record: - * [thread_id: 8] [interpreter_id: 4] [STACK_REPEAT: 1] [count: varint] - * [timestamp_delta_1: varint] [status_1: 1] ... [timestamp_delta_N: varint] [status_N: 1] - */ - - if (write_sample_header(writer, entry, STACK_REPEAT) < 0) { + uint32_t frame_count = (uint32_t)( + block->columns[COLUMN_FRAME_IDS].size / sizeof(uint32_t)); + BlockHeader header = {0}; + header.sample_count = block->sample_count; + header.base_timestamp = block->base_timestamp; + header.frame_width = (uint8_t)block_frame_width(block); + block_status_flags(block, &header.common_status_flags, + &header.status_bitmap_flags); + + /* Convert each column to its file form, then encode it */ + /* Leave room for the header, which is written once its size is known */ + if (buffer_reserve(&block->output, MAX_BLOCK_HEADER_SIZE) < 0) { return -1; } - - if (writer_write_varint_u32(writer, (uint32_t)entry->pending_rle_samples) < 0) { + block->output.size = MAX_BLOCK_HEADER_SIZE; + size_t decoded_payload_size = 0; + for (unsigned column = 0; column < COLUMN_COUNT; column++) { + ByteBuffer *decoded = &block->columns[column]; + int result = 0; + switch (column) { + case COLUMN_STATUS_BITMAPS: + result = transform_statuses(block, header.status_bitmap_flags, + &decoded); + break; + case COLUMN_STACK_INSTRUCTIONS: + result = transform_stack_instructions(block, &decoded); + break; + case COLUMN_FRAME_IDS: + result = transform_frame_ids( + block, frame_count, header.frame_width, &decoded); + break; + } + if (result < 0) { + return -1; + } + if (decoded->size > + MAX_BLOCK_SIZE - decoded_payload_size) { + PyErr_SetString(PyExc_RuntimeError, + "sample block exceeds the decoded size limit"); + return -1; + } + decoded_payload_size += decoded->size; + if (encode_column(writer, decoded, &header.columns[column]) < 0) { + return -1; + } + } + /* Put the header right before the columns and write the block */ + uint8_t header_bytes[MAX_BLOCK_HEADER_SIZE]; + size_t header_size = write_block_header(&header, header_bytes); + size_t unused = MAX_BLOCK_HEADER_SIZE - header_size; + uint8_t *header_start = block->output.data + unused; + memcpy(header_start, header_bytes, header_size); + if (fwrite_checked_allow_threads(header_start, + block->output.size - unused, + writer->fp) < 0) { return -1; } + writer->stats.bytes_written += decoded_payload_size; + block_reset(writer); + return 0; +} - if (writer_write_bytes(writer, entry->pending_rle, entry->pending_rle_bytes) < 0) { - return -1; +static Py_uhash_t +stack_hash(const uint32_t *stack, size_t depth) +{ + return (Py_uhash_t)Py_HashBuffer( + stack, (Py_ssize_t)(depth * sizeof(*stack))); +} + +static size_t +common_outer_frames(const ThreadEntry *entry, const uint32_t *stack, + size_t depth) +{ + size_t shared = 0; + while (shared < depth && shared < entry->prev_stack_depth && + stack[depth - 1 - shared] == + entry->prev_stack[entry->prev_stack_depth - 1 - shared]) { + shared++; + } + return shared; +} + +static int +cached_stack_matches(const StackCacheEntry *cached, + const ByteBuffer *frames, Py_uhash_t hash, + const uint32_t *stack, size_t depth) +{ + if (cached->hash != hash || cached->depth != depth) { + return 0; } + const uint32_t *frame_ids = (const uint32_t *)frames->data; + for (size_t frame = 0; frame < depth; frame++) { + if (stack[frame] != + frame_ids[cached->offset + depth - 1 - frame]) { + return 0; + } + } + return 1; +} - writer->stats.repeat_records++; - writer->stats.repeat_samples += entry->pending_rle_samples; - /* Each RLE sample saves writing the entire stack */ - writer->stats.frames_saved += entry->pending_rle_samples * entry->prev_stack_depth; +static StackCacheEntry * +stack_cache_lookup(StackCacheEntry *bucket, const ByteBuffer *frames, + Py_uhash_t hash, const uint32_t *stack, size_t depth) +{ + for (size_t i = 0; i < STACK_CACHE_BUCKET_SIZE; i++) { + if (cached_stack_matches(&bucket[i], frames, hash, stack, depth)) { + return &bucket[i]; + } + } + return NULL; +} - entry->pending_rle_bytes = 0; - entry->pending_rle_samples = 0; +static void +stack_cache_insert(StackCacheEntry *bucket, StackCacheEntry entry) +{ + memmove(&bucket[1], &bucket[0], + (STACK_CACHE_BUCKET_SIZE - 1) * sizeof(*bucket)); + bucket[0] = entry; +} +static int +write_stack_repeat(BinaryWriter *writer, ThreadEntry *entry, size_t depth) +{ + SampleBlock *block = writer->block; + if (entry->current_repeat == UINT32_MAX) { + Repeat repeat = { + (uint32_t)block->columns[COLUMN_STACK_INSTRUCTIONS].size, 1}; + if (buffer_append(&block->repeats, + &repeat, sizeof(repeat)) < 0) { + return -1; + } + block->size += sizeof(repeat); + entry->current_repeat = + (uint32_t)(block->repeats.size / sizeof(repeat) - 1); + writer->stats.repeat_records++; + } + else { + Repeat *repeats = (Repeat *)block->repeats.data; + repeats[entry->current_repeat].count++; + } + writer->stats.repeat_samples++; + writer->stats.frames_saved += depth; return 0; } -/* Write a single sample with the specified encoding. - * Returns 0 on success, -1 on failure. */ static int -write_sample_with_encoding(BinaryWriter *writer, ThreadEntry *entry, - uint64_t timestamp_delta, uint8_t status, - int encoding_type, - const uint32_t *frame_indices, size_t stack_depth, - size_t shared_count, size_t pop_count, size_t push_count) +write_stack_reuse(BinaryWriter *writer, + const StackCacheEntry *cached, size_t depth) { - /* Header: thread_id(8) + interpreter_id(4) + encoding(1) + delta(varint) + status(1) */ - uint8_t header_buf[SAMPLE_HEADER_MAX_SIZE]; - memcpy(header_buf + SMP_OFF_THREAD_ID, &entry->thread_id, SMP_SIZE_THREAD_ID); - memcpy(header_buf + SMP_OFF_INTERPRETER_ID, &entry->interpreter_id, SMP_SIZE_INTERPRETER_ID); - header_buf[SMP_OFF_ENCODING] = (uint8_t)encoding_type; - size_t varint_len = encode_varint_u64( - header_buf + SAMPLE_HEADER_FIXED_SIZE, - timestamp_delta); - header_buf[SAMPLE_HEADER_FIXED_SIZE + varint_len] = status; - - if (writer_write_bytes(writer, header_buf, - SAMPLE_HEADER_FIXED_SIZE + varint_len + 1) < 0) { + SampleBlock *block = writer->block; + if (column_write_varint(block, COLUMN_STACK_INSTRUCTIONS, + STACK_REUSE) < 0 || + column_write_varint(block, COLUMN_STACK_INSTRUCTIONS, + cached->offset) < 0 || + column_write_varint(block, COLUMN_STACK_INSTRUCTIONS, depth) < 0) { return -1; } + writer->stats.reuse_records++; + writer->stats.frames_saved += depth; + return 0; +} - uint8_t frame_buf[MAX_FRAME_BUFFER_SIZE]; - size_t frame_buf_pos = 0; - size_t frames_written = 0; +static int +write_stack_replace_suffix(BinaryWriter *writer, StackCacheEntry *bucket, + Py_uhash_t hash, const uint32_t *stack, + size_t depth, size_t shared) +{ + SampleBlock *block = writer->block; + ByteBuffer *frames = &block->columns[COLUMN_FRAME_IDS]; + size_t new_frames = depth - shared; + if (column_write_varint(block, COLUMN_STACK_INSTRUCTIONS, + STACK_REPLACE_SUFFIX) < 0 || + column_write_varint(block, COLUMN_STACK_INSTRUCTIONS, shared) < 0 || + column_write_varint(block, COLUMN_STACK_INSTRUCTIONS, + new_frames) < 0) { + return -1; + } - switch (encoding_type) { - case STACK_FULL: - /* [depth: varint] [frame_idx: varint]... */ - frame_buf_pos += encode_varint_u32(frame_buf, (uint32_t)stack_depth); - for (size_t i = 0; i < stack_depth; i++) { - frame_buf_pos += encode_varint_u32(frame_buf + frame_buf_pos, frame_indices[i]); + /* Append the new inner frames, outermost first */ + size_t stack_offset = frames->size / sizeof(uint32_t); + if (new_frames != 0) { + if (buffer_reserve( + frames, frames->size + new_frames * sizeof(uint32_t)) < 0) { + return -1; } - frames_written = stack_depth; - writer->stats.full_records++; - break; - - case STACK_SUFFIX: - /* [shared_count: varint] [new_count: varint] [new_frame_idx: varint]... */ - frame_buf_pos += encode_varint_u32(frame_buf, (uint32_t)shared_count); - frame_buf_pos += encode_varint_u32(frame_buf + frame_buf_pos, (uint32_t)push_count); - /* New frames are at the top (beginning) of current stack */ - for (size_t i = 0; i < push_count; i++) { - frame_buf_pos += encode_varint_u32(frame_buf + frame_buf_pos, frame_indices[i]); + uint32_t *new_frame_ids = (uint32_t *)(frames->data + frames->size); + for (size_t i = 0; i < new_frames; i++) { + uint32_t frame = stack[new_frames - 1 - i]; + new_frame_ids[i] = frame; + if (frame > block->max_frame) { + block->max_frame = frame; + } + } + frames->size += new_frames * sizeof(uint32_t); + block->size += new_frames * sizeof(uint32_t); + } + /* Complete stacks are cached so later samples can REUSE them */ + if (shared == 0) { + if (bucket != NULL) { + stack_cache_insert(bucket, (StackCacheEntry){ + hash, (uint32_t)stack_offset, (uint32_t)depth}); } - frames_written = push_count; + writer->stats.full_records++; + } + else { writer->stats.suffix_records++; - /* Saved writing shared_count frames */ - writer->stats.frames_saved += shared_count; - break; + } + writer->stats.total_frames_written += new_frames; + writer->stats.frames_saved += shared; + return 0; +} - case STACK_POP_PUSH: - /* [pop_count: varint] [push_count: varint] [new_frame_idx: varint]... */ - frame_buf_pos += encode_varint_u32(frame_buf, (uint32_t)pop_count); - frame_buf_pos += encode_varint_u32(frame_buf + frame_buf_pos, (uint32_t)push_count); - /* New frames are at the top (beginning) of current stack */ - for (size_t i = 0; i < push_count; i++) { - frame_buf_pos += encode_varint_u32(frame_buf + frame_buf_pos, frame_indices[i]); - } - frames_written = push_count; - writer->stats.pop_push_records++; - /* Saved writing shared_count frames (stack_depth - push_count if we had written full) */ - writer->stats.frames_saved += shared_count; - break; +static int +choose_stack_encoding(SampleBlock *block, const ThreadEntry *entry, + int new_stream, const uint32_t *stack, size_t depth, + size_t *shared, Py_uhash_t *hash, + StackCacheEntry **bucket, StackCacheEntry **cached) +{ + *shared = common_outer_frames(entry, stack, depth); + *hash = 0; + *bucket = NULL; + *cached = NULL; + if (!new_stream && *shared == depth && + *shared == entry->prev_stack_depth) { + return STACK_REPEAT; + } + if (depth - *shared < STACK_REUSE_MIN_FRAMES) { + return STACK_REPLACE_SUFFIX; + } + *hash = stack_hash(stack, depth); + *bucket = block->cache[*hash % STACK_CACHE_BUCKET_COUNT]; + *cached = stack_cache_lookup(*bucket, &block->columns[COLUMN_FRAME_IDS], + *hash, stack, depth); + if (*cached != NULL) { + return STACK_REUSE; + } + return STACK_REPLACE_SUFFIX; +} +static int +write_stack(BinaryWriter *writer, ThreadEntry *entry, int new_stream, + const uint32_t *stack, size_t depth) +{ + size_t shared; + Py_uhash_t hash; + StackCacheEntry *bucket; + StackCacheEntry *cached; + int encoding = choose_stack_encoding(writer->block, entry, new_stream, + stack, depth, &shared, &hash, + &bucket, &cached); + + int result; + switch (encoding) { + case STACK_REPEAT: + return write_stack_repeat(writer, entry, depth); + case STACK_REUSE: + result = write_stack_reuse(writer, cached, depth); + break; + case STACK_REPLACE_SUFFIX: + result = write_stack_replace_suffix(writer, bucket, hash, stack, + depth, shared); + break; default: PyErr_SetString(PyExc_RuntimeError, "Invalid stack encoding type"); return -1; } + if (result < 0) { + return -1; + } + entry->current_repeat = UINT32_MAX; + memcpy(entry->prev_stack, stack, depth * sizeof(uint32_t)); + entry->prev_stack_depth = depth; + return 0; +} - if (writer_write_bytes(writer, frame_buf, frame_buf_pos) < 0) { +size_t +binary_writer_buffered_size(BinaryWriter *writer) +{ + SampleBlock *block = writer->block; + if (block == NULL || block->sample_count == 0) { + return 0; + } + size_t size = block->columns[COLUMN_STREAM_TABLE].size + + block->columns[COLUMN_SAMPLE_STREAM_IDS].size + + block->columns[COLUMN_TIMESTAMP_RESIDUALS].size + + block->columns[COLUMN_STACK_INSTRUCTIONS].size; + const Repeat *repeats = (const Repeat *)block->repeats.data; + size_t repeat_count = block->repeats.size / sizeof(Repeat); + uint8_t bytes[MAX_VARINT_SIZE]; + for (size_t i = 0; i < repeat_count; i++) { + size += encode_varint_u64(bytes, STACK_REPEAT) + + encode_varint_u64(bytes, repeats[i].count); + } + uint8_t common_flags, bitmap_flags; + block_status_flags(block, &common_flags, &bitmap_flags); + size += (size_t)_Py_popcount32(bitmap_flags) + * status_bitmap_size(block->sample_count); + size += block->columns[COLUMN_FRAME_IDS].size / sizeof(uint32_t) + * block_frame_width(block); + return size; +} + +static int +block_add_stream(BinaryWriter *writer, ThreadEntry *entry) +{ + SampleBlock *block = writer->block; + /* Stream table entry: thread_id (8 bytes) + interpreter_id (4 bytes) */ + uint8_t stream_entry[STREAM_ENTRY_SIZE]; + memcpy(stream_entry + STM_OFF_THREAD_ID, + &entry->thread_id, sizeof(entry->thread_id)); + memcpy(stream_entry + STM_OFF_INTERPRETER_ID, + &entry->interpreter_id, sizeof(entry->interpreter_id)); + if (column_write(block, COLUMN_STREAM_TABLE, + stream_entry, sizeof(stream_entry)) < 0) { return -1; } + entry->block_stream_index = block->stream_count++; + entry->prev_timestamp = block->base_timestamp - writer->sample_interval_us; + entry->prev_stack_depth = 0; + return 0; +} - writer->stats.total_frames_written += frames_written; +static int +block_add_sample(BinaryWriter *writer, ThreadEntry *entry, + uint64_t timestamp, uint8_t status, + const uint32_t *stack, size_t depth) +{ + static_assert(BLOCK_FLUSH_SIZE + MAX_SAMPLE_SIZE + MAX_BITMAP_PADDING == + MAX_BLOCK_SIZE, + "block flush size needs headroom for one complete sample"); + + SampleBlock *block = writer->block; + /* Start a new block if this sample might not fit */ + if (block->size >= BLOCK_FLUSH_SIZE || + (entry->block_stream_index == UINT32_MAX && + block->stream_count == MAX_BLOCK_STREAMS)) { + if (block_flush(writer) < 0) { + return -1; + } + } + if (block->sample_count == 0) { + block->base_timestamp = timestamp; + } + int new_stream = entry->block_stream_index == UINT32_MAX; + if (new_stream && block_add_stream(writer, entry) < 0) { + return -1; + } + /* Store the difference from the expected timestamp */ + uint64_t residual = + timestamp - entry->prev_timestamp - writer->sample_interval_us; + uint8_t residual_bytes[MAX_VARINT_SIZE]; + size_t residual_size = + encode_varint_i64(residual_bytes, (int64_t)residual); + if (column_write_varint(block, COLUMN_SAMPLE_STREAM_IDS, + entry->block_stream_index) < 0 || + column_write(block, COLUMN_TIMESTAMP_RESIDUALS, + residual_bytes, residual_size) < 0 || + column_write(block, COLUMN_STATUS_BITMAPS, + &status, sizeof(status)) < 0) { + return -1; + } + if (write_stack(writer, entry, new_stream, stack, depth) < 0) { + return -1; + } + entry->prev_timestamp = timestamp; + block->sample_count++; + writer->total_samples++; return 0; } +static void +block_destroy(SampleBlock *block) +{ + if (block == NULL) { + return; + } + for (unsigned i = 0; i < COLUMN_COUNT; i++) { + buffer_clear(&block->columns[i]); + } + buffer_clear(&block->repeats); + buffer_clear(&block->transformed); + buffer_clear(&block->compressed); + buffer_clear(&block->output); + PyMem_Free(block); +} + BinaryWriter * binary_writer_create(PyObject *path, uint64_t sample_interval_us, int compression_type, uint64_t start_time_us, int profiling_mode, int capture_features) { + if (compression_type != COMPRESSION_NONE && + compression_type != COMPRESSION_ZSTD) { + PyErr_SetString(PyExc_ValueError, "invalid compression type"); + return NULL; + } BinaryWriter *writer = PyMem_Calloc(1, sizeof(BinaryWriter)); if (!writer) { PyErr_NoMemory(); @@ -761,12 +1118,11 @@ binary_writer_create(PyObject *path, uint64_t sample_interval_us, int compressio writer->profiling_mode = profiling_mode; writer->capture_features = capture_features; - writer->write_buffer = PyMem_Malloc(WRITE_BUFFER_SIZE); - if (!writer->write_buffer) { + writer->block = PyMem_Calloc(1, sizeof(SampleBlock)); + if (writer->block == NULL) { PyErr_NoMemory(); goto error; } - writer->buffer_size = WRITE_BUFFER_SIZE; writer->string_hash = _Py_hashtable_new_full( string_hash_func, @@ -964,6 +1320,13 @@ process_thread_sample(BinaryWriter *writer, PyObject *thread_info, if (status_long == -1 && PyErr_Occurred()) { return -1; } + if (status_long < 0 || status_long > UINT8_MAX) { + writer->state = BINARY_WRITER_LIMIT_REACHED; + PyErr_Format(PyExc_OverflowError, + "thread status %ld exceeds the binary format range", + status_long); + return -1; + } uint8_t status = (uint8_t)status_long; ThreadEntry *entry = writer_get_or_create_thread_entry( @@ -972,66 +1335,14 @@ process_thread_sample(BinaryWriter *writer, PyObject *thread_info, return -1; } - /* Calculate timestamp delta */ - uint64_t delta = timestamp_us - entry->prev_timestamp; - /* Process frames and build current stack */ uint32_t curr_stack[MAX_STACK_DEPTH]; size_t curr_depth; if (build_frame_stack(writer, frame_list, curr_stack, &curr_depth) < 0) { return -1; } - - /* Compare with previous stack to determine encoding */ - size_t shared_count, pop_count, push_count; - int encoding = compare_stacks( - entry->prev_stack, entry->prev_stack_depth, - curr_stack, curr_depth, - &shared_count, &pop_count, &push_count); - - if (encoding == STACK_REPEAT) { - /* Buffer this sample for RLE. - * - * STACK_REPEAT also covers the "first sample for a fresh thread, - * empty stack" case: a new ThreadEntry has prev_stack_depth == 0 - * and a zero-initialized prev_stack, so compare_stacks() returns - * STACK_REPEAT against an empty curr_stack (depth 0). Buffering - * it here is correct; the RLE flush path emits it as a normal - * STACK_REPEAT record. */ - if (entry->pending_rle == NULL) { - entry->pending_rle = PyMem_Malloc(MAX_RLE_BUF_SIZE); - if (!entry->pending_rle) { - PyErr_NoMemory(); - return -1; - } - } - if (entry->pending_rle_bytes + MAX_RLE_ENTRY_SIZE > MAX_RLE_BUF_SIZE - && flush_pending_rle(writer, entry) < 0) { - return -1; - } - entry->pending_rle_bytes += encode_varint_u64( - entry->pending_rle + entry->pending_rle_bytes, delta); - entry->pending_rle[entry->pending_rle_bytes++] = status; - entry->pending_rle_samples++; - } else { - /* Stack changed - flush any pending RLE first */ - if (flush_pending_rle(writer, entry) < 0) { - return -1; - } - - if (write_sample_with_encoding(writer, entry, delta, status, encoding, - curr_stack, curr_depth, - shared_count, pop_count, push_count) < 0) { - return -1; - } - - memcpy(entry->prev_stack, curr_stack, curr_depth * sizeof(uint32_t)); - entry->prev_stack_depth = curr_depth; - } - - entry->prev_timestamp = timestamp_us; - writer->total_samples++; - return 0; + return block_add_sample( + writer, entry, timestamp_us, status, curr_stack, curr_depth); } int @@ -1078,47 +1389,10 @@ binary_writer_write_sample(BinaryWriter *writer, PyObject *stack_frames, uint64_ int binary_writer_finalize(BinaryWriter *writer) { - for (size_t i = 0; i < writer->thread_count; i++) { - if (flush_pending_rle(writer, &writer->thread_entries[i]) < 0) { - return -1; - } - } - - if (writer_flush_buffer(writer) < 0) { + if (block_flush(writer) < 0) { return -1; } -#ifdef HAVE_ZSTD - /* Finalize compression stream */ - if (writer->compression_type == COMPRESSION_ZSTD && writer->zstd.cctx) { - ZSTD_inBuffer input = { NULL, 0, 0 }; - size_t remaining; - - do { - ZSTD_outBuffer output = { - writer->zstd.compressed_buffer, - writer->zstd.compressed_buffer_size, - 0 - }; - - remaining = ZSTD_compressStream2(writer->zstd.cctx, &output, &input, ZSTD_e_end); - - if (ZSTD_isError(remaining)) { - PyErr_Format(PyExc_IOError, "zstd finalization error: %s", - ZSTD_getErrorName(remaining)); - return -1; - } - - _Py_MSAN_UNPOISON(writer->zstd.compressed_buffer, output.pos); - if (output.pos > 0) { - if (fwrite_checked_allow_threads(writer->zstd.compressed_buffer, output.pos, writer->fp) < 0) { - return -1; - } - } - } while (remaining > 0); - } -#endif - /* Use 64-bit file position for >2GB files */ file_offset_t string_table_offset = FTELL64(writer->fp); if (string_table_offset < 0) { @@ -1314,13 +1588,12 @@ binary_writer_destroy(BinaryWriter *writer) Py_fclose(writer->fp); } - PyMem_Free(writer->write_buffer); + block_destroy(writer->block); #ifdef HAVE_ZSTD if (writer->zstd.cctx) { ZSTD_freeCCtx(writer->zstd.cctx); } - PyMem_Free(writer->zstd.compressed_buffer); #endif if (writer->string_hash) { @@ -1342,7 +1615,6 @@ binary_writer_destroy(BinaryWriter *writer) if (writer->thread_entries) { for (size_t i = 0; i < writer->thread_count; i++) { PyMem_Free(writer->thread_entries[i].prev_stack); - PyMem_Free(writer->thread_entries[i].pending_rle); } PyMem_Free(writer->thread_entries); } diff --git a/Modules/_remote_debugging/clinic/module.c.h b/Modules/_remote_debugging/clinic/module.c.h index 349113c36fb5f06..03092596944311c 100644 --- a/Modules/_remote_debugging/clinic/module.c.h +++ b/Modules/_remote_debugging/clinic/module.c.h @@ -1139,7 +1139,7 @@ PyDoc_STRVAR(_remote_debugging_BinaryWriter_get_stats__doc__, "Get encoding statistics for the writer.\n" "\n" "Returns a dict with encoding statistics including\n" -"repeat/full/suffix/pop-push record counts, frames written/saved, and\n" +"repeat/full/suffix/reuse record counts, frames written/saved, and\n" "compression ratio."); #define _REMOTE_DEBUGGING_BINARYWRITER_GET_STATS_METHODDEF \ @@ -1711,4 +1711,4 @@ _remote_debugging_get_gc_stats(PyObject *module, PyObject *const *args, Py_ssize exit: return return_value; } -/*[clinic end generated code: output=8424a993a85ef2bd input=a9049054013a1b77]*/ +/*[clinic end generated code: output=8ecb3b99d67de4e6 input=a9049054013a1b77]*/ diff --git a/Modules/_remote_debugging/module.c b/Modules/_remote_debugging/module.c index efb6d600247bf75..a6f076da4cdc9cf 100644 --- a/Modules/_remote_debugging/module.c +++ b/Modules/_remote_debugging/module.c @@ -1853,8 +1853,8 @@ _remote_debugging_BinaryWriter_set_stats_impl(BinaryWriterObject *self, /* Finalize the writer, cache total_samples, and destroy it. * * The cache assignment must happen AFTER binary_writer_finalize(): finalize - * flushes pending RLE records and updates the file header before the writer is - * destroyed. Caching before finalize would preserve a count for a file that + * flushes the open sample block and updates the file header before the writer + * is destroyed. Caching before finalize would preserve a count for a file that * might still fail to finish. */ static int binary_writer_finalize_and_cache(BinaryWriterObject *self) @@ -1968,13 +1968,13 @@ _remote_debugging.BinaryWriter.get_stats Get encoding statistics for the writer. Returns a dict with encoding statistics including -repeat/full/suffix/pop-push record counts, frames written/saved, and +repeat/full/suffix/reuse record counts, frames written/saved, and compression ratio. [clinic start generated code]*/ static PyObject * _remote_debugging_BinaryWriter_get_stats_impl(BinaryWriterObject *self) -/*[clinic end generated code: output=06522cd52544df89 input=a8bb8c8682ccd34b]*/ +/*[clinic end generated code: output=06522cd52544df89 input=09c317b6289ae311]*/ { if (!self->writer) { PyErr_SetString(PyExc_ValueError, "Writer is closed");