Skip to content

[FIX] Accept NumPy-array masks in the profiler's load/store statistics - #538

Open
mark14wu wants to merge 1 commit into
mainfrom
fix/profiler-block-pointer-mask
Open

mark14wu wants to merge 1 commit into
mainfrom
fix/profiler-block-pointer-mask

Conversation

@mark14wu

@mark14wu mark14wu commented Nov 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

The profiler crashes on every tensor-descriptor load or store:

src = tl.make_tensor_descriptor(in_ptr, shape=[N], strides=[1], block_shape=[BLOCK])
x = src.load([pid * BLOCK])
# AttributeError: 'memoryview' object has no attribute 'size'
#   tilelens/clients/profiler/profiler.py, in _get_mask_stats: total_count = mask.data.size

Triton's interpreter implements create_descriptor_load/create_descriptor_store by calling the masked load/store with the bounds mask from TensorDescHandle.materialize_pointers, which is a plain NumPy array, not a TensorHandle. The profiler's load/store callbacks read mask.data; for a NumPy array that is a memoryview, which has no .size. On Triton 3.7, tl.make_block_ptr loads and stores reach the same path through create_tensor_pointer_load/create_tensor_pointer_store, so block-pointer kernels crashed there too; on Triton 3.8 they pass a TensorHandle and are unaffected.

This PR adds _mask_data, which returns a NumPy-array mask as is and a TensorHandle's .data otherwise, and uses it in _get_mask_stats and _report_load_store_bytes. (np.count_nonzero happened to accept the memoryview, so only .size failed.)

Tests

  • New in tests/end_to_end/test_profiler.py: test_mask_percentage_of_tensor_descriptor_accesses copies a 100-element tensor through descriptors in 4 blocks of 32 and checks the mask counts (128 total, 28 masked, for loads and stores) and the byte counts. It fails on main with the error above.
  • pytest tests/: 366 passed, 6 skipped on main; 367 passed, 6 skipped with this PR (Python 3.12, Triton 3.8.0).
  • ruff check and ruff format --check (v0.1.9) pass on the changed files.

🤖 Generated with Claude Code

A tensor-descriptor load or store (and, on Triton 3.7, a block-pointer
one) reaches the masked load/store with its bounds mask as a plain NumPy
array rather than a TensorHandle. The profiler read `mask.data.size`,
and an array's `.data` is a memoryview, so every such access raised
AttributeError: 'memoryview' object has no attribute 'size'.
`_mask_data` now returns the array itself, and both the mask statistics
and the byte counts use it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown

Performance Benchmark

Benchmark main (min) PR (min) Change Samples
gemm 0.102s 0.101s -0.4% 20 / 20
gemm_oob 0.113s 0.113s -0.3% 20 / 20
indirect_load 0.021s 0.021s +1.1% 20 / 20
nested_loop 0.224s 0.224s -0.0% 20 / 20
block_pointer_loop_advance 0.121s 0.121s +0.3% 20 / 20
liger_jsd 0.133s 0.133s +0.2% 20 / 20
flaggems_layernorm 0.380s 0.380s -0.1% 20 / 20
swiglu 0.164s 0.163s -0.8% 20 / 20
cross_entropy 0.944s 0.944s -0.0% 20 / 20
fused_linear_jsd 0.203s 0.203s +0.2% 20 / 20
Total 2.405s 2.404s -0.1% N/A

Iterations: 1 warmup + 20 measured
Samples are shown as main / PR; long pytest benchmarks may use fewer samples.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant