Skip to content

Raise flatbuffers verifier max_depth for pipeline deserialization - #9427

Open
alexreinking wants to merge 1 commit into
mainfrom
fix-flatbuffer-verifier-depth
Open

Raise flatbuffers verifier max_depth for pipeline deserialization#9427
alexreinking wants to merge 1 commit into
mainfrom
fix-flatbuffer-verifier-depth

Conversation

@alexreinking

Copy link
Copy Markdown
Member

Summary

#9395 added flatbuffer verification before deserializing a pipeline, but used the verifier's default max_depth of 64. Each Stmt/Expr node serializes as one flatbuffer table, so any pipeline with a moderately deep expression nests past that and gets rejected as "malformed" even though it's valid. This is what broke the serialization round-trip CI bot (arm-64 / coverage, which only runs the roundtrip flag on push to main, not on PRs) and led to the revert in #9426.

This is a fix-forward: raise max_depth to 1000 at both verification call sites instead of leaving verification out entirely.

  • Reproduced the exact CI failures from the arm-64 / coverage job logs: correctness_align_bounds, correctness_code_explosion, correctness_integer_powers, correctness_loop_carry, correctness_lots_of_loop_invariants, correctness_many_inlined_selects — all build a single Expr nested past depth 64 (e.g. summing 100+ loop invariants into one Add tree).
  • 1000 is a judgment call, not a derived constant: comfortably above the deepest failing test (~200-400) and the coverage bot doesn't exercise apps/performance tests, so the deepest real pipelines are untested against this bound. It's also well within safe recursion depth for deserialize_stmt/deserialize_expr, which recurse to a depth matching the buffer's nesting.
  • Verified a corrupted buffer is still rejected, so this doesn't disable the ASAN-motivated protection verify serialized buffer before accessing it in the deserializer #9395 was added for — it only widens what counts as structurally valid.

Test plan

  • Reproduced all 6 originally-failing correctness tests locally under WITH_SERIALIZATION_JIT_ROUNDTRIP_TESTING, confirmed they pass with this fix
  • Added test/correctness/serialization.cpp: round-trips a pipeline with an Expr nested past the old default depth, and confirms a corrupted buffer still throws
  • Confirmed the new test fails without the fix and passes with it
  • correctness_streaming and correctness_generator_cache (the existing tests that exercise real serialize/deserialize round-trips) still pass
  • pre-commit run passes on all changed files

Co-authored-by: nashit hayat nashit@bugqore.com
Co-authored-by: Claude Sonnet 5 noreply@anthropic.com

PR #9395 added flatbuffer verification before deserializing a
pipeline, but used the verifier's default max_depth (64). Each
Stmt/Expr node serializes as one flatbuffer table, so any pipeline
with a moderately deep expression nests past that and gets rejected
as "malformed" even though it's valid -- this is what broke the
serialization round-trip CI bot and led to the revert in #9426.

Raise max_depth to 1000 at both verification call sites instead of
reverting: comfortably covers the deepest of the correctness tests
that were failing (~200-400), while still bounding the deserializer's
recursive descent against a corrupt or adversarial buffer. Verified
a corrupted buffer is still rejected.

Adds test/correctness/serialization.cpp, a regression test covering
both directions: a pipeline with an expression nested past the old
default round-trips cleanly, and a corrupted buffer still throws.

Co-authored-by: nashit hayat <nashit@bugqore.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 50.00000% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.16%. Comparing base (0537858) to head (178c03e).

Files with missing lines Patch % Lines
src/Deserialization.cpp 50.00% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9427      +/-   ##
==========================================
+ Coverage   70.02%   70.16%   +0.14%     
==========================================
  Files         261      261              
  Lines       79393    79403      +10     
  Branches    19359    19361       +2     
==========================================
+ Hits        55593    55713     +120     
+ Misses      17915    17892      -23     
+ Partials     5885     5798      -87     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants