runtime: add map_codec, replace inline map-entry wire loops in owned codegen#194
Merged
Conversation
|
All contributors have signed the CLA ✍️ ✅ |
The Docker path needs the private GHCR tools image (or a local image build) plus a Docker daemon — unavailable in some dev environments. conformance_test_runner drives the testee over stdin/stdout pipes, so nothing actually requires a container: - scripts/build-conformance-tools.sh (task conformance-tools-local): native equivalent of Dockerfile.tools — pre-installs jsoncpp, builds conformance_test_runner from the pinned protobuf tag into .local/bin/ (host-linked; the image's -static is only needed for distribution), and populates conformance/protos/ with the same proto set. - scripts/run-conformance-local.sh (task conformance-local): native equivalent of run-conformance.sh + the image build — cargo-builds the std/no_std conformance binaries and executes the same seven runs (including via-lazy) with the same failure lists and CONFORMANCE_OUT logging.
Every generated decode arm (merge_field, view decode, lazy record arms, map-entry loops) emitted a 7-line inline wire-type comparison + WireTypeMismatch construction — ~1,100 copies across the generated corpus, the single largest block of generated boilerplate (~8%). Add ::buffa::encoding::check_wire_type(tag, expected) with a #[cold] out-of-line wire_type_mismatch constructor, and rewrite the codegen wire_type_check helper to emit the one-line call. Repeated-field arms that accept two wire types (packed + unpacked) call the constructor directly, reporting the packed form as expected — same error payload as before. Moving error construction out of the per-field decode loop is icache-friendly; the hot path keeps a single inline comparison. The lazy-view record arms (added since this change was first written) go through the same helper, so the fold covers the lazy family too. Checked-in generated trees regenerated (buffa-types, buffa-descriptor); error values are byte-identical so wire behaviour is unchanged.
iainmcgin
force-pushed
the
codegen/check-wire-type
branch
from
bc3f644 to
0962e45
Compare
…codegen
Every owned map field expanded ~40-50 lines of entry size/write/merge
code that varied only by the key/value proto types. Capture the
variation as zero-sized per-proto-type codecs (proto types, not Rust
types — int32/sint32/sfixed32 share i32 with different encodings) in a
new buffa::map_codec module, plus generic field-level helpers:
field_len / write_field scalar-valued maps
message_field_len /
write_message_field message-valued maps (preserves the
two-pass SizeCache slot discipline:
both passes iterate the same map, so
slot order matches by construction)
merge_entry all maps (entry-limit arithmetic,
defaults, unknown-field skip,
position correction — identical to
the previous inline expansion)
Generated call sites name codecs by turbofish and let the map's own
types drive inference (Open<_>/Closed<_>/Msg<_>), so codegen needs no
type-path resolution. Everything monomorphizes to the previous code;
the fixed-width fast path (len() * const entry) is kept via
MapCodec::FIXED_LEN, which folds at compile time. Closed-enum map
values keep the existing drop-unknown behaviour (documented known gap).
View encode methods keep the previous duck-typed inline emission
(view maps store borrowed element types and view message values are
not buffa::Message), retained as map_view_* emitters; borrow-generic
codecs for views are a possible follow-up.
Checked-in generated trees regenerated; map wire output is unchanged.
Adapted to the DecodeContext threading introduced by the unknown-field
limit work (#184): MapValueDecode::merge and merge_entry take
DecodeContext instead of a bare depth, message values forward it to
merge_length_delimited, and unknown entry fields skip via ctx.depth() —
matching the inline expansion this replaces on current main.
iainmcgin
force-pushed
the
runtime/map-codec
branch
from
0f31feb to
2b7e734
Compare
Resolve conflicts from #192 and #193 being squash-merged onto main while their pre-squash commits remained on this branch: - impl_message.rs: keep this branch's map_codec rewrite, which already carries the check_wire_type refactor; main's only change here was that same refactor. - CHANGELOG.md: keep both the check_wire_type and map_codec entries under Unreleased. - Regenerate checked-in WKT and bootstrap descriptor types; only the descriptor view changes, picking up the proto2 required-field presence work from main.
rpb-ant
approved these changes
Jun 18, 2026
iainmcgin
added a commit
that referenced
this pull request
Jun 18, 2026
Propagate #194's merge of main (which carries #192, #193, #199, #200) up the stack. Clean merge: the put_*_field writer changes don't overlap the map_codec/main changes, and the merge base already contains the #192/#193 commits, so no squash-merge-artifact conflict this time. Generated trees unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Adds a
buffa::map_codecmodule and replaces the ~40–50-line inline entry size/write/merge expansion that generated code emitted per owned map field. The per-proto-type variation (int32/sint32/sfixed32 all map toi32with different encodings) is captured as zero-sized codec types; generated call sites name codecs by turbofish and let the map's own types drive inference, so codegen needs no type-path resolution.field_len/write_field— scalar-valued maps; the fixed-width fast path (len() * const entry) is preserved viaMapCodec::FIXED_LEN, which folds at compile time.message_field_len/write_message_field— message-valued maps, preserving the two-passSizeCacheslot discipline (both passes iterate the same map, so slot order matches by construction).merge_entry— all maps: entry-limit arithmetic, defaults, unknown-field skip, position correction, identical to the previous inline expansion.Everything monomorphizes to the previous code; map wire output is unchanged. Closed-enum map values keep the existing drop-unknown behaviour (documented known gap). View encode paths keep their duck-typed inline emission (view maps store borrowed element types); borrow-generic view codecs are a possible follow-up.
One adaptation relative to the original branch:
MapValueDecode::mergeandmerge_entrytake theDecodeContextintroduced by #184 (recursion + unknown-field budgets) instead of a bare depth, matching the inline expansion they replace on current main.Checked-in generated trees regenerated.
How we know it works
task lintclean.Stacked on #193.