Skip to content

feat(cdr)!: reflection-driven CDR/XCDR serialization (replaces CdrWriter/CdrReader) - #704

Closed
finger563 wants to merge 3 commits into
feat/reflect-cpp-componentfrom
feat/cdr-v2
Closed

feat(cdr)!: reflection-driven CDR/XCDR serialization (replaces CdrWriter/CdrReader)#704
finger563 wants to merge 3 commits into
feat/reflect-cpp-componentfrom
feat/cdr-v2

Conversation

@finger563

Copy link
Copy Markdown
Contributor

Description

Stacked on #703 (feat/reflect-cpp-component) — merge that first, then re-target this PR to main.

Replaces the manual espp::CdrWriter/espp::CdrReader component with finger563/cdr, a reflection-driven CDR/XCDR serialization library (vendored as a detail/ submodule): the compiler generates serialization code from plain struct definitions — no IDL compiler, no hand-written write/read call sequences.

struct ImuSample {
  uint64_t stamp_us;
  std::array<float, 3> accel;
  std::array<float, 3> gyro;
  float temperature;
};

auto bytes = cdr::serialize(sample);              // XCDR2, appendable (default)
auto ros2  = cdr::serialize<cdr::xcdr1>(sample);  // ROS 2 / classic-CDR peers
auto back  = cdr::deserialize<ImuSample>(*bytes); // std::expected<ImuSample, cdr::error>

What the new library adds over the old component:

  • XCDR2 (plain + delimited/appendable with DHEADER — FastDDS/OpenDDS defaults) alongside XCDR1 (ROS 2 / CycloneDDS defaults), both endiannesses
  • Appendable schema evolution in both directions (unknown trailing members skipped; missing members keep defaults)
  • Wire format byte-verified against pycdr2 (CycloneDDS's codec) — 10/10 differential fixtures identical; the Python side of a message is a plain pycdr2 dataclass, no native bindings
  • std::expected errors with code, payload offset, and failing field name; bounds-checked, fuzz-tested deserializers (ASan/UBSan-clean incl. a ~46k-case mutation stress suite in the library repo)
  • cdr::bounded_string<N> / bounded_vector<T, N>, PL_CDR param_list_writer/param_list_reader for SPDP/SEDP, zero-allocation serialize_into, body-only variants

BREAKING CHANGE

espp::CdrWriter / espp::CdrReader are gone, including their Python bindings (espp.CdrWriter/espp.CdrReader). Migrations in this PR:

  • rtps: discovery string/octet-sequence parameters via cdr::serialize_body/deserialize_body; PL_CDR encapsulation handled inline
  • pc tests + rtps/p4 examples: uint32 payload helpers via cdr::serialize<cdr::xcdr1> / cdr::deserialize<uint32_t>
  • lib: builds C++23; cdr python bindings removed (python CDR payloads use pycdr2 / struct.pack)
  • python: rtps scripts use pure-python CDR helpers

Verification

  • cdr and rtps examples build on ESP32-S3 (IDF 6; also expected green on CI's v5.5.1 — its default gnu++2b provides std::expected)
  • Host espp_pc lib + _espp python module build (C++23)
  • pc/build/rtps_pubsub end-to-end test passes: 16/16 samples through SPDP/SEDP discovery + user data on the new serializer
  • Migrated pure-python helpers byte-checked in both endiannesses
  • Not build-verified: the P4 board example (long display/LVGL build; its helper change is identical in shape to the rtps example's)

Notes

  • Requires C++23 (std::expected) — default on IDF 5.2+ toolchains (gnu++2b/gnu++26); manifest declares idf: >=5.2
  • Keep the reflect_cpp submodule pin and finger563/cdr's FetchContent tag on the same reflect-cpp version (currently v0.25.0)
  • Docs: doc/en/data/cdr.rst rewritten; example README updated; vendored detail/ excluded from cppcheck

🤖 Generated with Claude Code

Copilot AI lite review requested due to automatic review settings August 11, 2026 17:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces the manual espp::CdrWriter/espp::CdrReader API (and its Python bindings) with a reflection-driven CDR/XCDR serialization approach using the vendored finger563/cdr library backed by reflect_cpp, and migrates RTPS code/examples plus host/PC support to the new API (C++23).

Changes:

  • Migrate RTPS parameter-list encoding/decoding and examples/tests from espp::CdrWriter/espp::CdrReader to cdr::{serialize,deserialize,serialize_body,deserialize_body}.
  • Remove CDR Python bindings and update Python RTPS scripts to use pure-struct CDR helpers.
  • Update build/docs/packaging to support the new header-only CDR library and require C++23 (std::expected).

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
suppressions.txt Excludes vendored components/cdr/detail/* from cppcheck, consistent with other vendored trees.
python/rtps_subscriber.py Replaces espp.CdrReader usage with pure-Python struct decoding for uint32.
python/rtps_pubsub.py Replaces espp.CdrWriter/Reader with pure-Python struct packing/unpacking for uint32.
python/rtps_publisher.py Replaces espp.CdrWriter with pure-Python struct packing for uint32.
pc/tests/rtps_common.hpp Migrates test helpers to cdr::serialize/deserialize for uint32 payloads.
pc/CMakeLists.txt Bumps host PC build standard to C++23.
lib/python_bindings/module.cpp Removes registration of CDR Python bindings; keeps RTPS bindings.
lib/python_bindings/cdr_bindings.cpp Deletes the hand-written CDR pybind11 shim.
lib/espp.cmake Drops old cdr source/bindings; adds include paths for vendored CDR + reflect-cpp; bumps Python module to C++23.
lib/CMakeLists.txt Bumps library compile features to C++23.
lib/autogenerate_bindings.py Updates note/exclusion rationale for cdr.hpp now that Python uses pycdr2 instead of bindings.
doc/en/data/cdr.rst Rewrites component docs to describe reflection-driven XCDR1/XCDR2 + std::expected API.
components/rtps/src/rtps.cpp Migrates PL_CDR parameter writing/reading to new CDR helpers; introduces shared padding helper.
components/rtps/example/main/rtps_example.cpp Migrates example payload helpers to cdr::serialize/deserialize.
components/esp32-p4-function-ev-board/example/main/esp32_p4_function_ev_board_example.cpp Migrates example payload helpers to cdr::serialize/deserialize.
components/cdr/README.md Updates component README to new reflection-driven library and capabilities.
components/cdr/include/cdr.hpp Adds umbrella header that forwards to the vendored cdr library headers.
components/cdr/idf_component.yml Updates description/tags; raises minimum IDF to >=5.2; adds dependency on espp/reflect_cpp.
components/cdr/example/README.md Updates example documentation to match new functionality and API.
components/cdr/example/main/cdr_example.cpp Replaces old imperative reader/writer example with reflection-driven struct-based demo.
components/cdr/example/CMakeLists.txt Adds reflect_cpp and bumps example to C++23.
components/cdr/CMakeLists.txt Switches cdr component to header-only registration; includes vendored headers; requires reflect_cpp.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread components/cdr/example/main/cdr_example.cpp Outdated
Comment thread components/rtps/src/rtps.cpp
Comment thread components/rtps/example/main/rtps_example.cpp
Comment thread pc/tests/rtps_common.hpp
finger563 and others added 3 commits August 11, 2026 12:45
… library

The cdr component now vendors finger563/cdr (detail/ submodule): the
compiler generates CDR serialization from plain struct definitions — no
IDL, no hand-written write/read sequences. Adds XCDR2 (delimited/
appendable + DHEADER schema evolution), both endiannesses, std::expected
errors with field names, bounded types, and PL_CDR parameter-list
helpers for RTPS discovery. Wire format is byte-verified against pycdr2
(CycloneDDS). Depends on the new reflect_cpp component; requires C++23
(default on IDF 5.2+ toolchains).

BREAKING CHANGE: espp::CdrWriter / espp::CdrReader are gone; use
cdr::serialize / cdr::deserialize<T> / cdr::param_list_writer instead.

Example builds on ESP32-S3 / IDF 6 (389 KB total image incl. logger+fmt).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… API

- rtps: discovery string/octet-sequence parameters serialize via
  cdr::serialize_body / deserialize_body; PL_CDR encapsulation handled
  inline (the old CdrWriter/CdrReader helpers are gone)
- pc tests + rtps/p4 examples: uint32 payload helpers use
  cdr::serialize<cdr::xcdr1> / cdr::deserialize<uint32_t>
- lib: espp builds C++23; cdr/reflect-cpp detail headers added to the
  include set; cdr python bindings removed — the python side of a CDR
  message is a plain pycdr2 dataclass (or struct.pack for primitives),
  so espp.CdrWriter/CdrReader no longer exist
- python: rtps scripts use pure-python CDR helpers for uint32 payloads

Verified: rtps example builds on ESP32-S3/IDF 6; host lib + _espp python
module build; pc rtps_pubsub end-to-end test passes (16/16 samples
through SPDP/SEDP discovery + user data on the new serializer).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Updates the sphinx page and example README for the new struct-based
serialize/deserialize API, and excludes the vendored detail/ submodule
from cppcheck (matching lib/).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@finger563

Copy link
Copy Markdown
Contributor Author

Applied all review feedback (thanks — the example error-handling one was a real UB bug):

  • cdr_example.cpp: the two serialize results are checked separately, each reporting its own error
  • rtps.cpp: append_cdr_parameter drops parameters whose padded length exceeds the 16-bit parameterLength instead of truncating on the wire
  • rtps_example.cpp / rtps_common.hpp / P4 example: guards on the std::expected results before dereferencing

The branch was also rebased as part of fixing the stack split in #703 — the cdr submodule and old-file removals now land in this PR's first commit where they belong.

Re-verified after the rewrite: cdr and rtps examples build on ESP32-S3/IDF 6, host lib + _espp build, and pc/build/rtps_pubsub still passes 16/16 end-to-end. Note the example-build matrix only triggers once this PR re-targets to main after #703 merges.

@finger563

Copy link
Copy Markdown
Contributor Author

Superseded by #705 — after #703 merged and its branch was deleted, GitHub could not retarget or reopen this PR (the head branch had been force-pushed while it was closed). #705 is the same branch rebased onto main with all review feedback from this PR applied.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants