fix(net): store the checksum an incremental update computes - #1733
fix(net): store the checksum an incremental update computes#1733daniel-noland wants to merge 8 commits into
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
72c2121 to
87a5d73
Compare
56e8187 to
61049ae
Compare
87a5d73 to
b390fbe
Compare
61049ae to
dd6e234
Compare
b390fbe to
52fc0c9
Compare
dd6e234 to
1ba8747
Compare
52fc0c9 to
a6d9026
Compare
1ba8747 to
94cdc87
Compare
a6d9026 to
1c9f520
Compare
a9d9273 to
3268b44
Compare
47a4260 to
5499151
Compare
3268b44 to
d6c577e
Compare
Debug binaries retained the complete Rust toolchain through their standard-library source paths, adding roughly 2.4 GB to the closure. They also carried a sizable DWARF index that neither packaged debugger consumes. Point those paths at the much smaller rust-src component, omit unused prebuilt documentation, and remove .debug_names. Source browsing and symbols remain available while the resulting diagnostic images become practical to store and transfer. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Daniel Noland <daniel@githedgehog.com>
A core collected from the lab is useful only with the exact unstripped binaries and sources that produced it. A general debugging toolbox cannot reconstruct that relationship after the release has moved on. Provide a purpose-built gdb image alongside each build and teach it Rust's standard-library types without retaining rustc. This keeps post-mortem debugging reproducible while avoiding unrelated live-debugging tools. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Post-mortem inspection and live debugging need different tools. The core viewer cannot offer an editor-driven session, while bugstalker understands Rust layouts and can expose the running dataplane through the Debug Adapter Protocol. Track bugstalker upstream for its current remote DAP support and package it separately with the matching binaries and sources. Keeping the image single-purpose avoids making every diagnostic artifact carry every debugger. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Some failures need a record of the dataplane's kernel interactions rather than an interactive debugger. A small, repeatable tracing environment is easier to deploy and feed into existing log analysis than a general-purpose toolbox. Package lurk around the matching release binaries and follow the worker threads where the dataplane does its work. Because syscall tracing needs no symbols, this image can stay much smaller than the debugger images. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Daniel Noland <daniel@githedgehog.com>
The diagnostic images are useful only when they match the build being investigated, but building roughly 850 MB of extra images for every pull request would undermine the runner-load reduction this CI rework is meant to achieve. Build them automatically for pushes, the merge queue, and manual runs, with an explicit label available for debugging a pull request. Publish all three beside tagged releases so the matching tools remain available when a deployed build needs investigation. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Daniel Noland <daniel@githedgehog.com>
All three images built green while two of them could not do what the
README documented. Building proves an image links; it says nothing
about whether its entrypoint runs.
`smoke-container` runs each one the way the README tells a user to:
- the tracer under a plain `docker run`, with no seccomp relaxation,
and requires an `execve` in the trace. Its failure mode is the
reason this exists: lurk emitted eight well-formed JSON lines
recording its own child failing to start, then exited 0, which the
`jq -R 'fromjson? // empty'` filter we document accepts without
complaint. Both guards fire against the pre-fix image.
- the core viewer through its own entrypoint rather than by invoking
gdb directly, since the `--directory` and `source` flags that
register the printers live in that entrypoint. Checks that the
printer set is registered, not merely that gdb started.
- the debugger only for coming up and listening. Driving a real DAP
session from CI means carrying a protocol client in-tree, and the
contract it would pin is exercised better by pointing an editor at
the image. It also would not have caught the defect on that image,
which was in the documentation rather than the runtime.
The trace goes to a file rather than a shell variable: at a few
megabytes it overruns the here-string limit, and every grep against it
then fails with E2BIG, which reads exactly like a failed trace.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Daniel Noland <daniel@githedgehog.com>
The published images debug what CI built. Debugging what you are
building meant either rebuilding an image by hand or falling back to a
system gdb, which is exactly the case where symbols do not line up.
`just debug <tool> <target>` builds the image that carries the tool at
the current profile, platform, instrumentation, and sanitizer, and runs
the target inside it:
- `lurk` traces syscalls and streams JSON until the program exits.
- `gdb` runs gdbserver and waits, printing the `target remote` line.
- `bugstalker` waits for a DAP client, printing the `program` and
`args` for the launch request -- in remote-DAP mode it takes those
from the client rather than from its own command line.
`target` is either one of the binaries the images already carry, in
which case nothing needs mounting, or a nextest filter. For a test the
archive is built and unpacked, and that binary was built outside the
image, so the store comes along read-only and it runs with its package
directory as the working directory the way nextest runs it.
Naming a target exactly is the tedious part -- test paths are long and
nobody remembers them -- so leaving it out offers everything through
skim, and a filter matching several offers those. Resolution stays
unambiguous: exactly one match runs without asking, no match is an
error, and several with no terminal to ask at is an error listing them
rather than a guess. That last case is what keeps this safe to call
from a script.
`just debug-list` prints the same list without running anything, and
`just inspect-core` opens a core file in a gdb built from the same
settings.
Randomization stays enabled under gdbserver. Docker's default seccomp
answers personality(ADDR_NO_RANDOMIZE) with EPERM, and while gdbserver
treats that as non-fatal -- unlike lurk, which is why lurk is patched --
it otherwise opens with a warning that reads like a real failure.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Daniel Noland <daniel@githedgehog.com>
`Checksum::increment_update_checksum` works out the new checksum and hands it back. It does not store it, despite taking `&mut self`. `EmbeddedTransport::update_checksum` called it and dropped the answer on the floor, under a comment about ignoring errors on a truncated header -- but there is no error to ignore; the return value is the checksum. So no quoted transport header has ever had its checksum updated. NAT rewrites the ports of the packet quoted inside an ICMP error and calls this to keep the checksum in step, and the call has been doing nothing. The only correct uses of the trait method in the tree are in a test, which is presumably how the shape survived. Setting it can fail, on a header too truncated to hold a checksum -- but that is a header the caller could not have read a checksum out of either, and every caller reads one first. Hence discarding that error rather than the value, which is what the comment was reaching for. update_checksum_for_address is new, and folds a change of one of the quoted packet's addresses in. TCP, UDP and ICMPv6 are checksummed over a pseudo-header built from the source and destination addresses, so rewriting one leaves the quoted checksum describing an address that is no longer there. ICMPv4 has no pseudo-header and is left alone. Incremental for the same reason as the rest: a quote is usually truncated, so there is no payload to compute over from scratch. The tests build the one case where both routes are open -- a full header over a known payload -- and hold the fold against a computation from scratch, over two words for v4 and eight for v6. The checksum starts out correct on purpose: an RFC 1624 update is exact given a correct starting value and says nothing given a wrong one. Both fail against the discarded update; the ICMPv4 one passes either way, which is the point of it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Daniel Noland <daniel@githedgehog.com>
5499151 to
9f7481b
Compare
d6c577e to
f97fb60
Compare
fdf4a49 to
9016291
Compare
|
Its single commit now opens #1734; a one-commit PR with a ten-commit child was a review artifact, not a chapter. |
First of five, stacked on #1724. Extracts the finished fuzzing campaign from
pr/daniel-noland/icmp-flow-lock, which was never opened as a PR and haddrifted well behind
main.This one is a production bug, on its own so it can be taken without the rest.
Checksum::increment_update_checksumworks out the new checksum and hands itback; it does not store it, despite taking
&mut self.EmbeddedTransport::update_checksumcalled it and dropped the answer, under acomment about ignoring errors on a truncated header — but there is no error to
ignore, the return value is the checksum. So no quoted transport header has
ever had its checksum updated.
NAT rewrites the ports of the packet quoted inside an ICMP error and calls this
to keep the checksum in step. Receivers that validate the quoted checksum will
reject the error; that is the path PMTUD depends on. Plenty of stacks do not
validate it, which is likely why nothing has complained.
update_checksum_for_addressis new, and folds a change of one of the quotedpacket's addresses in — TCP, UDP and ICMPv6 checksum over a pseudo-header built
from the addresses, so rewriting one leaves the quoted checksum describing an
address that is no longer there. ICMPv4 has no pseudo-header and is left alone.
Verified locally rather than in CI, given the outage:
dataplane-net473/473,fmt --checkandclippy -D warningsclean. Reverting the fix fails exactlya_v4_address_change_matches_a_fresh_tcp_checksumand its v6/UDP twin.🤖 Generated with Claude Code