diff --git a/CHANGELOG.md b/CHANGELOG.md index 4e2194d..a9bceb3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -44,6 +44,41 @@ dpkt-throughput regression" section for the full profiling writeup, before/after numbers under both the old and new benchmark methodology, and updated 1.1/1.2/1.6 figures. +### Documentation +- **Corrected six README.md claims an independent audit against the + live repo found stale or overstated, none of them accidental + regressions — each had simply drifted from what it once described.** + Wheel size (85.6 KB / ~30× smaller than scapy) was measured against + the `2.0.0` wheel, two releases behind; rebuilt from `HEAD`, it's + 88.3 KB / ~29× (`docs/CLAIMS.md` §1.5 corrected to match). The typing + bullet's "107 of [scapy's] files" enable strict mypy checking was a + raw `wc -l` over scapy's enabled-files list including blank lines + and comments; filtered the way scapy's own tooling does, it's 89 + (`docs/CLAIMS.md` §2.1 corrected, with the mismeasurement documented + so it doesn't recur). The Pyodide/browser bullet credited "dpkt and + pypacker import cleanly" to the real-Pyodide CI job; that job only + ever imports netprotocols under Pyodide (confirmed by reading + `scripts/pyodide/check_in_pyodide.py`) — the dpkt/pypacker rows come + from a separate `sys.meta_path`-blocklist simulation under ordinary + CPython, not from CI (`docs/CLAIMS.md` §3.1 reworded to say so + explicitly). The CI-gating bullet's closing clause, "the rest have no + performance benchmark at all," was contradicted by the very citation + it points to — `docs/CLAIMS.md` §1.7 documents that scapy and + PyTCP-net_proto do ship benchmark scripts, just not CI-wired; reworded + to the claim the table actually supports (none of the ten *gates* on + a regression). The Contributing section's "`pytest`, `mypy`, and + `ruff check` — all three are enforced by CI on Python 3.12-3.14" + overstated the matrix: only `pytest` runs on 3.12-3.14; `mypy` and + `ruff check` each run on 3.12 only. The Roadmap section's "everything + through 2.2.0 has landed on `master`" didn't mention that a further, + unreleased fix (#147) has since landed on top of 2.2.0 — added a + clause naming it. A seventh finding — the Roadmap's claim that "only + 2.0.0 was an actual PyPI release" — is contradicted by PyPI itself + (2.2.0, and in fact five earlier minor/patch versions, are also + published there); left unfixed pending a maintainer decision on + whether 2.2.0's publication was intentional, since fixing the prose + first would paper over a real process question. + ## [2.2.0] - 2026-09-04 ### Added diff --git a/README.md b/README.md index 06d750f..d7a8133 100644 --- a/README.md +++ b/README.md @@ -70,18 +70,19 @@ below, its reproduction command, and its caveats): and never touches the host doing it — importing scapy populates live interface and routing tables as a side effect of the import statement. -- **An 85.6 KB wheel against scapy's 2.47 MB** — about 30× smaller. +- **An 88.3 KB wheel against scapy's 2.47 MB** — about 29× smaller. - **Regression-gated in CI, which — as far as we could establish by auditing ten comparable Python packet libraries' CI configurations (dpkt, scapy, pypacker, construct, pcapkit, dnspython, pyshark, nfstream, stackforge, PyTCP-net_proto; full citations in - [docs/CLAIMS.md §1.7](docs/CLAIMS.md)) — none of them do.** Two of - the ten ship real benchmark code that simply never runs in CI - (construct disables it explicitly; stackforge's Criterion benches are - never invoked); the rest have no performance benchmark at all. + [docs/CLAIMS.md §1.7](docs/CLAIMS.md)) — none of them gate on one.** + Four of the ten (scapy, construct, stackforge, PyTCP-net_proto) ship + real benchmark code that simply never runs in CI (construct disables + it explicitly; stackforge's Criterion benches are never invoked); none + of the ten fails a build on a performance regression. - **Typed where the alternatives are not.** `mypy --strict` over the whole of `src/`, enforced in CI. scapy ships `py.typed` but enables - strict checking on 107 of its files, 2 of the 121 under + strict checking on 89 of its files, 2 of the 121 under `scapy/layers/` — the dissectors most code touches stay `Any`. dpkt ships no `py.typed` at all, and no third-party stub package exists for it. @@ -92,10 +93,11 @@ below, its reproduction command, and its caveats): 2022-08-18 and last committed 2024-05-05, still targets Python 2.7 and 3.5–3.9, and remains marked Beta after twelve years. - **Runs where scapy cannot — including in the browser.** Verified - under a real Pyodide runtime in CI: scapy fails to import at all - under Pyodide (`from fcntl import ioctl` is unconditional in - `scapy/arch/`), where netprotocols, dpkt and pypacker all import - cleanly. + under a real Pyodide runtime in CI: netprotocols imports and decodes + the full corpus cleanly there, while scapy fails to import at all + (`from fcntl import ioctl` is unconditional in `scapy/arch/`). dpkt + was separately confirmed to import cleanly under the same missing + modules, though not inside this CI job; pypacker was not tested. - **The only one of these libraries a `match`/`case` statement dissects out of the box.** dpkt builds `__slots__` from a metaclass and generates no `__match_args__`; scapy routes fields through @@ -394,9 +396,12 @@ comparative-claims embargo lift that closed it out: | [2.1.0](https://github.com/EONRaider/NETProtocols/issues/105) | Typed accessors and pattern-matching ergonomics | | [2.2.0](https://github.com/EONRaider/NETProtocols/issues/106) | Universal round-trip properties, nightly fuzzing, a pcap reader | -Everything through 2.2.0 has landed on `master`; per the roadmap's own -release policy, only 2.0.0 was an actual PyPI release, and the -comparative claims above draw on the finished tree. New planned work +Everything through 2.2.0 has landed on `master`, plus a since-landed +decode-throughput fix +([#147](https://github.com/EONRaider/NETProtocols/issues/147), not yet +versioned); per the roadmap's own release policy, only 2.0.0 was an +actual PyPI release, and the comparative claims above draw on the +finished tree. New planned work will open fresh issues rather than restating a list here, so this section stays accurate without upkeep. The wave before this one — TCP and IPv4 option parsing, ICMP message bodies, NDP, IPv6 @@ -408,7 +413,8 @@ corpus fixture and `ipaddress` accessors — shipped in 1.3.0; see Development uses [uv](https://docs.astral.sh/uv/): `uv sync`, then `uv run pytest`, `uv run mypy`, and `uv run ruff check` — all three are -enforced by CI on Python 3.12–3.14. The test suite is anchored by a +enforced by CI; the test suite runs across Python 3.12–3.14, while +`mypy` and `ruff check` run on 3.12. The test suite is anchored by a 97-frame corpus of real captured traffic across 17 scenarios ([tests/fixtures/MANIFEST.md](tests/fixtures/MANIFEST.md)) plus property-based fuzzing of the decode path. See diff --git a/docs/CLAIMS.md b/docs/CLAIMS.md index 121f49e..330c6ab 100644 --- a/docs/CLAIMS.md +++ b/docs/CLAIMS.md @@ -367,19 +367,25 @@ on the measuring machine simply by being imported, unchanged from the original measurement. A codec that turns bytes into objects should never touch the host. -### 1.5 "~30× smaller wheel than scapy" +### 1.5 "~29× smaller wheel than scapy" **Status: VERIFIED** — re-measure at each release (Rule 4) -*Published 2026-09-04 (embargo lifted, #107).* - -85.6 KB wheel (`uv build`, `netprotocols-2.0.0-py3-none-any.whl`, -87,684 bytes) against scapy 2.7.0's 2.47 MB (2,590,982 bytes, current -PyPI `bdist_wheel`) — **29.6× smaller**, down from the ~46× recorded at -v1.3.0. 6,371 lines (`wc -l` over `src/`) against scapy's unchanged -246,813 — netprotocols grew from 3,732 lines as Tiers 2-4 added the -registry, walker, structured diagnostics and new protocol coverage; -scapy's line count and wheel size are both stable since the version on -file. Both move with releases — re-check before quoting. +*Published 2026-09-04 (embargo lifted, #107). Re-measured 2026-09-04: +the prior figure here was built from `netprotocols-2.0.0`, two releases +behind the tree every other number in this file now describes — this +was stale, not a real regression.* + +88.3 KB wheel (`uv build`, `netprotocols-2.2.0-py3-none-any.whl`, +90,390 bytes, deterministic across repeated builds) against scapy +2.7.0's 2.47 MB (2,590,982 bytes, current PyPI `bdist_wheel`) — +**28.7× smaller**, down from the ~46× recorded at v1.3.0 and the 29.6× +this section previously quoted from the 2.0.0-era build. 6,413 lines +(`wc -l` over `src/`) against scapy's unchanged 246,813 — netprotocols +grew from 3,732 lines as Tiers 2-4 added the registry, walker, +structured diagnostics and new protocol coverage, plus smaller +increments since; scapy's line count and wheel size are both stable +since the version on file. Both move with releases — re-check before +quoting. ### 1.6 "Decodes further into the stack than dpkt, on the same bytes" **Status: VERIFIED** @@ -474,11 +480,14 @@ Supporting facts, all re-verified against `secdev/scapy` @ `03f455c` - Scapy **does** ship `py.typed`, so the marker alone proves nothing. Its own mypy configuration (`.config/mypy/mypy_enabled.txt`) now - enables **107 files** (up from 88), of which **two of the - one-hundred-twenty-one under `scapy/layers/`** (`can.py`, `l2.py`; - up from "two of ninety-two" as the directory grew) — the dissectors - anyone actually touches are still unchecked. `Packet.__getattr__` - erases every field to `Any`. + enables **89 files** (up from 88; a prior pass at this figure counted + 107 by taking a raw `wc -l` of that file, which includes its blank + lines and comments — filtering those out the same way scapy's own + `.config/mypy/mypy_check.py` does gives 89, corrected 2026-09-04), of + which **two of the one-hundred-twenty-one under `scapy/layers/`** + (`can.py`, `l2.py`; up from "two of ninety-two" as the directory + grew) — the dissectors anyone actually touches are still unchecked. + `Packet.__getattr__` erases every field to `Any`. - dpkt still ships no `py.typed`; `types-dpkt` and `dpkt-stubs` still do not exist on PyPI (both 404 on `pypi.org/pypi//json`). Everything downstream becomes `Any`. @@ -548,12 +557,13 @@ out of a full TCP/IP stack rather than a standalone codec. We are MIT, ### 3.1 "Runs where scapy cannot — including the browser" **Status: VERIFIED** -*Published 2026-09-04 (embargo lifted, #107). Capability proved under -a real Pyodide runtime by #99; the comparative claim was held pending -the roadmap and is now stated.* +*Published 2026-09-04 (embargo lifted, #107). Corrected 2026-09-04: the +original text of this section credited the whole table below to #99's +real-Pyodide CI job, which overstated what that job actually runs — +see the note after the table.* Blocking exactly the modules Pyodide removes (`fcntl`, `termios`, -`resource`, `grp`, `pwd`) and re-importing: +`resource`, `grp`, `pwd`) under ordinary CPython and re-importing: ``` OK netprotocols @@ -569,9 +579,22 @@ Scapy's `from fcntl import ioctl` is **unconditional** in be *imported* under Pyodide — not "cannot sniff". It is also absent from Pyodide's built package set. -Was gated on testing it before selling it; #99 did exactly that -(`scripts/pyodide/check_in_pyodide.py`, CI's `pyodide` job), so the -claim is no longer conditional on anything. +This table is a **simulation** — a `sys.meta_path` blocklist run under +ordinary CPython, not Pyodide itself — and it is the only evidence +behind the dpkt/pypacker rows above. #99's CI `pyodide` job +(`scripts/pyodide/check_in_pyodide.py`) runs under a **real** Pyodide +WebAssembly runtime, but only imports and decodes the full corpus with +netprotocols there; it never attempts `import dpkt` or `import +pypacker` inside Pyodide at all. What #99 adds on top of the +simulation: it confirms, inside the real runtime, that the five +POSIX-only modules above are genuinely still absent (so the +simulation's precondition holds) and that netprotocols decodes the +entire real-capture corpus there raising nothing but `ProtocolError`. +So "scapy cannot import under Pyodide" is real-Pyodide-verified by +inference (scapy's unconditional `fcntl` import, confirmed absent in +CI) plus source citation; "dpkt and pypacker import cleanly under +Pyodide" is verified only by the CPython-side simulation above, not by +#99's CI job. ### 3.2 "Zero dependencies, no I/O, no sockets" **Status: VERIFIED — but do not lead with it**