Correct six stale/overstated README claims found by an independent audit - #153
Merged
Conversation
An audit ran every claim in README.md against the live repo -- executing every code example, running all three benchmark scripts, reproducing the exact mypy outputs, querying PyPI/GitHub live, cloning three external repos for license text. 63 of 71 claims held up exactly; 7 didn't. This fixes six of them (docs-only, no behavior change); the seventh -- the roadmap's claim that "only 2.0.0 was an actual PyPI release," contradicted by PyPI itself listing seven published versions including 2.2.0 -- is left for a maintainer decision, since it's a process question, not a wording fix. - Wheel size (85.6 KB / ~30x) was measured against the 2.0.0 wheel, two releases behind; rebuilt from HEAD it's 88.3 KB / ~29x. - The typing bullet's "107 of scapy's files" enable strict mypy was an unfiltered `wc -l` over scapy's enabled-files list; filtered the way scapy's own tooling does, it's 89. - The Pyodide bullet credited "dpkt and pypacker import cleanly" to the real-Pyodide CI job, which only ever imports netprotocols there -- the dpkt/pypacker rows come from a separate CPython-side simulation, not CI. - The CI-gating bullet's "the rest have no performance benchmark at all" was contradicted by its own citation (CLAIMS.md 1.7 documents scapy and PyTCP-net_proto do ship benchmark scripts, just not CI-wired); reworded to the claim the table actually supports. - The Contributing section overstated CI's Python matrix: only pytest runs 3.12-3.14, mypy/ruff check each run 3.12 only. - The Roadmap section didn't mention the unreleased #147 fix now on master beyond 2.2.0; added a clause naming it. docs/CLAIMS.md sections 1.5, 2.1 and 3.1 updated to match, each with a dated correction note explaining what was wrong and why, per this file's own Rule 4 discipline. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RzqEV89zgdGLb3QgnZy3vp
This was referenced Sep 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ran every claim in
README.mdagainst the live repo — executing every code example, running all three benchmark scripts, reproducing the exactmypyoutputs, querying PyPI/GitHub live, cloning three external repos for license text, and adversarially double-checking anything not immediately confirmed. 63 of 71 claims held up exactly; 7 didn't. This PR fixes six of them; the seventh is a process question left for a maintainer decision (see Notes).What's included
docs/CLAIMS.md§1.5): "85.6 KB / ~30× smaller than scapy" was measured against thenetprotocols-2.0.0wheel — two releases behind. Rebuilt fromHEAD(deterministic across repeated builds): 88.3 KB / ~29×.docs/CLAIMS.md§2.1): "enables strict checking on 107 of [scapy's] files" was an unfilteredwc -lover scapy's.config/mypy/mypy_enabled.txt, including blank lines and comments. Filtered the same way scapy's own.config/mypy/mypy_check.pydoes: 89 files (the "up from 88" phrasing already fits 89 far better than 107 ever did).docs/CLAIMS.md§3.1): credited "dpkt and pypacker import cleanly [under Pyodide]" to the real-Pyodide CI job. Readscripts/pyodide/check_in_pyodide.pyin full — it only ever imports netprotocols inside real Pyodide. The dpkt/pypacker rows in CLAIMS.md's table come from a separatesys.meta_path-blocklist simulation under ordinary CPython, not from CI. Reworded both docs to say so explicitly rather than implying the whole table is CI-verified.docs/CLAIMS.md§1.7 documents that scapy and PyTCP-net_proto do ship benchmark scripts, just never wired into CI. Reworded to the claim the table actually supports: none of the ten fails a build on a regression.pytest,mypy, andruff check— all three are enforced by CI on Python 3.12–3.14" overstates the matrix. Readci.yml: onlypytestruns 3.12-3.14;mypyandruff checkare each hardcoded to 3.12 only.master" didn't mention that a further, unreleased fix (Close the decode-throughput regression against dpkt #147) has since landed on top of 2.2.0. Added a clause naming it.CHANGELOG.md: new entry under the existing## [Unreleased]section.Not fixed — left for a maintainer decision
The Roadmap section also states "per the roadmap's own release policy, only 2.0.0 was an actual PyPI release." This is flatly contradicted by PyPI itself:
curl https://pypi.org/pypi/netprotocols/jsonshows seven published releases — 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.3.0, 2.0.0, and 2.2.0 (the last uploaded earlier in this same audit session). Issue #107's own text states the policy as "a PyPI release is cut only at a major version bump" — 2.2.0 is a minor bump. I deliberately left this sentence untouched: whether 2.2.0's PyPI publication was intentional (update the stated policy) or not (consider next steps for the 2.2.0 release) is a decision only the maintainer can make, and rewriting the prose first would paper over the actual question.Verification
uv run --frozen pytestpassesuv run --frozen ruff check .anduv run --frozen ruff format --check .are cleanuv run --frozen mypyis clean (strict)CHANGELOG.mdhas an entry under## [Unreleased]src/touched, no behavior change.Notes
Not tied to a tracking issue — this was a direct request to audit and then fix, not a new roadmap item.
🤖 Generated with Claude Code
https://claude.ai/code/session_01RzqEV89zgdGLb3QgnZy3vp
Generated by Claude Code