Skip to content

Correct six stale/overstated README claims found by an independent audit - #153

Merged
EONRaider merged 1 commit into
masterfrom
claude/new-session-ghy12u
Sep 4, 2026
Merged

Correct six stale/overstated README claims found by an independent audit#153
EONRaider merged 1 commit into
masterfrom
claude/new-session-ghy12u

Conversation

@EONRaider

Copy link
Copy Markdown
Owner

Summary

Ran every claim in README.md against the live repo — executing every code example, running all three benchmark scripts, reproducing the exact mypy outputs, querying PyPI/GitHub live, cloning three external repos for license text, and adversarially double-checking anything not immediately confirmed. 63 of 71 claims held up exactly; 7 didn't. This PR fixes six of them; the seventh is a process question left for a maintainer decision (see Notes).

What's included

  • Wheel size (README:73, docs/CLAIMS.md §1.5): "85.6 KB / ~30× smaller than scapy" was measured against the netprotocols-2.0.0 wheel — two releases behind. Rebuilt from HEAD (deterministic across repeated builds): 88.3 KB / ~29×.
  • Typing bullet (README:83-85, docs/CLAIMS.md §2.1): "enables strict checking on 107 of [scapy's] files" was an unfiltered wc -l over scapy's .config/mypy/mypy_enabled.txt, including blank lines and comments. Filtered the same way scapy's own .config/mypy/mypy_check.py does: 89 files (the "up from 88" phrasing already fits 89 far better than 107 ever did).
  • Pyodide/browser bullet (README:94-98, docs/CLAIMS.md §3.1): credited "dpkt and pypacker import cleanly [under Pyodide]" to the real-Pyodide CI job. Read scripts/pyodide/check_in_pyodide.py in full — it only ever imports netprotocols inside real Pyodide. The dpkt/pypacker rows in CLAIMS.md's table come from a separate sys.meta_path-blocklist simulation under ordinary CPython, not from CI. Reworded both docs to say so explicitly rather than implying the whole table is CI-verified.
  • CI-gating bullet (README:74-81): "the rest have no performance benchmark at all" was contradicted by the very citation it points to — docs/CLAIMS.md §1.7 documents that scapy and PyTCP-net_proto do ship benchmark scripts, just never wired into CI. Reworded to the claim the table actually supports: none of the ten fails a build on a regression.
  • Contributing section (README:409-411): "pytest, mypy, and ruff check — all three are enforced by CI on Python 3.12–3.14" overstates the matrix. Read ci.yml: only pytest runs 3.12-3.14; mypy and ruff check are each hardcoded to 3.12 only.
  • Roadmap section (README:397-399): "Everything through 2.2.0 has landed on master" didn't mention that a further, unreleased fix (Close the decode-throughput regression against dpkt #147) has since landed on top of 2.2.0. Added a clause naming it.
  • CHANGELOG.md: new entry under the existing ## [Unreleased] section.

Not fixed — left for a maintainer decision

The Roadmap section also states "per the roadmap's own release policy, only 2.0.0 was an actual PyPI release." This is flatly contradicted by PyPI itself: curl https://pypi.org/pypi/netprotocols/json shows seven published releases — 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.3.0, 2.0.0, and 2.2.0 (the last uploaded earlier in this same audit session). Issue #107's own text states the policy as "a PyPI release is cut only at a major version bump" — 2.2.0 is a minor bump. I deliberately left this sentence untouched: whether 2.2.0's PyPI publication was intentional (update the stated policy) or not (consider next steps for the 2.2.0 release) is a decision only the maintainer can make, and rewriting the prose first would paper over the actual question.

Verification

  • uv run --frozen pytest passes
  • uv run --frozen ruff check . and uv run --frozen ruff format --check . are clean
  • uv run --frozen mypy is clean (strict)
  • CHANGELOG.md has an entry under ## [Unreleased]
  • Docs-only change — no source under src/ touched, no behavior change.

Notes

Not tied to a tracking issue — this was a direct request to audit and then fix, not a new roadmap item.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RzqEV89zgdGLb3QgnZy3vp


Generated by Claude Code

An audit ran every claim in README.md against the live repo -- executing
every code example, running all three benchmark scripts, reproducing the
exact mypy outputs, querying PyPI/GitHub live, cloning three external repos
for license text. 63 of 71 claims held up exactly; 7 didn't. This fixes six
of them (docs-only, no behavior change); the seventh -- the roadmap's claim
that "only 2.0.0 was an actual PyPI release," contradicted by PyPI itself
listing seven published versions including 2.2.0 -- is left for a
maintainer decision, since it's a process question, not a wording fix.

- Wheel size (85.6 KB / ~30x) was measured against the 2.0.0 wheel, two
  releases behind; rebuilt from HEAD it's 88.3 KB / ~29x.
- The typing bullet's "107 of scapy's files" enable strict mypy was an
  unfiltered `wc -l` over scapy's enabled-files list; filtered the way
  scapy's own tooling does, it's 89.
- The Pyodide bullet credited "dpkt and pypacker import cleanly" to the
  real-Pyodide CI job, which only ever imports netprotocols there -- the
  dpkt/pypacker rows come from a separate CPython-side simulation, not CI.
- The CI-gating bullet's "the rest have no performance benchmark at all"
  was contradicted by its own citation (CLAIMS.md 1.7 documents scapy and
  PyTCP-net_proto do ship benchmark scripts, just not CI-wired); reworded
  to the claim the table actually supports.
- The Contributing section overstated CI's Python matrix: only pytest runs
  3.12-3.14, mypy/ruff check each run 3.12 only.
- The Roadmap section didn't mention the unreleased #147 fix now on
  master beyond 2.2.0; added a clause naming it.

docs/CLAIMS.md sections 1.5, 2.1 and 3.1 updated to match, each with a
dated correction note explaining what was wrong and why, per this file's
own Rule 4 discipline.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzqEV89zgdGLb3QgnZy3vp
@EONRaider
EONRaider merged commit aaf5913 into master Sep 4, 2026
8 checks passed
@EONRaider
EONRaider deleted the claude/new-session-ghy12u branch September 4, 2026 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants