Fix four claim-register drifts found by a fresh independent audit - #155
Merged
Conversation
Ran a structured, adversarially-verified audit against the live repo
(mypy/pytest execution, YAML config reading, external API spot-checks,
README<->CLAIMS.md cross-consistency) rather than trusting either
file's prose. 20 of 24 checks held up exactly; 4 were genuine drift,
not regressions:
- docs/CLAIMS.md 5.3: coverage's statement count (1,935) had drifted
from the live 2,091 as the codebase grew; the percentage looked
stable only because the miss count never moved. pyproject.toml's own
inline comment was still quoting an even older 1,441 - fixed too.
- docs/CLAIMS.md 5.7: test_walk.py's test count (79) undercounted the
live 85 - one test parametrized over the corpus's first 40 frames
now collects its full 40 instances because the corpus grew past that
slice, not because tests were added.
- docs/CLAIMS.md 1.6 overstated its own claim about README.md ("in the
same sentence"); the two figures are adjacent bullets there, not one
sentence.
- README's Pyodide bullet flatly said "pypacker was not tested",
contradicted by CLAIMS.md 3.1's own simulation table two paragraphs
away, which shows pypacker passing the identical CPython-side check
used for dpkt; also softened the lead-in so it no longer implies the
scapy-import failure was observed directly inside the real CI job
rather than by source citation plus that simulation. README's
CI-gating bullet lumped PyTCP-net_proto in with three libraries whose
benchmark code targets the compared library; CLAIMS.md's own citation
says PyTCP-net_proto's benchmark exercises its RX daemon instead -
reworded rather than flattened.
Also deep-linked the README -> CLAIMS.md §1.7 reference instead of
pointing at the file root.
Docs/comment-only change: uv run ruff check/format --check, mypy
--strict and the full pytest suite all pass unchanged.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011k6j6qzsttkU6RzZBgjjG7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Continuing the discipline issue #101 tracks: ran a fresh, structured, adversarially-verified audit of
docs/CLAIMS.mdandREADME.mdagainst the live repo (re-ran all three benchmark scripts, ranmypy/pytestmyself, read the CI YAML directly, cross-checked README against CLAIMS.md claim-by-claim, and spot-checked a few externally-sourced facts against PyPI/GitHub live) rather than trusting either file's own prose. 20 of 24 checks held up exactly — no drift found in this file's own comparative benchmark numbers. 4 checks surfaced genuine drift, none of them regressions, each independently re-confirmed by a second, adversarial verification pass before being fixed here.Issue #101 itself stays open by design (it's explicitly a "rolling" register with no fixed completion state) — this PR is this round's contribution to it, matching the pattern of the prior three merged PRs (#144, #152, #153/#154) that also fed it.
What's included
docs/CLAIMS.md§5.3: coverage's statement count (1,935) had drifted from the live 2,091 as the codebase grew — the coverage percentage looked stable only because the miss count never moved, masking the drift.pyproject.toml's own inline comment was still quoting an even older 1,441; fixed too.docs/CLAIMS.md§5.7:tests/test_walk.py's "79 tests" undercounted the live 85 — one test parametrized over the corpus's first 40 frames now collects its full 40 instances because the corpus has since grown past that slice, not because tests were added.docs/CLAIMS.md§1.6 overstated its own claim about README.md, saying the decode-depth figure travels with the throughput figure "in the same sentence" there — they're adjacent bullets, not one sentence.docs/CLAIMS.md§3.1's own simulation table two paragraphs away, which shows pypacker importing cleanly under the identical CPython-side check used for dpkt; reworded, and softened the lead-in so it no longer implies the scapy-import failure was observed directly inside the real Pyodide CI run rather than by source citation plus that simulation. README's CI-gating bullet also lumped PyTCP-net_proto in with three libraries whose benchmark code targets the library being compared;docs/CLAIMS.md§1.7's own citation says PyTCP-net_proto's benchmark exercises its RX daemon instead — reworded rather than flattened.docs/CLAIMS.md§1.7 reference to the actual section anchor instead of the bare file path.CHANGELOG.mdentry under## [Unreleased].Verification
uv run ruff checkanduv run ruff format --checkare cleanuv run mypyis clean (strict)uv run pytestpasses locally (1088 passed)CHANGELOG.mdhas an entry under## [Unreleased]benchmark.py --compare --depth,benchmark_encode.py,benchmark_import.py) and the CI regression check (--check --threshold 15, -4.8% against baseline) as part of the audit — all within stated noise bands, no fixes needed there.Notes
Not tied to a fixed-scope tracking issue beyond #101 itself, which is explicitly rolling and isn't meant to close.
🤖 Generated with Claude Code
https://claude.ai/code/session_011k6j6qzsttkU6RzZBgjjG7
Generated by Claude Code