Skip to content

docs(contributing): record the get-override rulings on the conventions page (#918) - #948

Merged
JarryShaw merged 2 commits into
mainfrom
docs/918-harvest-get-override-rulings
Sep 30, 2026
Merged

JarryShaw merged 2 commits into
mainfrom
docs/918-harvest-get-override-rulings

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

What is the purpose of your pull request?

  • fix — corrects a defect
  • feat — adds a feature
  • perf — changes performance, not behaviour
  • refactor — changes neither behaviour nor performance
  • test — tests only
  • docs — documentation only
  • ci — workflows or build tooling
  • chore — anything else

Description of your pull request and other information

Part 1 of #918. Part 2 merged as d7489c61c and created the page-per-anchor directory; this writes the get-override rulings into it.

Three rulings harvested onto registry-protocol, each with its quote and the reasoning rather than the outcome — the code already encodes the outcome:

# ruling
#933 "Oh wait. I meant, they should follow house convention and not to be loud." — the reversal is the ruling; a loud BaseError sets sys.tracebacklimit process-wide (#362), so uniformity beat the per-class argument
#935 "I lean on 1." — widen and delegate; a type: ignore[override] plus a docstring is not an answer to a contract the class advertises and breaks
#940 "I prefer (2) directly." — delete an override that only reimplements the base, rather than widening it

The trap behind #940 is recorded because it is what a future override will hit: the base branches on isinstance(key, str) and treats the rest as a value, while the deleted overrides branched on isinstance(key, int) and fell to the name path — so get(None) was a quiet EnumKeyError on two classes and a loud EnumValueError on the other five, with no prose anywhere saying so.

Also corrects an audit row #940 falsified: it claimed two of that group define their own get; both were deleted, so the answer is now none. pcapkit/protocols/internet/mh.py contains zero def get(. That row rendered fine and failed nothing.

The @classmethod item is deliberately not presented as a ruling. It is a language constraint — zero-argument super() in a @staticmethod has nothing to bind — with #913 setting the shape and #908 following it. Worth flagging because the first draft of this change did quote it as a maintainer ruling, attributing a sentence that is actually from analysis prose on #908; every comment on this repo carries the same author, so that distinction has to be made by reading, not by author field.

Not included, and the reason #918 cannot close on these four pages alone: three settled rulings fit none of them — the all extra being core addons only (#910), one commit per changelog entry, and the breaking label's meaning. They are process rather than code conventions. Asked separately on #918 rather than inventing a fifth page.

Test. GetOverrideContractTests — 4 tests, 10 subtests — pins the page's wording and the tree it describes: vars(Method)['get'] is a classmethod while Command/OptionType are staticmethods whose source has no super(); a name miss leaves sys.tracebacklimit unset; 'get' not in vars(cls) for both mh classes; get(None) and get(1.5) raise EnumValueError.

tests/project: 210 passed, 1 skipped, 572 subtests (206 + 4 before). Each assertion was shown to fail with its claim removed from the page.

UNVERIFIED: Sphinx rendering — no make html run. The section adds no new :ref:/:doc: targets, only roles already used on the same page.

@JarryShaw JarryShaw added docs Pull requests that change documentation only (docs: subject prefix) review: pending No verdict for the current head - never reviewed, or the head moved since the last one test Pull requests that add or correct tests (test: subject prefix) labels Sep 30, 2026
@JarryShaw
JarryShaw force-pushed the docs/918-harvest-get-override-rulings branch from 2b98b32 to d72a029 Compare September 30, 2026 13:37
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at 2b98b3251 — sonnet cross-review (author opus). It found a false claim I had carried onto the page, and it was right. Verified myself before accepting:

The page said a delegating @staticmethod raises RuntimeError: super(): no arguments. It does not. Zero-argument super() binds the enclosing function's first positional parameter, whatever its name — so in @staticmethod def get(key, ...) it binds the lookup key:

staticmethod WITH a param   -> TypeError: super(type, obj): obj (instance of str)
                               is not an instance or subtype of type (SubWithParam).
staticmethod with NO params -> RuntimeError: super(): no arguments
first arg IS an instance    -> base:<__main__.SubInst object ...>     # no error at all

That third row is mine, beyond what the review reported, and it is the worst of the three: a @staticmethod override cannot even be relied on to fail loudly. RuntimeError is reachable only with zero parameters, which no real get override has.

Provenance, since it matters: the wrong claim originated as unverified analysis prose on #908 and this PR imported it unchanged. The review's sharpest point is that my own test could not catch it — it asserted the page contained the error string, never executed the shape. So the test now builds the three class shapes and runs them, and the page states all three outcomes.

Also fixed, its second finding: test_the_base_raises_a_name_miss_quietly guarded its behavioural check with if not had:, so a prior test setting sys.tracebacklimit turned it into a prose-only check that still reported pass. Now unconditional, clearing first and restoring in addCleanup. Verified by running it with sys.tracebacklimit = 999 pre-set: the check executes and the value is restored to 999.

Two notes on the review itself. Its first line carried both verdicts, which breaks the one-line contract these reviews run under — the body was unambiguous, but the verdict had to be read out of the prose. And its Sphinx build resolved pcapkit to the main checkout rather than its worktree, which it flagged honestly and correctly as UNVERIFIED.

Everything else it checked held: three quotes verbatim, the mh.py zero-get count, the base's isinstance(key, str) asymmetry against #940's diff, the corrected audit row, and three of four tests failing under counterfactual.

New revision d72a029e2. tests/project: 210 passed, 1 skipped, 572 subtests.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: pending No verdict for the current head - never reviewed, or the head moved since the last one labels Sep 30, 2026
…s page (#918)

Part 1 of #918: the rulings that settled what a ``get`` override owes
``EnumLookup.get`` lived only in issue and PR comments, where nothing keeps them
findable. Part 2 split ``conventions.rst`` into a page per anchor so they have
somewhere to land; this writes them onto the registry-protocol page.

- Add *What a ``get`` Override May and May Not Do*, carrying the three rulings
  (#933 on ``quiet=True``, #935 on honouring an advertised signature rather
  than suppressing it, #940 on deleting an override that only reimplements the
  base) plus the ``@classmethod`` requirement, which is a language constraint
  rather than a ruling. Each is recorded with the reasoning, not the outcome:
  the code already encodes the outcome, and the reasoning is what is expensive
  to rediscover.
- Paraphrase the rulings rather than block-quoting the maintainer, per his
  request on #918, and pin the page's *claim* in the tests instead of his
  wording. Quoting him verbatim had made an off-hand reply load-bearing in CI:
  an assertion required the literal ``I prefer (2) directly.`` to appear on a
  docs page. The quotes that predate this change are tracked in #949.
- State the ``@classmethod`` constraint from measurement, on every supported
  version. Zero-argument ``super()`` binds the enclosing function's **first
  positional parameter**, whatever its name, so in ``@staticmethod def get(key,
  ...)`` it binds the lookup key and raises ``TypeError``. CPython words that
  error differently either side of 3.13 -- ``obj must be an instance or subtype
  of type`` on 3.10-3.12, ``obj (instance of str) is not an instance or subtype
  of type (Cls)`` on 3.13+ -- so the page shows both and the test asserts only
  ``instance or subtype of type``, which is common to them. Pinning either full
  sentence passes on two of the five required Compat legs and fails the other
  three, invisibly, since this venv is 3.14.
- Record the two corollaries that make the obvious summary wrong in both
  directions: ``RuntimeError: super(): no arguments`` needs a function with no
  parameters at all, which no real override has; and an instance first argument
  makes the delegation **succeed silently**, so a ``@staticmethod`` override
  cannot be relied on to fail loudly.
- Correct the audit row #940 falsified. It claimed **two** of the
  ``mh.py``/``ngap.py`` group define a ``get`` of their own; #940 deleted both,
  so the count is now none. That row rendered fine and failed nothing while
  naming methods that no longer exist.
- Extend ``tests/project/test_conventions_doc_claims.py`` with
  ``GetOverrideContractTests``: four tests, ten subtests. The ``@classmethod``
  test **executes** all three ``super()`` outcomes rather than grepping the page
  for an error string -- a prose-only assertion is what let a wrong exception
  stand. The ``sys.tracebacklimit`` test runs its behavioural check
  unconditionally instead of skipping it when the attribute is already set,
  which had made it degrade to a doc-text check that still reported pass.

tests/project: 210 passed, 1 skipped, 572 subtests. Each new assertion was shown
to fail with its claim removed; the tracebacklimit check verified to run, fail
when the base raises loudly, and restore prior state with ``sys.tracebacklimit``
pre-set; and the ``TypeError`` wording measured on 3.10 through 3.14.
@JarryShaw
JarryShaw force-pushed the docs/918-harvest-get-override-rulings branch from d72a029 to 726dd36 Compare September 30, 2026 13:51
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at d72a029e2 — sonnet round 2, and it found a defect my local run structurally could not see.

The TypeError message I pinned is not stable across supported Python versions. CPython reworded it at 3.13. Measured on this machine, all five required Compat versions:

3.10 -> super(type, obj): obj must be an instance or subtype of type
3.11 -> super(type, obj): obj must be an instance or subtype of type
3.12 -> super(type, obj): obj must be an instance or subtype of type
3.13 -> super(type, obj): obj (instance of str) is not an instance or subtype of type (Sub).
3.14 -> super(type, obj): obj (instance of str) is not an instance or subtype of type (Sub).

My assertion required is not an instance or subtype of type, which appears on neither 3.10, 3.11 nor 3.12 — three of the five required Compat legs would have failed. Invisible locally, because the venv is 3.14.7 and so is the only version where it passes. That is the more useful half of the finding: a green local run on one interpreter is not evidence about a five-version gate.

Fixed by asserting only instance or subtype of type, common to both forms, and by showing both wordings on the page — a reader on 3.11 could not otherwise match what they see. Added a second assertion pinning the older wording so the page cannot quietly drop it.

It also confirmed, by re-deriving from real git history rather than from the page, the two paragraphs I flagged as most likely to be resting on reasoning: #935's --warn-unused-ignores claim reproduces at exactly mh.py:645:5 and :782:5 with the suppression stripped, and #940's nine-case matrix differs in 0 of 9 for both classes. Its addCleanup-ordering and pytest-xdist analysis of the tracebacklimit change is right — cleanup is registered before the del, and xdist isolates by process.

Two honest gaps it reported: it did not break each new executable assertion to watch it fail, and it did not rebuild the docs. I have covered the first myself.

Separately, and independent of the review: the maintainer asked not to be quoted verbatim, so this revision paraphrases the four rulings and pins the page's claims rather than his sentences. That mattered functionally, not just stylistically — one assertion required the literal I prefer (2) directly. to appear on a docs page, making a one-line reply a build dependency. Pre-existing quotes elsewhere are tracked in #949.

New revision 726dd364c. tests/project: 210 passed, 1 skipped, 572 subtests.

@JarryShaw JarryShaw added review: pending No verdict for the current head - never reviewed, or the head moved since the last one and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Sep 30, 2026
@JarryShaw
JarryShaw force-pushed the docs/918-harvest-get-override-rulings branch from 9a21758 to a99db83 Compare September 30, 2026 14:23
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at 9a21758da — sonnet round 3. Two defects, both verified before acting, and the second was mine.

1. "Labels are applied by hand, with dependabot the single exception" was false. The issue templates are a second automated path:

$ grep -H '^labels:' .github/ISSUE_TEMPLATE/*.md
.github/ISSUE_TEMPLATE/bug_report.md:labels: bug
.github/ISSUE_TEMPLATE/feature_request.md:labels: enhancement

Front-matter labels: applies before anyone reads the issue, so bug and enhancement on a template-opened issue are defaults rather than assessments — and those are two of the issue-kind labels the page lists two sections later. The page now names both paths and rules out the two near-misses explicitly: .github/release.yml only reads labels to bucket release notes, and PCAPKIT_CONDA_LABEL is a conda channel label.

The review's sharper point is that my test could not see it — it asserted labels was absent from dependabot.yml and never looked at ISSUE_TEMPLATE/, so it passed against a false claim. It now sweeps ISSUE_TEMPLATE/*.md and requires every template that sets a label to be named on the page.

2. "Three flat kind-runs" was wrong, and it was my phrasing. I wrote it in a #918 comment without measuring and it was carried onto the page verbatim:

$ grep -oE '^\* \*\*[A-Za-z]+\*\*' docs/source/changelog/1.5.0.rst | uniq -c | wc -l
26

Twenty-six runs, not three — blocked for roughly the first 40% of the file, interleaved after it. The ruling is unaffected, but the characterisation was: the restructure is a regrouping of scattered entries, not a transposition of three tidy blocks. Nothing pinned it in either direction, so there is a new test.

That test was vacuous on its first attempt, which I caught by running the counterfactual: it asserted '26' appeared on the page, and the page's own wc -l output satisfied that — so it passed with "three flat runs" back in the prose. It now asserts the figure in the sentence plus assertNotIn('three flat').

Everything else it checked held: the all extra against pyproject.toml via tomllib, the two distinct exclusion reasons, lint.yml:183's tracked import-error count, the breaking census re-derived independently, every label in gh label list, release/const having no template tickbox, the five-tuple ANCHORS with no stale four-counting, and sentinels.py/test_sentinel_exports_unit.py reading whole files rather than anchor-slicing. It also built the docs with the root confirmed in this worktree.

New revision a99db835d. tests/project: 218 passed, 1 skipped, 606 subtests.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment review: pending No verdict for the current head - never reviewed, or the head moved since the last one and removed review: pending No verdict for the current head - never reviewed, or the head moved since the last one review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Sep 30, 2026
@JarryShaw
JarryShaw force-pushed the docs/918-harvest-get-override-rulings branch from a99db83 to 29c178a Compare September 30, 2026 14:46
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at a99db835d — haiku round 4. Six defects, all verified before acting, and the first is the worst kind: a self-contradiction inside one test.

1. The round-1 RuntimeError defect survived in the test's own prose. I corrected the page and left the failure message wrong:

:766  'zero-argument super() inside a staticmethod raises RuntimeError'
:813  'the no-parameter case is the one that raises RuntimeError;
       the page must not attribute it to a get(key) override'

Line 766 does exactly what line 813 forbids, 45 lines apart, in the same test. The class docstring's "has nothing to bind" was wrong the same way — super() binds key. Both now say TypeError.

2. "They fall into five groups" was false. 29 labels exist; the five groups plus breaking account for 24. wontfix, invalid and help wanted are in live use, duplicate and good first issue are not. The page now says the five groups are not the whole set, gives the count as a command, and records the local ruling that a closed-as-unnecessary issue takes invalid rather than bug.

3. "Seven pre-0.15 distribution pull requests" was false. Only three are:

3  test/rc/abc                  breaking,refactor
4  Regular update [test/rc/abc] feat,breaking,refactor
6  Regular update               breaking,refactor
7  Reconstruction accomplished  feat,breaking
25 New distribution [0.14.0]    fix,ci,breaking,release
26 New distribution [0.14.1]    ci,breaking,release
28 New distribution [0.14.2]    ci,breaking,release

Seven, #3–#28 and pre-0.15 all hold; "distribution" did not.

4. Five claims on the page were unpinned, which it proved by falsifying all five at once — 40%→95%, #3 to #28→#3 to #900, five groups→nine groups, and more — and getting 218 passed. That is the real finding behind the other three.

5. A docstring contradicting its own assertion — it said the run count was "pinned as a floor rather than an exact figure" while the next line required the exact figure in the prose. The docstring now says why both: the floor makes "three flat runs" unsayable, and the exact figure is meant to go red when #657 moves it, as the signal to re-measure the sentence.

6. assertTrue(issubclass(ProtocolError, Exception)) was tautological — true of every exception class. Replaced with checks that it comes from pcapkit.utilities.exceptions and descends from the library base, which is what the page's worked example actually claims.

It confirmed the rest by re-derivation, including the 26 runs and — usefully — that the "roughly first 40%" holds only by line position (run 6 starts at line 509 of 1275, 39.9%); by entry index it is 53.8%.

Two minor points recorded rather than fixed: github_actions is historically true but cannot recur, since dependabot.yml has no github-actions ecosystem; and "nothing else automates a label" is an unpinned universal negative, verified by hand across all 15 files in .github/ but not by a test.

New revision 29c178a52. tests/project: 218 passed, 1 skipped, 606 subtests.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment review: pending No verdict for the current head - never reviewed, or the head moved since the last one and removed review: pending No verdict for the current head - never reviewed, or the head moved since the last one review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Sep 30, 2026
@JarryShaw
JarryShaw force-pushed the docs/918-harvest-get-override-rulings branch from 29c178a to 20f895c Compare September 30, 2026 15:11
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at 29c178a52 — haiku round 5. Its technique was the finding: it corrupted 46 claims across both pages simultaneously and the suite stayed green — 218 passed, byte-identical to the clean baseline. That is the honest measure of how much of that prose was decorative.

The blocking defect is a count I wrote. The page said get(None) raised quietly "on two classes and a loud EnumValueError on the other five", implying seven. Measured — every EnumLookup subclass those two modules actually define:

mh    FastBindingAcknowledgmentStatus, IPv6AddressPrefixCode,
      LocalizedRoutingStatus, LMAAddressCode
ngap  PDUKind, Criticality
TOTAL 6

ProcedureCode/ProtocolIE are re-exports from pcapkit.const.ngap.*, not helpers. So it is two + four of six, and the page carried three incompatible figures for one population — the audit table's five, my prose's seven, and the real six.

Worse, the test was named test_a_non_string_key_raises_a_value_miss_on_all_seven and exercised three. Its own name asserted a number its body could not see. It now derives the population by walking both modules for EnumLookup subclasses whose __module__ is that module, asserts the count is six and the exact set, and loops over what it derived — so the name cannot disagree with the body again. Subtests rose 606 → 626.

duplicate is in live use — the page said it and good first issue were not. PR #8 carries it, applied on 2026-09-23, thirty-two minutes after the #275 invalid swap the next paragraph cites as live practice. Only good first issue is genuinely unused (0/0). Corrected.

The invalid ruling is now attributed. The page called it a local ruling while naming no issue, against its own preamble promising the issue is named. Now cites #275 (where bug came off and invalid went on in the same second) and #707 as the worked example.

New test pinning what can be pinned: the changelog entry count, every module the page's ls -d listing names against pcapkit/'s real packages, and every file path it cites. Verified it bites — corrupting 80 entries → 81 entries fails it.

And the limit stated rather than implied. Every claim whose ground truth is a GitHub query — the label count, which defaults are in live use, the breaking census, the issue numbers — cannot be pinned here, because CI has no network. Inverting the live-use claim still passes, measured. That is why the page gives the command alongside the figure: the command is the pin, run by a reader rather than by CI. Recorded in the test's docstring.

Two of its findings I am not acting on, with reasons: github_actions is historically true (PR #184) though dependabot.yml can no longer produce it, and "the five supported versions" means the five CI-tested versions while requires-python declares >=3.6 — both are hedged correctly on the page.

New revision 20f895c36. tests/project: 219 passed, 1 skipped, 626 subtests.

@JarryShaw
JarryShaw force-pushed the docs/918-harvest-get-override-rulings branch from 20f895c to e77c640 Compare September 30, 2026 15:44
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at 20f895c36 — sonnet round 6, plus a CI failure I have to report separately because it is mine and the review did not cause it.

The red mark first: Python 3.10 and Required checks passed both FAILED, and they are one defect, not two.

tests/project/test_conventions_doc_claims.py:1018  ModuleNotFoundError: No module named 'tomllib'
tests/project/test_conventions_doc_claims.py:1048  ModuleNotFoundError: No module named 'tomllib'

tomllib is 3.11+, and 3.10 is in the matrix. Required checks passed failed because Python 3.10 did. tomli, the usual backport, is not a dependency here either — grep -n tomli pyproject.toml finds nothing — so there was nothing to fall back to. Replaced with a small reader for the [project.optional-dependencies] block, which is uniform enough that a regex is honest, and its docstring says exactly what it handles and what it does not. Verified it parses identically under python3.10 and 3.14.

This is the second time a local green run on 3.14 hid a 3.10 failure on this PR — round 2 was the TypeError message wording. Same lesson, not learned the first time: a green local run on one interpreter is not evidence about a five-version gate.

The review's headline: 72 of 77 corruptions survived. Its classification is the useful part — (a) genuinely unpinnable offline versus (b) pinnable but unpinned — and it found the largest (b) class: ~30 link-number mismatches, where the displayed #NNN and the number in its own URL disagree. Invisible to a reader and unwarnable by Sphinx, since both halves are well-formed. Now closed by one test over every page; verified it fails when issues/918 is repointed to issues/919.

A live defect, not a corruption: registry-protocol.rst:461 still read "The 5 mh and ngap helper enumerations" against the measured six at :277. So round 5's three incompatible figures had become two, not one. Corrected — and it is pre-existing on main, not introduced here.

And a vacuous assertion of exactly the shape I had already guarded against elsewhere. It changed the page's labels: bug to labels: defect and test_the_page_says_the_labels_are_set_by_hand still passed, because the bare word bug occurs elsewhere on the page. Now anchored to ``labels: bug``; verified it fails on that same corruption.

What I am not doing, with the reason. The remaining (b) survivors are the Audit, per Class population figures — 127 EnumRegistry subclasses, 117 int-valued, 10 aenum.StrEnum, 151 total, the R1_Counter collision. Pinning them means extending the runtime enumeration walk, which is a substantial piece of work on a table this PR does not touch. It belongs with #949, which already owns cleanup on these pages, rather than growing this PR further — and I would rather say so than quietly leave it looking covered.

New revision e77c640ea. tests/project: 220 passed, 1 skipped, 626 subtests.

@JarryShaw
JarryShaw force-pushed the docs/918-harvest-get-override-rulings branch from e77c640 to ac08400 Compare September 30, 2026 16:54
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at e77c640ea — sonnet readiness pass (Fable died on a 429 after clearing items 1–5; third such death today, so the substitution is noted rather than silent).

It found a real defect in the newest code, which is exactly where I pointed it. _optional_dependencies() matched arrays with a non-greedy bracket pattern, so it stopped at the first literal ]. Verified against a tomllib oracle:

MISMATCH dev     oracle: [... 'requests[socks]', 'beautifulsoup4[html5lib]']
                 mine  : [...]                              # last two dropped
MISMATCH test    oracle: [... 'requests', 'beautifulsoup4', 'isort']
                 mine  : [... '9 skipped']                  # invented from comment prose
MISMATCH vendor  oracle: ['requests[socks]', 'beautifulsoup4[html5lib]', 'pycrate']
                 mine  : []                                 # emptied entirely
3 of 14 extras mis-parsed

vendor emptied on the ] inside "requests[socks]"; test truncated at a ] in a comment and then picked up a quoted example, inventing a requirement called 9 skipped.

And my docstring claimed it "handles exactly that shape and nothing else in TOML" — which is precisely what it did not do. That claim was the real defect; the parser was only its consequence.

The review was fair that no shipped assertion reads those three keys, so nothing gave a wrong verdict today. I am fixing rather than documenting around it anyway, because dev and vendor are both described in the page's own prose — the next assertion to check either would have got wrong data with nothing raised, which is the "renders fine and fails nothing" failure the previous six rounds were raised to close.

Fixed by stripping # comments (skipping quoted spans, since a # inside a requirement string is not a comment) and tracking bracket depth. Now zero mismatches across all 14, and verified byte-identical under python3.10 and 3.14 — the reader's whole reason for existing.

New test: test_the_extras_reader_agrees_with_tomllib. The reader was an unverified dependency of every extras claim on the page, which is how this got in. It now compares against a real parser key for key wherever one exists, and skips on 3.10 rather than pretending — honest, since 3.10 is the one version it was written for and has no oracle. Verified it bites: restoring the naive pattern fails it, naming dev, test and vendor individually.

Everything else it checked held: the link-number test examines 44 of 44 links with none silently skipped; the anchored labels: bug assertion fails on the exact corruption round 6 found; the audit table's "The 6" matches the measured six and the "four of the six" sentence agrees; entry count 80, kind-runs 26, labels 29 all match.

New revision ac0840081. tests/project: 221 passed, 1 skipped, 640 subtests.

@JarryShaw
JarryShaw force-pushed the docs/918-harvest-get-override-rulings branch from ac08400 to 192f281 Compare September 30, 2026 17:13
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at ac0840081 — haiku round 8. One factual defect, and it is round 4's shape recurring in the same file.

tests/project/test_conventions_doc_claims.py:932 said the deleted overrides made it "a KeyError on two of the seven and a ValueError on the other five", and :971 said #940's deletion "converged the seven". Twenty lines below, the same method asserts len(helpers) == 6 and enumerates six names — and the comment between them explicitly disavows the figure. The page says "the other four of the six". So the file contradicted the page, the tree, and its own assertion, and nothing failed. Corrected to six and four at both sites.

Two unpinned claims closed, and both are recurrences of shapes I have already been burned by:

  1. registry-protocol.rst's "The 6" was unpinned. That is the exact figure round 6 found reading "The 5" against the measured six — the correction landed with nothing guarding it, so it could have regressed precisely as it arrived. Now asserted from len(helpers).
  2. The quiet=True ruling could be inverted without failing anything. assertIn('``quiet=True``', flat) was satisfied by a later mention — quiet=True occurs four times on the page — so changing the headline to quiet=False passed. Third instance of this shape, after labels: bug and the bare 26, so the rule is now written into the code: never assert a token that appears more than once on the page it is meant to pin.

Verified both bite: inverting the ruling and downgrading "The 6" each fail now, and the page is byte-restored after.

On the extras reader it declined to call defects, and I agree with the judgement. Five shapes break it — single-quoted TOML strings, mixed quoting, an escaped \" that overruns into the next key, triple-quoted strings, a column-0 [ inside an array — all outside the docstring's stated scope. Each produces a missing key or a wrong list, and test_the_extras_reader_agrees_with_tomllib compares the full key set and all 14 values, so any of them fails loudly on four of the five CI legs the moment pyproject.toml grows one. It confirmed MATCH (14 keys) against tomllib on all five interpreters — 3.10 through 3.14 — which is stronger than the two I had checked. The likeliest trigger is a marker containing a double quote, which forces a TOML literal string; noted rather than pre-solved.

It also confirmed the 3.10 skip is a real skip, that the test compares every key rather than the page-mentioned ones, and that a reader returning an empty dict, or only the four page-mentioned keys, or all 14 with one requirement silently dropped from dev, each fail it.

Its category (a) accounting is the useful part of the 63 survivors: ~40 are genuinely unpinnable offline because their ground truth is a GitHub query and CI has no network, plus a handful whose ground truth is deleted code — historical measurements recoverable only from git. Those are not defects and I am not manufacturing pins for them.

New revision 192f28155. tests/project: 221 passed, 1 skipped, 640 subtests.

@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at 192f28155 — sonnet round 9, which found the fourth instance of the duplicate-token shape. That is what it was briefed to hunt, and the census it produced is the more useful half.

assertIn('instance or subtype of type', flat) targeted a substring occurring three times on the page: the quoted 3.13+ TypeError, the quoted 3.10–3.12 one, and the sentence explaining what they share. So it pinned neither wording — corrupting the 3.13+ quote to something false still passed, which it proved by doing it and restoring by md5. The narrower must be an instance or subtype of type correctly pinned the older wording all along; the newer one had nothing.

Fixed by asserting is not an instance or subtype of type, unique to the 3.13+ form. Verified: corrupting that quote now fails, and the page's md5 is restored after.

Its full census of all 35 assertIn calls is the part worth keeping, because it turns the shape from a recurring surprise into a closed question:

  • 1 vulnerable — the one above.
  • 2 benign duplicates — feat and refactor each occur twice, but those are presence-only claims with no direction to invert.
  • Everything else unique, including all three earlier fixes holding: the labels: bug anchor, the dynamic **26** separate runs phrase (while the bare 26 still occurs three times), and round 8's quiet=True sentence anchor (while the bare token occurs four times).

So the rule is now stated once in the code rather than rediscovered: an assertIn whose needle appears more than once on the page pins nothing. Count occurrences before asserting.

It also re-derived rather than trusted round 8's _optional_dependencies() result: MATCH, 14 keys, 0 diffs against tomllib on 3.11, 3.12, 3.13 and 3.14 individually, plus the 3.10 run returning the same 14 keys without a parser available. And it checked every docstring in both new classes for round 8's contradiction shape — arguments-differ genuinely 4 times on read/__post_init__, the helper population genuinely six, Method.get a classmethod, Command.get/OptionType.get staticmethods with no super() — finding no second instance.

One thing it declined to verify, correctly: the docstring's reference to "a cross-review corrupted 46 claims" is historical narrative with no local ground truth, and the test says so itself.

New revision ef7fbe971. tests/project: 221 passed, 1 skipped, 640 subtests.

@JarryShaw
JarryShaw force-pushed the docs/918-harvest-get-override-rulings branch from 192f281 to ef7fbe9 Compare September 30, 2026 17:29
Three settled rulings fit none of the four code-convention pages, because they
govern the repository rather than the library. The owner ruled on #918 that they
get a fifth page rather than staying in their threads.

- Add ``docs/source/contributing/conventions/process.rst`` (``.. _process:``),
  covering what the ``all`` extra carries (#910), what a changelog entry is, and
  what the issue and pull request labels mean. Paraphrased throughout rather
  than quoting the owner, on his instruction on the same issue.
- Record the changelog grouping as ruled: a section per top-level module with
  ``Added``/``Changed``/``Fixed`` nested inside each. The restructure belongs to
  #657, which owns the file and merges last. The one case the rule does not
  settle, an entry spanning modules, is flagged rather than decided.
- Describe the file's **current** shape from measurement. An earlier draft called
  it "three flat kind-runs", carried from a comment nobody had checked; the 80
  entries actually carry the three kind labels in **26** runs.
- Document the label scheme the owner asked for alongside ``breaking``, and name
  **both** automated paths: dependabot, and the issue templates, which apply a
  label from front matter before anyone reads the issue. An earlier draft called
  dependabot the single exception.
- Correct two ``breaking`` claims measurement contradicts. It **is** applied below
  #350 -- carriers ``#3``-``#28``, nothing between ``#28`` and ``#350``, issues
  from ``#775``. And no ruling defines the label, so the page states the
  behavioural test rather than attributing a phrasing to anyone.
- Correct the audit table's population to **six** -- the ``mh``/``ngap`` helpers
  ``EnumLookup`` subclasses those modules define, ``ProcedureCode``/``ProtocolIE``
  being re-exports. The page had carried three figures for one population.

Tests, and the reason each exists rather than the shape of it:

- ``ProcessConventionTests`` pins the page's claims against the tree. Two were
  rewritten because they could pass for the wrong reason: one read only
  ``dependabot.yml`` and never ``ISSUE_TEMPLATE/``, and one asserted the bare
  string ``26``, which the page's own ``wc -l`` output satisfied.
- ``test_a_non_string_key_raises_a_value_miss_on_every_helper`` **derives** the
  population instead of listing it. Its predecessor was named ``..._on_all_seven``
  and exercised three.
- ``test_every_issue_link_number_matches_its_own_url`` closes the largest class of
  unpinned claim: a cross-review corrupted ~30 link numbers and not one was
  caught, because nothing compared the displayed ``#NNN`` against the number in
  its own URL. Invisible to a reader, unwarnable by Sphinx.
- ``_optional_dependencies`` replaces ``tomllib``, which is 3.11+ and broke the
  ``Python 3.10`` leg while passing locally on 3.14 -- ``tomli`` is not a
  dependency either, so there was nothing to fall back to. It now strips comments
  and tracks bracket **depth**: a version matching to the first ``]`` mis-parsed
  **3 of the 14** extras silently, emptying ``vendor`` on the ``]`` inside
  ``"requests[socks]"``, truncating ``dev``, and **inventing a requirement called**
  ``9 skipped`` **out of comment prose**. No shipped assertion read those keys, but
  ``dev`` and ``vendor`` are both described in the page's prose, so the next
  assertion to check either would have got wrong data with nothing raised.
  ``test_the_extras_reader_agrees_with_tomllib`` now compares the two key for key
  wherever a real parser exists, and skips on 3.10 rather than pretending to.
- The page states plainly what cannot be pinned here: any claim whose ground truth
  is a GitHub query, since CI has no network. The command beside the figure is the
  pin, run by a reader rather than by CI.

tests/project: 221 passed, 1 skipped, 640 subtests. The extras reader verified
byte-identical under python3.10 and 3.14, and every new assertion shown to fail
with its claim removed or its defect reintroduced.
@JarryShaw
JarryShaw force-pushed the docs/918-harvest-get-override-rulings branch from ef7fbe9 to e4340e8 Compare September 30, 2026 17:50
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at ef7fbe971 — haiku round 10. Three defects, all in code this PR adds, and the two censuses it was asked for.

1. The page attributed a hand-applied label to dependabot — inverting the section's own point. Verified:

$ grep -rn 'package-ecosystem' .github/
.github/dependabot.yml:8:  - package-ecosystem: "pip"

#233 [dependencies,python]   #227 [dependencies,python]   #222 [dependencies,python] ...

Only pip is configured, so dependabot never opens a workflow bump here and cannot apply github_actions. Every dependabot PR carries dependencies,python. The page said dependencies and github_actions in one place and three labels including python in another — the two lines contradicted each other. Both corrected, and github_actions is now stated as hand-applied like the rest.

That claim had no pin at all, which is how it survived. It does now: the test derives the configured ecosystems and asserts the page does not re-attribute github_actions. Verified biting.

2. An assertion that could never fail. assertNotIn(extra, extras['all']) compared an extra name against a list of requirement strings — 'DPKT' in ['emoji', 'cryptography>=3.4', 'pycrate'] is element equality, never true. The reviewer proved it by widening all back to the pre-#910 engine set: the test whose message reads "{extra} is back inside all" stayed green with four engines back inside all. Now compares requirement sets; verified it catches that exact corruption.

3. A test with no floor. test_every_issue_link_number_matches_its_own_url ends in assertEqual(mismatched, []), and zero matches produces [] == [] — so it passed on wholesale-emptied pages. Today it examines 44 links, but any link-style change takes the regex to zero and the test goes quiet rather than red. Floor added. The reviewer's sharpest point: this module already guards that shape five times elsewhere, and even prefers an explicit skipTest over a silent pass — this one had been left out of the pattern it established.

The censuses, which are the durable half:

  • assertNotIn, 11 calls — 1 positive (defect 2). It checked the type: ignore[override] # pylint: disable=arguments-differ needle byte-for-byte against git show b337cdbc2^ — 2 occurrences before, 0 now, spacing exact. That was the likeliest trap and it is clean.
  • assertEqual, 23 calls — 0 transcriptions masquerading as independent. 17 genuinely independent, 4 deliberate round-trips. A clean negative, reported because that is itself the result after four positives in the assertIn class.
  • Wholesale section deletion: 12 of 15 tests fail, 3 survive — one of them defect 3, the other two documented as tree-only checks.

It also caught a pytest-subtests undercount while doing it: pytest reported test_the_page_names_every_commit_type_the_template_ticks as PASSED while all 8 subTests failed; plain unittest fails it correctly.

New revision e4340e8a6. tests/project: 221 passed, 1 skipped, 641 subtests.

@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO at e4340e8a6 — sonnet round 11. Eleven rounds, and this is the first clean verdict.

The check that matters for the merge decision, re-derived myself rather than relayed. main now carries 793aecd11, which retargeted four RFC 959 anchors in registry-protocol.rst — the same file this branch rewrites. Merged origin/main into a scratch worktree:

Auto-merging docs/source/contributing/conventions/registry-protocol.rst
Automatic merge went well
conflicts: (none)
merged tree: 223 passed, 1 skipped, 658 subtests

Both edit sets compose: three 959#section-5 anchors from main, and this branch's The 6 audit figure and get-override section all present. The subtest count rises from 641 to 658 because main brings #946's own tests. A clean auto-merge that breaks a test is the failure worth catching before you merge, and it does not happen here.

One thing I nearly mis-reported: grep -c 'forced by Python rather than decided' returns 0 on the file while the assertion on it passes — the phrase is line-wrapped, so it exists only in the whitespace-flattened text. Same trap that bit the is not an instance or subtype of type needle. Confirmed present in both the branch and the merged tree via the flattened form.

Its assertTrue/assertFalse census — the one class never audited — came back clean. All 11 calls (8 assertTrue, 3 assertFalse), no vacuous passes. It checked the ones that could plausibly be empty for unrelated reasons: assertTrue(multi) sees 49 real multi-issue changelog entries with no duplicate-number false positives; assertFalse(named - set(packages)) extracts all nine module names correctly from the ls -d block; all six cited paths appear on the page so all six subTests execute; both issue templates set labels:.

It also verified the three assertions e4340e8a6 added all fail when their subject is corrupted, and that the link-test floor has real margin — 44 links against a floor of 40.

One finding I am not acting on, with the reason. vars(Method)['get'] raises a bare KeyError rather than a diagnostic AssertionError if that override is ever folded into the base — which is exactly what #940 did to the mh/ngap helpers. It fails loudly either way, so no regression passes silently; it is diagnostics quality, not correctness. Folding it into #949, which already owns test-quality cleanup on these files, rather than spending a twelfth round to bless a three-line change.

It also noted the new assertNotIn pins one exact phrasing, so a differently-worded re-attribution of github_actions would slip past it — an inherent limit of substring pinning that this suite documents elsewhere. Fair, and recorded rather than smoothed over.

tests/project on the branch, under plain unittest with a subTest counter rather than pytest: 221 passed, 1 skipped, 641 subtests.

Unpublished and yours to merge.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: pending No verdict for the current head - never reviewed, or the head moved since the last one labels Sep 30, 2026
@JarryShaw
JarryShaw merged commit bb3562e into main Sep 30, 2026
63 checks passed
@JarryShaw
JarryShaw deleted the docs/918-harvest-get-override-rulings branch September 30, 2026 18:28
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Sep 30, 2026
JarryShaw added a commit that referenced this pull request Sep 30, 2026
…view

The by-module restructure rewrote ``docs/source/changelog/1.5.0.rst`` wholesale,
which conflicted with #946's one-word anchor fix on ``main``. Resolved by taking
the restructured file and applying that fix to it
(``959#section-5.3`` -> ``959#section-5``), then regenerating ``CHANGELOG.md``.

Merging ``main`` brings #948's tests, which the restructure makes **vacuous**
rather than merely stale, so they are rewritten rather than renumbered:

* ``test_the_page_describes_the_changelog_kind_runs_as_they_are`` measured the
  inline kind labels the restructure removed: its ``findall`` matched nothing,
  ``runs`` was empty, and both ``assertGreater`` floors failed before the prose
  needle. Replaced by ``..._grouping_as_it_is``, which counts the ``-`` and ``~``
  underlined sections, requires the kinds nested inside the modules, requires no
  inline label anywhere, and requires the section count in the page's prose.
* ``test_the_changelog_file_is_not_shaped_one_entry_per_commit`` and
  ``test_the_page_pins_its_own_measured_numbers`` both keyed on
  ``^\* \*\*Kind\*\*``; an entry is now a column-zero bullet.
* ``process.rst`` described the file as "neither shape", with figures for a
  layout that no longer exists. It now describes what shipped: 9 module-level
  sections holding 155 entries, no inline kind labels.

Cross-review findings on ``a84020c3a``, both confirmed before fixing:

* The stated reason for retargeting
  ``test_a_sub_heading_underline_joined_into_the_prose_is_fatal`` was backwards,
  in the commit message and in the test's own comment. ``_reject`` is built on
  ``assertRaises``, so the old ``-`` input converting cleanly made that test
  **fail**; the retarget was required, not a tidy-up. Comment corrected.
* ``tests/project/test_isort_clean.py`` cited ``1.5.0.rst`` lines 691 and 1169 --
  already stale by 2 and 18 before this work, and off by ~2700 after the
  reshuffle. Line numbers dropped; the file is cited alone.
* Added ``test_an_over_long_sub_heading_underline_is_fatal``: ``_RESIDUAL``'s
  comment now claims ``=``, ``-`` and ``~`` all need to stay in its alternation,
  and only ``=`` was pinned.

``pytest tests/project``: 225 passed, 1 skipped, 660 subtests, 0 failed;
``unittest`` agrees at 226 tests OK. Changelog drift gate exit 0.
@JarryShaw JarryShaw added this to the 1.5 milestone Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Pull requests that change documentation only (docs: subject prefix) test Pull requests that add or correct tests (test: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant