Skip to content

perf(reg): drop the per-member annotation and TransportProtocol.get() calls - #769

Merged
JarryShaw merged 1 commit into
mainfrom
fix-744-768-apptype-annotation-and-get
Sep 25, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix-744-768-apptype-annotation-and-get

Conversation

@JarryShaw

@JarryShaw JarryShaw commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

What is the purpose of your pull request?

  • fix — corrects a defect
  • feat — adds a feature
  • perf — changes performance, not behaviour
  • refactor — changes neither behaviour nor performance
  • test — tests only
  • docs — documentation only
  • ci — workflows or build tooling
  • chore — anything else

Description of your pull request and other information

Fixes #744. Fixes #768. Both are one-line changes to the same generator forcing a full regen of the same five files, so one PR avoids conflicting ~12,000-line diffs.

#744 drops : {self.NAME!r} from each member's assignment (apptype.py:717), plus the # mypy: disable-error-code=assignment pragma it forced. Removes 86,838 bytes. Autodoc's sphinx.util.typing.get_type_hints() re-resolves the whole class per member — 378.8s on CI, the docs build's largest stall; measured on this PR, deploy-pages drops 567s → 213s.

#768: flag() now emits TransportProtocol.tcp instead of TransportProtocol.get('tcp') — ~21x cheaper, 23,941 of 23,942 sites rewritten (the 23,942nd, in _dispatch, resolves a caller-supplied string and stays get()). 7 bytes/site, 167,587 bytes. Covers #764's 4 range-row conditions too.

Regen: one pinned IANA snapshot; unmodified crawler reproduces the five files byte-identically first, then the diff after both changes is mechanical throughout (254,615 bytes total: 190 pragma + 86,838 annotation + 167,587 .get()). Exhaustive member-set (name/port/svc/proto) compare across all 12,391 members: identical. #764's behaviours re-confirmed.

Tests: updated #760's regex for the new form and tightened its trailing match; added generator pins (0%→20% is the wrong framing — the module was already at 19% via test_vendor_dest_path_unit.py/test_crawler_reachability_unit.py; the new file's flag() pin takes it to 20%), each shown to fail without its fix. make isort clean.

mypy: removing the pragma surfaces 4 pre-existing errors (112→116), all pre-existing TransportProtocol.undefined-as-default mistyping in the base template, unrelated to either fix. Advisory-only in CI; left unfixed/unsuppressed rather than silently re-adding the pragma.

@JarryShaw JarryShaw added perf Pull requests that improve performance (perf: subject prefix) const Regenerated IANA or vendor constant tables; members keep their numeric values test Pull requests that add or correct tests (test: subject prefix) review: pending No verdict for the current head - never reviewed, or the head moved since the last one labels Sep 25, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Decision needed: what to do with 4 mypy errors this PR unmasks.

Removing the annotation makes # mypy: disable-error-code=assignment redundant, so the PR drops it from all five generated files (it was on 5, not the 4 I had guessed). That pragma was also hiding four errors unrelated to this change. Measured on the apptype file alone:

main (pragma present) :  Success: no issues found in 1 source file
this branch (removed) :  4 errors
  :2258  Incompatible types in assignment (expression has type "int", variable has type "TransportProtocol")
  :2282  Incompatible default for parameter "proto" (default has type "int", parameter has type "TransportProtocol")
  :2371  Incompatible default for parameter "proto" (default "int", parameter "TransportProtocol | str")
  :2429  Incompatible default for parameter "proto" (default "int", parameter "TransportProtocol | str")

Whole-package: 112 errors in 38 files → 116 in 39. All four are in the base template's static code (__new__, _dispatch, get, get_all), every one about TransportProtocol.undefined used as a default.

Root cause, isolated. undefined = 0 is a bare int literal so mypy infers its attribute type as int, while tcp/udp/sctp/dccp are auto()-valued and infer as Any — which is why only undefined trips it. mypy has no aenum plugin, so the class is treated as plain rather than as an enum. Pre-existing and independent of this PR: stripping only the pragma on unmodified main reproduces all four, and none of the ~12,391 per-member lines contributes a new error.

Three options:

  1. Leave them unsuppressed and file them as their own issue. mypy is advisory in CI (continue-on-error: true in lint.yml), so nothing breaks, and ci(lint): the workflow's recorded analyser counts have drifted (mypy 115/39 vs 112/38) #753's whole premise is that recorded drift beats hidden drift. The count in lint.yml's header would need updating to 116/39 — or the issue would note it deliberately.
  2. Re-add a narrow suppression naming only [assignment] on those four lines, rather than the file-wide pragma. Keeps the count at 112 and keeps the fix honest about what it is suppressing.
  3. Fix the inference. Likely needs an aenum stub or an explicit annotation on the TransportProtocol members; I have not established that it is achievable without one, so this may not be a real option today.

I would take (1): they are real, they were always there, and this PR's job is not to carry them. Recommending it rather than deciding, since it moves a published count.

This does not block the PR — CI is fully green, including deploy-pages, which means the docs build no longer hangs.

@JarryShaw JarryShaw added the needs: decision Waiting on the maintainer to decide — not blocked by other work label Sep 25, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

take (1) then.

@JarryShaw JarryShaw removed the needs: decision Waiting on the maintainer to decide — not blocked by other work label Sep 25, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Ruling recorded: option (1) — leave the four errors unsuppressed. Filed as #770 with the root cause, the isolation showing they are pre-existing, and the four sites (__new__, _dispatch, get, get_all).

#770 also carries the one follow-up this creates: lint.yml's header pins mypy at 112 errors / 38 files measured at 932cb48d1, which #761 has just repinned and is merge-ready. That pin stays honest because it names its commit, but once this PR merges the live figure is 116/39 — so whichever of the two lands second leaves the other reading stale unless the count is refreshed. Flagged there rather than silently letting them drift apart.

… calls

- apptype.py:717 dropped `: {self.NAME!r}` from each member's assignment.
  Every annotated member put a forward-ref string into AppType's
  __annotations__, and Sphinx's default autodoc re-resolves the whole
  class once per member -- 378.8s on CI, the largest stall in the docs
  build. Also drops the now-redundant `# mypy: disable-error-code=
  assignment` pragma that annotation forced on all five generated files.
- apptype.py's flag() now emits `TransportProtocol.tcp` instead of
  `TransportProtocol.get('tcp')` -- 21x cheaper per call, 23,941 of the
  23,942 call sites on this checkout rewritten (the 23,942nd, in
  _dispatch, resolves a caller-supplied string and stays). Also covers
  the 4 range-row conditions #764 added to records(). get() itself is
  untouched; other callers still use it.
- Regenerated all five files under pcapkit/const/reg/apptype/ from one
  pinned IANA snapshot. The unmodified crawler reproduced the current
  files byte-identically first; the diff after both changes is
  mechanical throughout.
- Updated the #760 regression test's regex for the new attribute form
  and tightened its trailing match, and added generator-level pins for
  both changes.

Fixes #744. Fixes #768.

Regenerated from one pinned IANA snapshot; unmodified crawler reproduced
the committed files byte-identically before either change landed.
Exhaustive member-set comparison (name/port/svc/proto) across all 12,391
members in AppType/TCP/UDP/SCTP/DCCP: identical. make isort clean.
@JarryShaw
JarryShaw force-pushed the fix-744-768-apptype-annotation-and-get branch from 1a4b773 to 9ae0e29 Compare September 25, 2026 03:04
@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: pending No verdict for the current head - never reviewed, or the head moved since the last one labels Sep 25, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Cross-review at 9ae0e2991: GOOD TO GO (opus, a different model from the author).

All four prose corrections landed. The worked example is now character-for-character identical to tcp.py:946 (same sha256 once the indent is stripped), and the byte figures re-derive from the real member counts: 12,290 × 7 + 101 × 8 = 86,838 for the annotations — 7 bytes where the class name is 3 letters, 8 where it is 4 — plus 7 × 23,941 = 167,587 for the .get() rewrite, which is a flat 7 per site because TransportProtocol.get('sctp') is 29 → 22 as well. With the five pragmas that is 190 + 86,838 + 167,587 = 254,615, matching the measured file-size delta to the byte.

git diff --stat 1a4b77353..9ae0e2991 -- pcapkit/ is empty, so the generator and all five generated files are untouched this round and last round's byte-identical regeneration proof carries over. The assertIn → condition.endswith(claim) tightening was checked against a truth table over the real emitted conditions: it keeps every real shape, still rejects a wrong proto and a negated is not, and additionally rejects both directions of the prefix collision the NOTE describes. The generator emits the claim as the suffix by construction, so requiring it to be final is correct rather than over-tight.

One correction to the headline number, which I had overstated. The deploy-pages baseline is noisier than a single pair suggests:

main 57b2c1761  567 s     main 60bb62d8f  574 s     main 1a852698b  717 s
PR   1a4b77353  213 s     PR   9ae0e2991  146 s

Round 2 touched only tests/, which the docs build never reads, so 146 s versus 213 s is runner variance on identical docs input, not a further improvement. The reduction is large and unambiguous — roughly 3-5× — but it should be stated as a range, ~570-720 s on main against 146-213 s on this branch, rather than as a precise pair. I will not hold the PR for that.

Also worth recording: the commit message calls the pragma "now-redundant … on all five generated files". On four it is redundant; on the base module it was suppressing four unrelated errors, since keeping it there alone restores 112 errors in 38 files exactly. The decision to leave them visible is settled and tracked in #770.

Merge with current main 1a852698b re-confirmed conflict-free, with Ran 115 tests … OK on the merged tree across test_tier_guard (#755), test_isort_clean (#761) and both of this PR's test files. Unpublished and awaiting the owner.

@JarryShaw
JarryShaw merged commit 83b58eb into main Sep 25, 2026
31 checks passed
@JarryShaw
JarryShaw deleted the fix-744-768-apptype-annotation-and-get branch September 25, 2026 03:39
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

const Regenerated IANA or vendor constant tables; members keep their numeric values perf Pull requests that improve performance (perf: subject prefix) test Pull requests that add or correct tests (test: subject prefix)

Projects

None yet

1 participant