chore: generate pyshark's encap-type and filter-name maps from a util script

[JarryShaw]

Please follow the guide below

• Searched for similar pull requests
• Followed the coding style (make pylint, make mypy, make isort)
• make test passes, and a test case covers the change
• Added a changelog entry — N/A — changelog centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657

What is the purpose of your pull request?

• chore — anything else

---

Description of your pull request and other information

Closes #851. Adds util/pyshark_encap_map.py, which regenerates ENCAP_TYPE_TO_LINKTYPE (152 entries) and
FILTER_NAME_TO_LINKTYPE (58) in place in pcapkit/toolkit/pyshark.py, so they stop being hand-maintained.
Per your ruling on #850 — "They may exist in toolkit still - if generated by scripts" — the tables stay
where they are; only their provenance changes. util/ rather than pcapkit/vendor/ because
Vendor._dest_path() is hardwired to pcapkit/const/, Vendor.context() emits a whole enum class file, and
Vendor.request() is an HTTP fetch, whereas this generator's source of truth is the local tshark/editcap
binaries.

4 files, +1359/−1. pcapkit/toolkit/pyshark.py is modified by exactly one line — the IPMB_LINUX →
I2C_LINUX rename that #848 made canonical for value 209. Textual only: they are the same member
(LinkType.IPMB_LINUX is LinkType.I2C_LINUX), so ENCAP_TYPE_TO_LINKTYPE[112] is 209 before and after.

The HAS_WIRESHARK gate — the part an earlier draft of this PR got backwards

The real-sweep tests need tshark and editcap. Those were first guarded by unprefixed shutil.which
constants, on the theory that avoiding the HAS_* prefix avoided the tests/_dependency_gates.py scan that
cost #848 ten CI legs. That was the defect, not the fix: it made the scan blind to two classes skipping on
every leg. This head declares HAS_WIRESHARK properly and registers it in NON_DISTRIBUTION_FLAGS
(tests/_dependency_gates.py:240), next to HAS_PROC_FD — the bucket for "asks about something pip cannot
install at all". dependency_gate_gaps() returns no gap for it; CI installs tshark at
.github/workflows/unit-tests.yml:379 and :469, which pulls wireshark-common for editcap.

The version gate, because the sweep arithmetic is version-pinned

editcap -T accepts 226 encapsulations on Wireshark 4.6.9 (this host) and 224 on 4.2.2 (CI's Ubuntu
noble). The count assertions therefore run only when detect_tshark_version() reports exactly
MEASURED_WIRESHARK_VERSION = '4.6.9'; every mismatch and every parse failure lands on one named skip.
VersionIndependentInvariantTests carries the checks that hold at any version, so CI is not left asserting
nothing.

Verification on head 7f42887f8

md5 pcapkit/toolkit/pyshark.py   main 6afa4ed60c5bb9541d95c04dbffbd8f4 -> head c9ec0f6952da3be0c07f13e6a49e06d0
util/pyshark_encap_map.py   sweep: 226 accepted, 157 writable, 69 refused; nothing to write
tests/project/test_pyshark_encap_map.py   29 passed, 0 skipped
tests/test_tier_guard.py                  102 passed / 552 subtests

Also confirmed on a real Python 3.10.21 interpreter (CI's floor leg), because an earlier head used
unittest.TestCase.enterContext, which is 3.11+: all 29 tests pass there, with
hasattr(unittest.TestCase, 'enterContext') is False printed from that same interpreter.

The bug the generator had to work around, which is the most valuable part

A naive try: LinkType(dlt) except ValueError cannot detect an unmapped DLT, because LinkType._missing_
mints a placeholder Unassigned_N member rather than raising. The first lookup of DLT 121 would have
minted Unassigned_121 and written it straight into the table. Fixed by snapshotting
known_values = frozenset(member.value for member in linktype) before any lookup, so LinkType(dlt) is
only ever called once that has already said yes. Pinned by
BuildTablesExclusionTests.test_an_unmapped_dlt_is_left_out_and_noted.

Both traps from #851 are covered: a showname containing / (ParseParenthesisedNumberTests) and the
editcap -T banner tokens (ListEncapTypesTests, including a fabricated fake-binary round trip).

Caveats

• make pylint and make mypy cover pcapkit/ only, so no lint or type gate applies to util/ or tests/
  here. isort --check-only -l100 -ppcapkit is clean on both new files.
• 16 filter names are ambiguous and reported as note: lines rather than dropped silently (user_dlt,
  bluetooth, usb, usbll, raw, arcnet, gfp, i2c, lapd, mtp2, netanalyzer, null, ppp,
  sll, wlan, wpan), matching the exclusions pyshark.py's own comments already document.
• VersionIndependentInvariantTests has never executed on 4.2.2, the version it exists to compensate for.
  Its invariants are closed under taking subsets of measurements, so 224 ⊂ 226 should pass; the CI leg
  settles it.

[JarryShaw]

Three legs failed, and the root cause is a genuine finding rather than a code defect: the tests are pinned
to one Wireshark version, and CI runs a different one. Python 3.10, Engines Python 3.11 and
Engines Python 3.13 — 2 failures each, both in RealSweepTests, one defect.

FAILED tests/project/test_pyshark_encap_map.py::RealSweepTests::test_sweep_arithmetic_matches_the_committed_comment
  AssertionError: 224 != 226
FAILED …::RealSweepTests::test_check_mode_reports_the_committed_file_as_current
  EncapSweepError: editcap -T accepted 224 encapsulations, expected 226; Wireshark has moved
  since 4.6.9 and every table comment measured against that version needs re-checking

The versions, from the job log and my host:

CI    (ubuntu noble)  libwireshark-data 4.2.2-1.1build3   -> editcap -T accepts 224
local (homebrew)      TShark 4.6.9 (2d548b197c75)         -> editcap -T accepts 226

The script behaved exactly as designed. It refused to rewrite the table when the sweep disagreed with the
committed arithmetic, and said why in one precise sentence. That is the "fail loudly rather than quietly
rewrite the table" property #851 asked for, demonstrated in the field on its first CI run. Worth keeping.

What is wrong is RealSweepTests pinning absolute counts, which makes them pass only on Wireshark 4.6.9.
Two ways out, and I want the first:

1. Skip RealSweepTests unless the local tshark --version matches the version the tables were measured
   against, and name that version in one place so it is visible. The tables are 4.6.9 artefacts; a test
   asserting them against 4.2.2 is asserting the wrong thing, not finding a bug.
2. Replace the absolute-count assertions with version-independent invariants — no encap key mapping to two
   DLTs, no DLT to two keys, every committed entry reproducible from whatever the local editcap does accept.

(1) is honest about what is being pinned; (2) is weaker but survives an upgrade. Doing (1) as the fix, with
(2)'s invariants added alongside where they cost nothing.

A consequence worth stating even though it is not a defect: the committed tables describe 4.6.9's 226
encapsulations, so they carry entries a 4.2.2 tshark would never produce. That is harmless at runtime — an
encapsulation value that never appears simply never gets looked up — and arguably the right way round, since
the table should be at least as rich as the newest Wireshark a user might have. But it means this generator
cannot be run in CI to verify the committed tables, only locally against a pinned version. That is a real
limitation of the approach and belongs in the script's docstring.

[JarryShaw]

NEEDS CHANGES on 4e5a31a8b — cross-review (opus, a different model from the sonnet author). Four
changes required; three are the reviewer's, one is mine from CI. One correction to the reviewer's severity
framing, verified before relaying.

1. Rebase onto current main and include the regenerated pcapkit/toolkit/pyshark.py. origin/main
carries f990f2149 (#848), which is not in this PR's base, and the generator finds a real difference at
pyshark.py:148:

-    112: Enum_LinkType.IPMB_LINUX,    # i2c-linux
+    112: Enum_LinkType.I2C_LINUX,     # i2c-linux

But this is textual, not behavioural — I checked, and the reviewer's framing overstates it. On current
main, at runtime:

LinkType(209).name          = I2C_LINUX
ENCAP_TYPE_TO_LINKTYPE[112] = I2C_LINUX  (value 209)

IPMB_LINUX and I2C_LINUX are the same member — aliases for 209 — so .name already returns the
canonical one and the table was never wrong at runtime. So this is not drift that "hand-maintenance missed";
it is the generator preferring the canonical spelling. The rebase and the one-line change are still required,
and the description's "pyshark.py is not modified — that is the point" must become three files.

2. The docstring's trap numbers do not reproduce — and they are my error, inherited from #851's body.
Measured against the binary the docstring names (editcap, Wireshark 4.6.9):

claimed	measured
str.split() → 646	1237
whitespace-run split → 227	1237
"the banner's editcap: line" (one)	two — editcap: "" isn't a valid encapsulation type and editcap: The available encapsulation types…

646 was unique tokens under a sort -u pipeline, which I then restated in #851 as a plain token count.
Correct both the docstring and #851. The <tok> - <desc> shape gives 226, 226 unique — that part holds.

3. ParseParenthesisedNumberTests does not pin the trap it is named after. Reverting _PAREN_NUMBER to
#851's naive form leaves all 4 tests passing, because .search is unanchored so \w+ matches Loopback and
steps over the /. The real trap was extracting the name from showname; this function extracts only the
number. Strengthen it or drop the claim.

4. My own finding from CI: RealSweepTests is pinned to Wireshark 4.6.9 and CI runs 4.2.2 (224
encapsulations), so three legs failed. Already sent to the worker; see my earlier comment.

On the gate design, the reviewer disagrees with the approach and I think it is right.
tests/_dependency_gates.py ships NON_DISTRIBUTION_FLAGS for "flags that ask about something pip cannot
install at all", with HAS_PROC_FD as precedent. A Wireshark binary is exactly that class, so
HAS_WIRESHARK + a NON_DISTRIBUTION_FLAGS entry is supported and visible, where the _EDITCAP naming
dodge leaves 2 of 20 tests silently skipping on all 10 legs — and item 4 is the concrete cost of that
silence. Folding this into the version-gate fix.

Two things confirmed clean and worth recording: idempotence survives perturbation — reversed order,
trailing whitespace, deleted entry, changed value all restore to the gold md5 — though CRLF is invisible to
it (read_text universal newlines makes --check call a CRLF file clean); newline='\n' on the write
closes it, and .gitattributes * text=auto already prevents it reaching a commit. And a mismatched
assertion cannot write a wrong table: flipping EXPECTED_WRITABLE to 156 exits 1 with the target md5
unchanged, because the assertions sit before build_tables.

[JarryShaw]

Correcting my own root cause: there are two defects here, not one. I said earlier the three red legs were
"one defect" — the Wireshark version mismatch. That was right for the Engines legs and wrong for
Python 3.10. Now six legs are red and the split is clean:

leg	failures	cause
Python 3.10	2	self.enterContext is Python 3.11+
Engines Python 3.10	4	both causes
Engines Python 3.11/3.12/3.13/3.14	2 each	Wireshark 4.2.2 vs 4.6.9 (224 ≠ 226)

Every failure is in tests/project/test_pyshark_encap_map.py; zero elsewhere on all six legs.

Defect 2, new and unrelated to Wireshark:

AttributeError: 'ListEncapTypesTests' object has no attribute 'enterContext'
  tests/project/test_pyshark_encap_map.py:143   (also :128)

unittest.TestCase.enterContext landed in Python 3.11. This repo supports 3.10, which is why only the
3.10 legs fail and 3.11–3.14 pass — and why Python 3.10 fails even though the test job installs no
Wireshark at all. That asymmetry is what gave it away; a version-mismatch story could not explain a leg with
no editcap on it.

Fix: use contextlib.ExitStack registered via self.addCleanup(stack.close), or
self.addCleanup(tempdir.cleanup) directly — both work on 3.10. Do not reach for a sys.version_info guard;
the test should run on every supported version, not skip on the oldest.

Worth noting how close this came to being missed. Both 3.10 failures sit in ListEncapTypesTests, which
is the class that does genuinely pin the editcap -T banner trap — the review confirmed reverting
_ENCAP_LINE turns it red. So the one class carrying real protection was the one silently not running on the
oldest supported Python.

Both defects are now with the worker. review: needs-changes stands.

[JarryShaw]

All four rounds of feedback cleared. Head 4e5a31a8b → 7f42887f8, rebased onto f990f2149, so the
needs-changes verdict is superseded — relabelled review: pending, fresh cross-review to follow. Verified by
me, not taken from the summary:

• enterContext is gone. The two remaining textual hits are docstring prose explaining why it is avoided
  (:275,:278), not calls. All five sites now use tempfile.TemporaryDirectory() + self.addCleanup(tmp.cleanup),
  supported since 3.6.
• HAS_WIRESHARK is a real gate registered where it belongs, not a naming dodge:
  NON_DISTRIBUTION_FLAGS now reads ['HAS_PROC_FD', 'HAS_WIRESHARK']. And the tier guard's subtest count rose
  547 → 552, which is the evidence that matters — the new gate is being exercised, not silently skipped.
  tests/test_tier_guard.py: 102 passed / 552 subtests, re-derived as Ran 102 tests … OK.
• Idempotence holds on the rebased base: md5 c9ec0f6952da3be0c07f13e6a49e06d0 unchanged, git status
  clean, and --check reports matches the sweep with 226 accepted, 157 writable, 69 refused.
• The new paren test is genuinely discriminating — I ran the naive regex against its fixture
  'Encapsulation type: NULL/ (15)' and it returns None, so reverting the implementation really does fail
  it. That was the whole objection to the old class.
• New test file: 29 passed under both pytest and plain unittest.

Two things the worker found that are worth keeping:

It hit the editable-install trap inside its own generator. main()'s
from pcapkit.const.reg.linktype import LinkType was resolving through the venv's editable install — the main
checkout, not the worktree — because a meta_path finder wins over sys.path regardless of insertion
order. It now evicts any editable-named finder and raises EncapSweepError if pcapkit.__file__ is not
under ROOT. That makes the generator self-guarding rather than relying on the caller's environment.

requires-python is >=3.6, <4, while CI's floor leg is Python 3.10. The fix targets 3.6, so it is safe
either way, but the declared floor and the tested floor differ by four minor versions — worth its own look
sometime, not here.

One honest UNVERIFIED, which I am carrying rather than smoothing over: the 3.10 compatibility claim rests on
a targeted hasattr(self, 'enterContext') is False check on a real 3.10.21 interpreter plus py_compile, not
a full pytest run there, because that interpreter has no aenum/deps. CI's Python 3.10 leg is what will
actually settle it.

[JarryShaw]

GOOD TO GO on 7f42887f8 — second cross-review (opus, a different model from the sonnet author). It
returned NEEDS CHANGES, but the single required change was not in the diff: the PR description was
still the pre-review draft and argued for the anti-pattern this head rejects. I have rewritten it. All four
code fixes verified good, nothing wrong found in the diff.

The description's four falsehoods, each re-measured by me before I touched it:

it said	head 7f42887f8 actually
"pyshark.py is not modified — 2 files added, 980 lines. That is the point"	4 files, +1359/−1, and pyshark.py is modified (one line)
"no new dependency gate … deliberately not HAS_* … the defect that cost #848 ten CI legs, avoided by construction"	HAS_WIRESHARK, registered in NON_DISTRIBUTION_FLAGS (tests/_dependency_gates.py:240, +11/−0)
md5 before/after 6afa4ed60c5bb9541d95c04dbffbd8f4	6afa4ed… is main's; the head is c9ec0f6952da3be0c07f13e6a49e06d0
"new test file: 20 passed"	29 (grep -c '^+ def test')

It also never mentioned MEASURED_WIRESHARK_VERSION, the version gate, or VersionIndependentInvariantTests —
the whole of the new load-bearing machinery. Fixed.

What the review established beyond the previous round:

• enterContext on Python 3.10 is settled, not inferred. It installed the five runtime deps outside the
  repo venv and ran the whole new test file on a real 3.10.21 interpreter: Ran 29 tests … OK, with
  hasattr(unittest.TestCase, 'enterContext') is False printed from that same interpreter. The generator runs
  there too (it spawns sys.executable).
• The version gate does not silently skip or silently pass. Driven against 11 fabricated banners and 4
  failure modes; every mismatch and every parse failure lands on one named skip that interpolates the
  detected version. 4.6.9rc1 would let the sweep run, which fails loudly on a divergent count rather than
  passing quietly.
• VersionIndependentInvariantTests has teeth — mutation-proven. Flipping ENCAP_TYPE_TO_LINKTYPE[6] and
  FILTER_NAME_TO_LINKTYPE['fddi'] to SLIP turned both comparison tests red with precise diagnostics; the
  two ambiguity tests correctly stayed green. Reverted, md5 back to c9ec0f69….
• HAS_WIRESHARK is not dark: dependency_gate_gaps() returns no gap for it, and CI installs tshark at
  .github/workflows/unit-tests.yml:379 and :469, which pulls wireshark-common for editcap.

Three non-blocking observations recorded rather than acted on: measure_encap's ET.fromstring is the one
unguarded failure path (an invalid PDML is a setUpClass ERROR, not a skipped entry); detect_tshark_version
runs subprocess.run at import time with no timeout; and the test module imports pcapkit without pinning
ROOT onto sys.path. Also: VersionIndependentInvariantTests has never executed on 4.2.2, the version it
exists to compensate for — its invariants are closed under subsets, so 224 ⊂ 226 should pass, and the CI leg
settles it.