Repository navigation
fix(reg): emit tcpdump legacy link-type names after current ones - #848
Conversation
6aaee9c to
f04a343
Compare
|
The generator-level tests parse their fixture with Fixed with the repo's established guard, mirroring The consequence, stated rather than buried: Verified both ways before pushing:
Amended into the single commit rather than stacked; no verdict had been issued on the old head, so |
… IPMB_LINUX (#844) - pcapkit/vendor/reg/linktype.py: LinkType.process() now sinks any tcpdump row whose notes column says "legacy" into a bucket appended after every other row, so the current name is always defined first for a value the registry double-assigns. - pcapkit/const/reg/linktype.py: regenerated as a consequence -- IPMB_LINUX (legacy) moved after I2C_LINUX (current), so LinkType(209).name now resolves to 'I2C_LINUX' instead of the legacy 'IPMB_LINUX'. - tests/const/test_const_linktype_209_unit.py: pins the resolution against the committed const module and, offline, against the generator's own process() fed tcpdump's real two rows for value 209. Verified against a live fetch of tcpdump.org/linktypes.html: 209 is the table's only doubly-assigned value and its only "legacy" note. Build: tests/const/ + tests/vendor/, 188 passed / 39644 subtests passed.
f04a343 to
88f52fd
Compare
|
My previous fix was wrong and the repo's own guard caught it. Head is now
I introduced a new dependency gate, The right fix was to mint no new gate at all. with Net effect, which is better than my first attempt: the root-cause pin runs per-PR on Verified before pushing:
Amended into the single commit; |
|
GOOD TO GO on I got the label wrong, and the reasoning is worth recording. I argued "no name disappears and no value So What the review verified independently, each derived rather than read off the PR:
Three residual risks it raised as follow-ups rather than blockers — filed as #852 so they are not lost. |
…il script - Add util/pyshark_encap_map.py: sweeps every editcap -T encapsulation, writes it as pcap from examples/captures/in.pcap, reads the DLT from the file's own header and frame.encap_type back via tshark PDML, and rewrites ENCAP_TYPE_TO_LINKTYPE/FILTER_NAME_TO_LINKTYPE in pcapkit/toolkit/pyshark.py in place. Evicts any editable-install finder and pins sys.path to this checkout before importing LinkType, so the sweep cannot silently measure against a different tree's registry. - Extracts the PDML showname's trailing number with a regex anchored to the end of the string, and parses editcap -T's own help listing by the "<token> - <desc>" line shape, so neither of its two banner lines is miscounted as a token. - Excludes a DLT with no LinkType member and an ambiguous filter name, checking against the registry's genuinely-registered values up front so LinkType's own _missing_ cannot mint a placeholder member as a side effect of the check (#575). Rebased onto #848 (I2C_LINUX canonical for DLT 209): regenerating now correctly prefers it over the legacy IPMB_LINUX alias. - Asserts the sweep arithmetic (226 accepted, 157 writable, 69 refused) only against the Wireshark version it was measured on (MEASURED_WIRESHARK_VERSION = 4.6.9); skips cleanly when tshark/editcap are absent or on a different version, since CI's packaged Wireshark (Ubuntu noble, 4.2.2) accepts 224. - Add tests/project/test_pyshark_encap_map.py: version-independent invariants (no key/DLT ambiguity, agreement with the committed table) run on whatever Wireshark is on PATH; the exact-count sweep is gated on HAS_WIRESHARK, a NON_DISTRIBUTION_FLAGS entry in tests/_dependency_gates.py (a binary, not a pip distribution) plus the pinned version, so the gap is visible rather than silently dark. 3.10-safe (no TestCase.enterContext, 3.11+ only). - Verified: regenerating against the rebased tree reproduces pcapkit/toolkit/pyshark.py byte-identically on a second run (md5 c9ec0f6952da3be0c07f13e6a49e06d0 before and after --check); one legitimate line changes from origin/main (IPMB_LINUX -> I2C_LINUX, textual only, same aliased member); tests/test_tier_guard.py is green (102 passed, plain unittest cross-check OK); py_compile clean on Python 3.10.
Follow-ups from the cross-review of #848's legacy-sink rule in pcapkit/vendor/reg/linktype.py: - The sink predicate tested a row's notes for the word "legacy" alone, with no check that the row's value was actually claimed by another row -- so a future current row whose notes coincidentally mention "legacy" would be wrongly sunk. Gate the sink on the row's value being a genuine duplicate elsewhere in the table (pre-scanned via a Counter), computed independent of row order. - The range branch (USER0-USER15) reused the same per-row sink, so one range row worded "legacy" would sink all sixteen expanded members at once. Range rows now always land in enum, unconditionally. - Reworded the generator comment so it matches what the predicate actually tests, not "shares its value" when it never checked that. - Added a comment at the const file's emission site, as a plain `#` line rather than `#:`, so it explains the source layout without leaking into IPMB_LINUX's rendered Sphinx docstring; qualified the self-reference as pcapkit.vendor.reg.linktype.LinkType.process. Cross-review of this fix itself then caught a second regression: the duplicate-value pre-scan counted only single-value rows, so a value duplicated across a range boundary (a single-value row sharing a value with one member of a USER0-style range) was invisible to it, and that row's legacy alias stopped being sunk. The pre-scan now expands en-dash ranges the same way the main loop does before counting, so no overlap case is missed; an ASCII hyphen still isn't treated as a range. Also fixed: the order-independence test previously proved nothing beyond what the 209 fixture already covered (mutating the pre-scan into a prefix-only, order-dependent variant left it passing); it now runs the same duplicate pair through process() in both orders itself. Added tests for: a non-duplicated value never sunk regardless of wording, a duplicate pair resolving correctly in either table order, range-sink isolation, the cross-range duplicate regression, and a malformed en-dash range still raising loudly -- each shown failing against the relevant prior code. Regenerated pcapkit/const/reg/linktype.py; the range-aware pre-scan alone is byte-identical (md5 7f4db8d612d74fbdbc880f75040bff12) to the prior fix, confirmed by isolating it from the docstring-comment change, which is the only other diff. Invariants hold: LinkType(209).name == 'I2C_LINUX', IPMB_LINUX and I2C_LINUX both alias value 209, 220 members / 219 canonical iteration, 209 the only duplicated value, USER0-USER15 still 147-162. tests/const + tests/vendor: 193 passed; tests/test_tier_guard.py: 102 passed (unittest-confirmed).
Follow-ups from the cross-review of #848's legacy-sink rule in pcapkit/vendor/reg/linktype.py: - The sink predicate tested a row's notes for the word "legacy" alone, with no check that the row's value was actually claimed by another row -- so a future current row whose notes coincidentally mention "legacy" would be wrongly sunk. Gate the sink on the row's value being a genuine duplicate elsewhere in the table (pre-scanned via a Counter), computed independent of row order. - The range branch (USER0-USER15) reused the same per-row sink, so one range row worded "legacy" would sink all sixteen expanded members at once. Range rows now always land in enum, unconditionally. - Reworded the generator comment so it matches what the predicate actually tests, not "shares its value" when it never checked that. - Added a comment at the const file's emission site, as a plain `#` line rather than `#:`, so it explains the source layout without leaking into IPMB_LINUX's rendered Sphinx docstring; qualified the self-reference as pcapkit.vendor.reg.linktype.LinkType.process. Two rounds of cross-review on this fix itself then each caught a narrower-than-main-loop hole in the duplicate-value pre-scan: - Round 1: the pre-scan counted only single-value rows, so a value duplicated across a range boundary (a single-value row sharing a value with one member of a USER0-style range) was invisible to it. The pre-scan now expands en-dash ranges the same way the main loop does before counting. - Round 2: the pre-scan's single-value branch tested str.isdigit(), narrower than the main loop's int(temp), which also accepts a leading sign and PEP 515 underscores. The pre-scan now tries int(temp) directly, before the en-dash check, so it recognises exactly what the main loop does. Neither hole is a live defect -- every one of the 220 committed members matches a plain unsigned decimal -- but each was the same class of silent under-count this fix exists to close for ranges, narrowed further. Also fixed: the order-independence test previously proved nothing beyond what the 209 fixture already covered (mutating the pre-scan into a prefix-only, order-dependent variant left it passing); it now runs the same duplicate pair through process() in both orders itself. And code_int (already parsed by int(temp)) is now reused at the sink = line instead of re-parsing code a second time. Added tests for: a non-duplicated value never sunk regardless of wording, a duplicate pair resolving correctly in either table order, range-sink isolation, the cross-range duplicate regression, a malformed en-dash range still raising loudly in the main loop (not the pre-scan), and signed/underscored duplicate pairs -- each shown failing against the relevant prior code. Regenerated pcapkit/const/reg/linktype.py at every round; each pre-scan widening alone is byte-identical (md5 7f4db8d612d74fbdbc880f75040bff12) to the prior fix, confirmed by isolating it from the docstring-comment change, which is the only other diff. Invariants hold: LinkType(209).name == 'I2C_LINUX', IPMB_LINUX and I2C_LINUX both alias value 209, 220 members / 219 canonical iteration, 209 the only duplicated value, USER0-USER15 still 147-162. tests/const + tests/vendor: 196 passed; tests/test_tier_guard.py: 102 passed (unittest-confirmed).
…ames LinkType(209).name returned 'IPMB_LINUX', the name tcpdump's own table (and pcapkit's generated comment) marks "Legacy names (do not use)", because that row was emitted above the current I2C_LINUX = 209 and aenum gives a value to whichever member is defined first. Fixed in the generator: pcapkit/vendor/reg/linktype.py now sinks any row whose note mentions "legacy" into a bucket appended after every current row, so the current name is always defined first for a value the table double-assigns; today that is value 209 alone. Not breaking: no name disappears, no value changes and no membership changes -- only the value-to-name direction for the shared value moves.
Adds util/pyshark_encap_map.py, a generator that regenerates ENCAP_TYPE_TO_LINKTYPE (152 entries) and FILTER_NAME_TO_LINKTYPE (58) in place inside pcapkit/toolkit/pyshark.py, so the two tables #850 hand-built stop being hand-maintained. Guards against its own worst failure mode: a naive LinkType(dlt) lookup cannot detect an unmapped DLT, because _missing_ mints a placeholder rather than raising, so the generator snapshots every known value before any lookup. The real-tshark sweep is gated on a new HAS_WIRESHARK flag and is version-pinned -- editcap -T accepts 226 encapsulations on Wireshark 4.6.9, 224 on 4.2.2 (CI's Ubuntu noble) -- so the count assertions run only under the measured version. One line of pyshark.py itself changes as a result: the IPMB_LINUX table entry becomes I2C_LINUX, the canonical name #848 gave value 209. util/changelog_md.py regenerated CHANGELOG.md for all seven entries added across this and the six preceding commits (#838, #846, #847, #848, #849, #850, #853); `--check` exit 0.
#846 and #850 - #838's "the 21 whose vendor crawler leaves Vendor.process() unmodified" measures 22, not 21 -- pcapkit.const.hip.transport.Transport also has an unmodified process(), but its FLAG-bound range has no unassigned gap, so its _missing_ carries no bounded-range extend_enum branch to fix. The real discriminator is that branch, not the process() override; reworded, and the "other 84" split into the 83 that do override process() and the one that doesn't but has nothing to fix either. Re-derived independently against 5e25db2^..5e25db2 (not main, which would fold in #847's own edit to ipx/socket.py): 22 registries have no process() override, 21 of them have the bounded-range branch, matching tests/const/test_const_enum_no_mint.py's own REGISTRIES_WITH_UNASSIGNED_RANGES (21 entries) vs ALL_REGISTRIES (22, +hip.transport, with a NOTE explaining the exclusion). - #846's "Only tests/integration/test_engine_runtime.py and test_engine_parity.py change" is false -- 30bca99's own numstat also touches .github/workflows/unit-tests.yml (17+/2-). Qualified to "only these two test files change". - #850 gets the breaking marker: its own prose already says the fallback is gone and an unrecognised encapsulation or filter name now raises MissingKeyError instead of substituting a plausible DLT -- the same shape as the two existing markers at AppType.get (1.5.0.rst:2456) and the four .get()-backed enum fields (1.5.0.rst:2720). Restructured to lead with the marker and the subject, matching their wording and placement; the later restatement of the same fact is dropped. None of these are code or test changes -- prose only, matching the cross-review's own framing. #848 stays unmarked (its own "Not breaking" paragraph holds); #838, #849 and #853 stay unmarked per explicit instruction not to add markers beyond what was asked.
…ames LinkType(209).name returned 'IPMB_LINUX', the name tcpdump's own table (and pcapkit's generated comment) marks "Legacy names (do not use)", because that row was emitted above the current I2C_LINUX = 209 and aenum gives a value to whichever member is defined first. Fixed in the generator: pcapkit/vendor/reg/linktype.py now sinks any row whose note mentions "legacy" into a bucket appended after every current row, so the current name is always defined first for a value the table double-assigns; today that is value 209 alone. Not breaking: no name disappears, no value changes and no membership changes -- only the value-to-name direction for the shared value moves.
Adds util/pyshark_encap_map.py, a generator that regenerates ENCAP_TYPE_TO_LINKTYPE (152 entries) and FILTER_NAME_TO_LINKTYPE (58) in place inside pcapkit/toolkit/pyshark.py, so the two tables #850 hand-built stop being hand-maintained. Guards against its own worst failure mode: a naive LinkType(dlt) lookup cannot detect an unmapped DLT, because _missing_ mints a placeholder rather than raising, so the generator snapshots every known value before any lookup. The real-tshark sweep is gated on a new HAS_WIRESHARK flag and is version-pinned -- editcap -T accepts 226 encapsulations on Wireshark 4.6.9, 224 on 4.2.2 (CI's Ubuntu noble) -- so the count assertions run only under the measured version. One line of pyshark.py itself changes as a result: the IPMB_LINUX table entry becomes I2C_LINUX, the canonical name #848 gave value 209. util/changelog_md.py regenerated CHANGELOG.md for all seven entries added across this and the six preceding commits (#838, #846, #847, #848, #849, #850, #853); `--check` exit 0.
#846 and #850 - #838's "the 21 whose vendor crawler leaves Vendor.process() unmodified" measures 22, not 21 -- pcapkit.const.hip.transport.Transport also has an unmodified process(), but its FLAG-bound range has no unassigned gap, so its _missing_ carries no bounded-range extend_enum branch to fix. The real discriminator is that branch, not the process() override; reworded, and the "other 84" split into the 83 that do override process() and the one that doesn't but has nothing to fix either. Re-derived independently against 5e25db2^..5e25db2 (not main, which would fold in #847's own edit to ipx/socket.py): 22 registries have no process() override, 21 of them have the bounded-range branch, matching tests/const/test_const_enum_no_mint.py's own REGISTRIES_WITH_UNASSIGNED_RANGES (21 entries) vs ALL_REGISTRIES (22, +hip.transport, with a NOTE explaining the exclusion). - #846's "Only tests/integration/test_engine_runtime.py and test_engine_parity.py change" is false -- 30bca99's own numstat also touches .github/workflows/unit-tests.yml (17+/2-). Qualified to "only these two test files change". - #850 gets the breaking marker: its own prose already says the fallback is gone and an unrecognised encapsulation or filter name now raises MissingKeyError instead of substituting a plausible DLT -- the same shape as the two existing markers at AppType.get (1.5.0.rst:2456) and the four .get()-backed enum fields (1.5.0.rst:2720). Restructured to lead with the marker and the subject, matching their wording and placement; the later restatement of the same fact is dropped. None of these are code or test changes -- prose only, matching the cross-review's own framing. #848 stays unmarked (its own "Not breaking" paragraph holds); #838, #849 and #853 stay unmarked per explicit instruction not to add markers beyond what was asked.
Please follow the guide below
make pylint,make mypy,make isort)make testpasses, and a test case covers the changeWhat is the purpose of your pull request?
fix— corrects a defectDescription of your pull request and other information
Closes #844.
LinkType(209).namereturned'IPMB_LINUX'— the name tcpdump and pcapkit's own generatedcomment label "Legacy names (do not use)" — because that row was emitted above
I2C_LINUX = 209and aenum gives the value to whichever member is defined first. Introduced by
cfdf2ab5c7(2024-05-04), where a regeneration inserted the legacy row above the current one.
The fix is in the generator:
pcapkit/vendor/reg/linktype.pynow sinks any row whose notecolumn mentions "legacy" into a bucket appended after every current row, so the current name is
always defined first for a shared value.
pcapkit/const/reg/linktype.pychanges only as theregenerated consequence — a 3-line move, verified byte-reproducible from the generator.
Verification, re-run by me on this rebased branch rather than taken from the worker's report:
IPMB_LINUXis not removed — it stays an alias, so existingLinkType['IPMB_LINUX']lookups keep working. Only the reverse direction changes.'legacy'matches exactly one row in the whole generated table, so the rule is narrow today. It is deliberately a rule rather than a special case for 209, which means a future tcpdump row noting "legacy" would be sunk too — intended, and the reason the generator carries a comment saying so.coverage run -m pytest tests/const/→ 103 passed, 39458 subtests passedtests/const/test_const_linktype_209_unit.pyagainst unfixedmain→ 3 failed(
'IPMB_LINUX' != 'I2C_LINUX'on both lookups, plus the generator-ordering check)Not
breaking: no name disappears and no value changes; only the value→name direction for 209starts returning the non-deprecated name, which is the defect being fixed.