Repository navigation
fix(vendor): make the LinkType legacy-sink rule value-aware - #854
Conversation
|
NEEDS CHANGES on
Two-row table: a single-value row
The fix is provably free, which is the part that settles it: patching the pre-scan to expand Everything else confirmed, several beyond what I had checked:
One claim disputed as attributed: order-independence holds, but Also worth fixing while in there: the three new |
a302c96 to
f8b13b3
Compare
|
NEEDS CHANGES on 1. One genuine hole, and it is the same class of defect this PR exists to close. Same for 2. A false clause in the PR description. It claims a malformed numeric column now raises "earlier — now 3. Two of my own framings were refuted, both in this PR's favour, and I re-measured both before relaying:
Confirmed: the regression is fixed (head emits
|
Follow-ups from the cross-review of #848's legacy-sink rule in pcapkit/vendor/reg/linktype.py: - The sink predicate tested a row's notes for the word "legacy" alone, with no check that the row's value was actually claimed by another row -- so a future current row whose notes coincidentally mention "legacy" would be wrongly sunk. Gate the sink on the row's value being a genuine duplicate elsewhere in the table (pre-scanned via a Counter), computed independent of row order. - The range branch (USER0-USER15) reused the same per-row sink, so one range row worded "legacy" would sink all sixteen expanded members at once. Range rows now always land in enum, unconditionally. - Reworded the generator comment so it matches what the predicate actually tests, not "shares its value" when it never checked that. - Added a comment at the const file's emission site, as a plain `#` line rather than `#:`, so it explains the source layout without leaking into IPMB_LINUX's rendered Sphinx docstring; qualified the self-reference as pcapkit.vendor.reg.linktype.LinkType.process. Two rounds of cross-review on this fix itself then each caught a narrower-than-main-loop hole in the duplicate-value pre-scan: - Round 1: the pre-scan counted only single-value rows, so a value duplicated across a range boundary (a single-value row sharing a value with one member of a USER0-style range) was invisible to it. The pre-scan now expands en-dash ranges the same way the main loop does before counting. - Round 2: the pre-scan's single-value branch tested str.isdigit(), narrower than the main loop's int(temp), which also accepts a leading sign and PEP 515 underscores. The pre-scan now tries int(temp) directly, before the en-dash check, so it recognises exactly what the main loop does. Neither hole is a live defect -- every one of the 220 committed members matches a plain unsigned decimal -- but each was the same class of silent under-count this fix exists to close for ranges, narrowed further. Also fixed: the order-independence test previously proved nothing beyond what the 209 fixture already covered (mutating the pre-scan into a prefix-only, order-dependent variant left it passing); it now runs the same duplicate pair through process() in both orders itself. And code_int (already parsed by int(temp)) is now reused at the sink = line instead of re-parsing code a second time. Added tests for: a non-duplicated value never sunk regardless of wording, a duplicate pair resolving correctly in either table order, range-sink isolation, the cross-range duplicate regression, a malformed en-dash range still raising loudly in the main loop (not the pre-scan), and signed/underscored duplicate pairs -- each shown failing against the relevant prior code. Regenerated pcapkit/const/reg/linktype.py at every round; each pre-scan widening alone is byte-identical (md5 7f4db8d612d74fbdbc880f75040bff12) to the prior fix, confirmed by isolating it from the docstring-comment change, which is the only other diff. Invariants hold: LinkType(209).name == 'I2C_LINUX', IPMB_LINUX and I2C_LINUX both alias value 209, 220 members / 219 canonical iteration, 209 the only duplicated value, USER0-USER15 still 147-162. tests/const + tests/vendor: 196 passed; tests/test_tier_guard.py: 102 passed (unittest-confirmed).
f8b13b3 to
bb20fe8
Compare
|
NEEDS CHANGES on The md5 at body line 100 is wrong. It cites The claim itself is true — comment-stripped, the head's const file is byte-identical to Note this is a stronger result than the The valuable new finding: Completeness confirmed structurally and empirically. Also confirmed: the two new tests fail on Three residuals recorded, none blocking and none introduced here: a unicode-digit row would emit |
|
GOOD TO GO on The wrong md5 is corrected and re-derived rather than pasted. Verified independently by me: Byte-identity, not a hash coincidence. The body also now records what the third review found and neither the author nor I had spotted: Three residuals recorded in the body, none fixed here and none introduced by this PR: a unicode-digit row would CI on this head: 58 CheckRuns pass, 0 fail, 0 incomplete. Three rounds of cross-review, all on opus against a |
Please follow the guide below
make pylint,make mypy,make isort)make testpasses, and a test case covers the changeWhat is the purpose of your pull request?
fix— corrects a defectDescription of your pull request and other information
Closes #852 — the four follow-ups from #848's cross-review, plus two regressions this PR's own
cross-review found across two rounds. None was a live defect: a live fetch of
tcpdump.org/linktypes.html(205 rows) matches the old rule on exactly one row. These close theways it could go wrong later.
1. The predicate is now value-aware. #848 sank any row whose notes mentioned "legacy", never checking
whether the value was actually duplicated — so a future current row worded "supersedes the legacy
DLT_FOO"would have been sunk and recreated #844 in mirror image.
process()now pre-scansdatawith aCounterto build
dup_values, computed independent of row order, and sinks only whencode_int in dup_values and 'legacy' in cmmt.lower().Round 1 of that pre-scan only counted single-value rows (filtering the numeric column on
str.isdigit()), so a value duplicated across a range boundary — a single-value row sharing a valuewith one member of a
USER0-style range — was invisible to it, and that row's legacy alias stoppedbeing sunk:
main-equivalent behaviour for such a pair emits['USER0', ..., 'OLDUSER'](currentcanonical), the value-blind pre-scan emitted
['OLDUSER', 'USER0', ...](legacy canonical, silentlywrong) — exactly the failure mode #844 and #852 exist to close, reopened by narrowing the predicate.
The pre-scan now expands
a–b(en dash) ranges the same way the main loop does before counting, so arange-boundary duplicate is no longer missed.
Round 2's fix still recognised a set of single values that was both narrower and wider than the main
loop's:
str.isdigit()rejects a leading sign (+209,-5) and PEP 515 underscore grouping (2_09)that the main loop's own
int(temp)accepts, but it also accepts a Unicode decimal digit character(e.g.
'²⁰⁹'.isdigit()isTrue) thatint()rejects — which on the previous head crashed thepre-scan with an uncaught
ValueErrorraised from inside theCountergenerator expression itself,before the main loop ever ran. The single-value branch now tries
int(temp)directly, before theen-dash check, so it recognises exactly what the main loop does in both directions — closing the
under-count for a signed/underscored value and the pre-scan-level crash for a Unicode-digit one at the
same time. Not reachable today (every one of the 220 committed members matches a plain unsigned
decimal), same as the range case above.
An ASCII hyphen is still never treated as a range, so a malformed numeric column (
900-903,900–903–905, an empty string) keeps raising loudly — in the main loop, atstart, stop = map(int, temp.split('–')), the same statement as before this PR. The pre-scan's own_expandswallows every one of those forms and contributes no counted values for the row; it does notitself raise, and does not change where the real raise happens.
2. The range branch no longer routes through
sink.USER0–USER15(147–162) append straight toenumunconditionally, so one range row worded "legacy" can no longer move all sixteen expanded members at once.
Unaffected by the range-aware pre-scan above:
dup_valuesis only consulted in the single-value branch.3. The comment now describes what the code tests. The old one claimed a value-sharing check the predicate
never performed; it now documents value-duplication plus wording, at both the
dup_valuescomputation andthe
sink =line, and documents the range-expansion and signed/underscored recognition the pre-scan doesbefore counting.
4. The out-of-order member is explained in the generated file (
const/reg/linktype.py), so a readerwho finds
IPMB_LINUX = 209afterDECT_NR_TAP = 304sees why. That note is emitted as a plain#commentrather than
#:— a#:line immediately above an attribute becomes that attribute's rendered Sphinxdocstring, and this note is about the generator's own source layout, not about what
IPMB_LINUXmeans to acaller. It also names the fully-qualified
pcapkit.vendor.reg.linktype.LinkType.processrather than thebare
LinkType.process, which is ambiguous with the const class of the same name.Also fixed: the order-independence test previously proved nothing beyond what the 209 fixture already
covered — mutating the pre-scan into a prefix-only, order-dependent variant left it passing, because that
variant is only wrong for the first occurrence of a value it has not seen yet. It now drives the same
duplicate pair through
process()in both orders itself, so the property is earned by the test named for itrather than borrowed from
LinkTypeGeneratorLegacyOrderingTests's separate 209 fixture. Also: the mainloop's
code_int(already parsed byint(temp)) is now reused at thesink =line instead of re-parsingcodea second time.Verification, re-run by me on the current head, freshly regenerated
the value-blind, word-only predicate (fix(reg): emit tcpdump legacy link-type names after current ones #848 as merged,
f990f2149), with the current 11-test file:2 failed, 9 passed —
test_non_duplicated_legacy_wording_does_not_reorderandtest_legacy_worded_range_row_still_lands_entirely_in_enum. Against round 1's range-blind pre-scan:test_legacy_row_duplicating_a_range_member_is_sunkfails, emitting['OLDUSER', 'USER0', 'USER1', 'USER2', 'USER3']instead of['USER0', 'USER1', 'USER2', 'USER3', 'OLDUSER']. Against round 2'sstr.isdigit()-gated pre-scan:test_signed_value_pair_resolves_to_the_current_memberandtest_underscored_value_pair_resolves_to_the_current_memberboth fail, emitting['OLD', 'CUR']instead of
['CUR', 'OLD']. On this head: 11 passed (Ran 11 tests … OKunder plainunittest).coverage run -m pytest tests/const tests/vendor→ 196 passed, 39647 subtests passedtests/test_tier_guard.py→ 102 passed, 553 subtests — no new dependency gate; the file continues toguard on the existing
HAS_VENDOR_DEPSlive table and no member is spelled with a sign, an underscore, or a Unicode digit, so
dup_valuesstays
{209}throughout — the range-aware, then the signed/underscored-aware, pre-scan change nothingabout which rows get sunk. Isolated from the item-4 docstring-comment rewording (the only other
change to the generated file), this head's committed const file with exactly its four new plain-
#comment lines removed is byte-identical to merged
main's:git cat-file blob origin/main:pcapkit/const/reg/linktype.py | md5sumand this head's blob with those four linesstripped both give md5
8210ae8084b10a8ea1e5f14c9266b35a, and a directdiffbetween the two isempty. (A stronger result than the all-comment-stripped
d5d673b7324b9cd14f4eb1a8d0b5aa43postedearlier on this PR: removing only the four new lines already reproduces
mainexactly, with no needto strip every
#line to get there — which is what "notbreaking" below actually rests on.) Withthe four comment lines included, the full const file regenerates reproducibly at md5
9bf2f4b49c394df180beb4eff51d562f— confirmed to be exactly those four lines' worth of diff frommerged
main, nothing else.One limitation stated rather than papered over
A wording-only signal still cannot save a genuine duplicate pair if tcpdump attaches "legacy" to the wrong
member of that pair.
dup_valuesnarrows the rule to real duplicates — including, now, duplicates that span arange boundary or are spelled with a sign or an underscore — but if upstream ever words the current row of
a true pair as the legacy one, no amount of value-awareness detects it: the only signal available is the
prose. That residual is documented in the generator's comment rather than left implicit.
Three more, all pre-existing or deliberate, none introduced here and none needing a code change:
emission, not recognition, is the weak link for a non-ASCII decimal-digit row (e.g. Arabic-Indic
digits) —
int()accepts such digits and recognises the value correctly, butcode = tempreuses theraw text verbatim in the emitted
NAME = codeliteral, and Python source numeric literals areASCII-only, so the generated file would still raise a
SyntaxErroron import; pre-existing (code = temppredates this PR) and unreachable today, since every live row's number column is a plain ASCIIdecimal. A legacy range row still cannot be sunk, since ranges never route through
sinkat all —deliberate, item 2 of #852. And
_expandmaterialiseslist(range(...))where the main loop is lazy,so a corrupt huge range would
MemoryErrorin the pre-scan rather than hang — hypothetical either way,and no more or less true of one branch than the other.
Not
breaking:LinkType(209).nameisI2C_LINUXbefore and after, no name disappears, no valuechanges — and, per the byte-identity result above, the only change to the committed const file at all
is the four-line comment.