Skip to content

fix(vendor): source ExtensionHeader from the extension-header registry, not the protocol-numbers flag (#925) #926

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JarryShaw merged 1 commit into from
Sep 29, 2026
Merged

fix(vendor): source ExtensionHeader from the extension-header registry, not the protocol-numbers flag (#925) #926

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions docs/source/pcapkit/protocols/internet/ipv6_ext.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,9 @@ so those two octets are parseable without knowing anything else about
the header. See the module docstring below for the closed exception
table (``IPv6-Frag`` and ``AH`` each use their own length rule; ``ESP``
has a dedicated parser whose own info reports no next header rather than
lacking one, and ``BIT-EMU``, ``253`` and ``254`` have no dedicated
parser at all -- none of the four ever reaches this class), the two ways
this class is dispatched to, and why an overrun stops the walk instead of
clipping it.
lacking one, and ``253`` and ``254`` have no dedicated parser at all --
none of the three ever reaches this class), the two ways this class is
dispatched to, and why an overrun stops the walk instead of clipping it.

.. autoclass:: pcapkit.protocols.internet.ipv6_ext.IPv6_Ext
:no-members:
Expand Down
13 changes: 5 additions & 8 deletions pcapkit/const/ipv6/extension_header.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,13 @@ class ExtensionHeader(EnumRegistry, IntEnum):
#: HOPOPT, IPv6 Hop-by-Hop Option [:rfc:`8200`]
HOPOPT = 0

#: IPv6-Route, Routing Header for IPv6 [Steve Deering]
#: IPv6-Route, Routing Header for IPv6 [:rfc:`8200`][:rfc:`5095`]
IPv6_Route = 43

#: IPv6-Frag, Fragment Header for IPv6 [Steve Deering]
#: IPv6-Frag, Fragment Header for IPv6 [:rfc:`8200`]
IPv6_Frag = 44

#: ESP, Encap Security Payload [:rfc:`4303`]
#: ESP, Encapsulating Security Payload [:rfc:`4303`]
ESP = 50

#: AH, Authentication Header [:rfc:`4302`]
Expand All @@ -47,11 +47,8 @@ class ExtensionHeader(EnumRegistry, IntEnum):
#: Shim6, Shim6 Protocol [:rfc:`5533`]
Shim6 = 140

#: BIT-EMU, Bit-stream Emulation [:rfc:`9801`]
BIT_EMU = 147

#: Use for experimentation and testing [:rfc:`3692`]
#: Use for experimentation and testing [:rfc:`3692`][:rfc:`4727`]
Use_for_experimentation_and_testing_253 = 253

#: Use for experimentation and testing [:rfc:`3692`]
#: Use for experimentation and testing [:rfc:`3692`][:rfc:`4727`]
Use_for_experimentation_and_testing_254 = 254
32 changes: 19 additions & 13 deletions pcapkit/protocols/internet/ipv6.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,26 +82,32 @@ class IPv6(IP[Data_IPv6, Schema_IPv6],
#: :class:`IPv6_Ext` by *direct* registration instead (see the
#: bottom of :mod:`pcapkit.protocols.internet.ipv6_ext`), which
#: already produces exactly this class without needing this set to name
#: it. ``ESP``, ``BIT-EMU``, ``253`` and ``254`` are absent, but not for
#: the same reason as each other, and not because a generic fallback
#: would help them:
#: it. ``ESP``, ``253`` and ``254`` are absent, but not for the same
#: reason as each other, and not because a generic fallback would help
#: them:
#:
#: * ``ESP`` *does* have a dedicated, registered parser
#: (:class:`~pcapkit.protocols.internet.esp.ESP`) -- it is excluded
#: because :rfc:`4303` places the real Next Header byte inside the
#: encrypted trailer, so its own info always *carries* a ``next``
#: attribute (unlike ``BIT-EMU``/``253``/``254`` below), just one that is
#: attribute (unlike ``253``/``254`` below), just one that is
#: :data:`None` whenever the payload could not be decrypted -- which,
#: with no key material available to a generic parse, is always. The
#: :meth:`_decode_next_layer` walk below still ends there, one iteration
#: later, because :data:`None` fails
#: :class:`~pcapkit.const.ipv6.extension_header.ExtensionHeader`'s
#: constructor at the top of the loop -- the *existing* end-of-chain
#: path, unrelated to the structural check this set exists for.
#: * ``BIT-EMU``, ``253`` and ``254`` have no dedicated parser at all, so
#: they resolve to plain :class:`~pcapkit.protocols.misc.raw.Raw`, whose
#: info has no ``next`` *attribute* -- this is what the structural check
#: catches.
#: * ``253`` and ``254`` have no dedicated parser at all, so they resolve
#: to plain :class:`~pcapkit.protocols.misc.raw.Raw`, whose info has no
#: ``next`` *attribute* -- this is what the structural check catches.
#: (``BIT-EMU``/147 used to sit here too, until GitHub issue #925 found
#: it was never in IANA's authoritative extension-header registry to
#: begin with; :class:`~pcapkit.const.ipv6.extension_header
#: .ExtensionHeader` no longer carries it, so a next-header byte of 147
#: now fails that same constructor at the *top* of the loop instead --
#: the ordinary end-of-chain path any unrecognised upper-layer protocol
#: code already takes, one step earlier than it used to.)
#:
#: :meth:`_decode_next_layer`'s walk stops cleanly at whichever of these
#: (or any other IANA code this package has not implemented) it meets,
Expand Down Expand Up @@ -421,8 +427,8 @@ def _decode_next_layer(self, ipv6: 'Data_IPv6', proto: 'Optional[int]' = None,
# which also carries ``next`` (possibly :data:`None`, on an
# overrun -- see its module docstring). Every IANA extension
# header code this package has not implemented a dedicated
# parser for -- today that is ``BIT-EMU``, ``253`` and ``254``,
# and tomorrow it is whatever IANA assigns next -- has no
# parser for -- today that is ``253`` and ``254``, and tomorrow
# it is whatever IANA assigns next -- has no
# generic fallback either (see :attr:`__generic_ext_codes__`'s
# docstring for why), so :meth:`_import_next_layer` returns a
# plain :class:`~pcapkit.protocols.misc.raw.Raw`, whose info
Expand Down Expand Up @@ -502,9 +508,9 @@ def _import_next_layer(self, proto: 'int', length: 'Optional[int]' = None, *, #
method's own* behaviour for anything outside that closed set is
exactly what it was before this method learned the
substitution: a plain ``Raw`` for that one layer. What changed
for ``BIT-EMU``, ``253`` and ``254`` -- which have no dedicated
parser at all, so they were *already* reaching plain ``Raw``
with no exception involved -- is one level up:
for ``253`` and ``254`` -- which have no dedicated parser at
all, so they were *already* reaching plain ``Raw`` with no
exception involved -- is one level up:
:meth:`_decode_next_layer` now stops its walk structurally on
any layer whose info carries no ``next`` attribute, ``Raw``
included, instead of reading ``info.next`` unconditionally and
Expand Down
30 changes: 15 additions & 15 deletions pcapkit/protocols/internet/ipv6_ext.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,19 +57,19 @@
encrypted trailer (:rfc:`4303`), so its own
info's ``next`` is :data:`None` rather than a
value to continue on
``BIT-EMU``, ``253``, ``254`` terminal -- no dedicated parser exists, so no
``253``, ``254`` terminal -- no dedicated parser exists, so no
next header field is ever read at all, either
======================================================= ===========================================

This table classifies by *wire format* alone, over the twelve codes
:class:`~pcapkit.const.ipv6.extension_header.ExtensionHeader` enumerates.
Note that IANA's own *IPv6 Extension Header Types* registry has **eleven**:
the twelfth, ``BIT_EMU`` (147), comes from this package generating that
enumeration out of the *Protocol Numbers* registry's extension-header
column instead, where 147 is flagged ``Y`` while the extension-header
registry omits it. That discrepancy is GitHub issue #925 and is not this
class's to resolve; the classification below holds either way, since 147
is not reachable through here.
This table classifies by *wire format* alone, over the eleven codes
:class:`~pcapkit.const.ipv6.extension_header.ExtensionHeader` enumerates --
matching IANA's own *IPv6 Extension Header Types* registry exactly, as of
GitHub issue #925. Before that fix this package generated the enumeration
out of the *Protocol Numbers* registry's extension-header column instead,
which carried a twelfth code, ``BIT_EMU`` (147), that the authoritative
registry does not; 147 was never reachable through this class either way
(it had no dedicated parser and was never generic-dispatched here), so
fixing the enumeration's source changed nothing this table classifies.

``Shim6`` conforms to it (:rfc:`5533`), but this package has never
had a dedicated parser class for it to begin with -- see "Two entry paths"
Expand All @@ -83,18 +83,18 @@
Next Header inside the encrypted trailer, with no length field anywhere
in the cleartext part; and 253/254 are reserved for private
experimentation (:rfc:`3692`) with no wire format at all. None of the
four is reachable through this class, by construction -- see
three is reachable through this class, by construction -- see
:meth:`pcapkit.protocols.internet.ipv6.IPv6._import_next_layer`. Each of
them still enters :meth:`IPv6._decode_next_layer
<pcapkit.protocols.internet.ipv6.IPv6._decode_next_layer>`'s walk (it is a
real :class:`~pcapkit.const.ipv6.extension_header.ExtensionHeader` member),
but the four part ways there: ``ESP`` resolves to its own dedicated parser,
but the three part ways there: ``ESP`` resolves to its own dedicated parser,
whose info carries a ``next`` that is simply :data:`None` -- so the walk
ends the ordinary way, ``ExtensionHeader(None)`` failing at the top of the
next iteration, exactly as it did before this class existed. ``BIT-EMU``,
``253`` and ``254`` have no dedicated parser and resolve to plain
next iteration, exactly as it did before this class existed. ``253`` and
``254`` have no dedicated parser and resolve to plain
:class:`~pcapkit.protocols.misc.raw.Raw`, whose info has no ``next``
*attribute* at all; for these three (and any future IANA code nobody has
*attribute* at all; for these two (and any future IANA code nobody has
implemented yet) the walk stops on a *structural* check -- does the parsed
layer carry a ``next`` at all? -- rather than on a list of codes.

Expand Down
94 changes: 63 additions & 31 deletions pcapkit/vendor/ipv6/extension_header.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,53 @@ class {NAME}(EnumRegistry, IntEnum):
class ExtensionHeader(Vendor):
"""IPv6 Extension Header Types"""

#: Keyword-style names carried over from this crawler's *previous* data
#: source, keyed by protocol number. The Protocol Numbers registry
#: (``protocol-numbers-1.csv``, this crawler's :attr:`LINK` before GitHub
#: issue #925) paired each of these headers with a short ``Keyword``
#: column value, e.g. ``IPv6-Route`` for header 43. The registry
#: :attr:`LINK` now points at -- IANA's authoritative *IPv6 Extension
#: Header Types* registry -- has no such column, only a verbose
#: ``Description`` (``Routing Header for IPv6`` for the same header), so
#: deriving names the same way :meth:`process` does for every other
#: 3-column registry (see e.g. :class:`~pcapkit.vendor.ipv6.router_alert.
#: RouterAlert`) would silently rename these members. This mapping keeps
#: the existing, shorter names instead; any header not listed here still
#: falls back to a name derived from its description, same as always.
NAMES = {
0: 'HOPOPT',
43: 'IPv6-Route',
44: 'IPv6-Frag',
50: 'ESP',
51: 'AH',
60: 'IPv6-Opts',
139: 'HIP',
140: 'Shim6',
} # type: dict[int, str]

#: Link to registry.
LINK = 'https://www.iana.org/assignments/protocol-numbers/protocol-numbers-1.csv'
#:
#: .. note::
#:
#: Until GitHub issue #925, this pointed at the *Protocol Numbers*
#: registry (``protocol-numbers/protocol-numbers-1.csv``), filtered on
#: its ``IPv6 Extension Header`` column -- a derived signal, not the
#: registry :rfc:`8200#section-4` names as authoritative for this
#: enumeration. That registry disagreed with this one on header 147
#: (``BIT-EMU``): it flagged 147 as an IPv6 extension header, citing
#: :rfc:`9801`, while this registry omits 147 entirely -- see the issue
#: for the reading of :rfc:`9801` that makes the omission look
#: intentional rather than an erratum. Fixing :attr:`LINK` also let
#: :class:`~pcapkit.const.ipv6.extension_header.ExtensionHeader` drop
#: ``BIT_EMU``: :attr:`pcapkit.protocols.internet.ipv6.IPv6
#: ._decode_next_layer`'s walk used to rely on ``ExtensionHeader(147)``
#: resolving, but its own test (``tests.protocols.internet
#: .test_ipv6_ext_unit.IPv6ExtUnitTests
#: .test_unimplemented_terminal_code_stops_the_walk_not_the_packet``)
#: now exercises the identical code path on 253 -- a code this
#: registry *does* list -- so nothing outside this package depends on
#: 147 resolving any more.
LINK = 'https://www.iana.org/assignments/ipv6-parameters/extension-header.csv'

def count(self, data: 'list[str]') -> 'Counter[str]':
"""Count field records.
Expand All @@ -68,8 +113,9 @@ def count(self, data: 'list[str]') -> 'Counter[str]':
"""
reader = csv.reader(data)
next(reader) # header
return collections.Counter(map(lambda item: self.safe_name(item[1] or item[2]),
filter(lambda item: len(item[0].split('-')) != 2, reader)))
return collections.Counter(
map(lambda item: self.safe_name(self.NAMES.get(int(item[0]), item[1])),
filter(lambda item: len(item[0].split('-')) != 2, reader)))

def process(self, data: 'list[str]') -> 'tuple[list[str], list[str]]':
"""Process CSV data.
Expand All @@ -87,12 +133,12 @@ def process(self, data: 'list[str]') -> 'tuple[list[str], list[str]]':
enum = [] # type: list[str]
miss = [] # type: list[str]
for item in reader:
flag = item[3]
if flag != 'Y':
continue
code_str = item[0]
desc = item[1]
rfcs = item[2]

name = item[1]
rfcs = item[4]
keyword = self.NAMES.get(int(code_str)) if code_str.isdigit() else None
name = keyword or desc

temp = [] # type: list[str]
for rfc in filter(None, re.split(r'\[|\]', rfcs)):
Expand All @@ -101,42 +147,28 @@ def process(self, data: 'list[str]') -> 'tuple[list[str], list[str]]':
temp.append(f'[:rfc:`{rfc[3:]}`]')
else:
temp.append(f'[{rfc}]'.replace('_', ' '))
lrfc = re.sub(r'( )( )*', ' ', f" {''.join(temp)}".replace('\n', ' ')) if rfcs else ''

subd = re.sub(r'( )( )*', ' ', item[2].replace('\n', ' '))
tmp1 = f' {subd}' if item[2] else ''

split = name.split(' (', 1)
if len(split) == 2:
name, cmmt = split[0], f" ({split[1]}"
else:
name, cmmt = name, '' # pylint: disable=self-assigning-variable

if name:
tmp1 = f',{tmp1}' if tmp1 else ''
else:
name, tmp1 = item[2], ''
desc = self.wrap_comment(f'{name}{tmp1}{lrfc}{cmmt}')
name_part = f'{keyword}, {desc}' if keyword else desc
comment = self.wrap_comment(re.sub(r'\r*\n', ' ', '%s %s' % ( # pylint: disable=consider-using-f-string
name_part, ''.join(temp) if rfcs else '',
), flags=re.MULTILINE))

try:
code, _ = item[0], int(item[0])
if not name:
name, desc = item[2], ''
renm = self.rename(name, code, original=item[1])
code, _ = code_str, int(code_str)
renm = self.rename(name, code, original=keyword)

pres = f"{renm} = {code}"
sufs = f"#: {desc}"
sufs = f"#: {comment}"

#if len(pres) > 74:
# sufs = f"\n{' '*80}{sufs}"

#enum.append(f'{pres.ljust(76)}{sufs}')
enum.append(f'{sufs}\n {pres}')
except ValueError:
start, stop = item[0].split('-')
start, stop = code_str.split('-')

miss.append(f'if {start} <= value <= {stop}:')
miss.append(f' #: {desc}')
miss.append(f' #: {comment}')
miss.append(f" return extend_enum(cls, '{self.safe_name(name)}_%d' % value, value)")
return enum, miss

Expand Down
Loading
Loading