An unassigned PCAP-NG Block Type raises a bare ValueError from aenum, so UnknownBlock is unreachable and one unknown block costs the whole extraction

[JarryShaw]

Found while fixing #678 (PR #699), and deliberately left out of it: the site is
pcapkit/corekit/fields/numbers.py, not the PCAP-NG schema, and the fix changes every protocol's
enum handling rather than one module's.

An unassigned PCAP-NG Block Type cannot be parsed at all, and says so with a bare ValueError

ValueError: 28 is not a valid BlockType

not one of pcapkit.utilities.exceptions, so a caller cannot tell it from a bug in its own code, and
not an EOFError, so Extractor.record_frames does not catch it. One unknown block therefore costs
the whole extraction — the same consequence #678 has, from a different cause.

Measured on f0999858e and unchanged by #699. A well-formed 16-octet block whose only unusual
property is a type nobody has assigned:

SHB     = struct.pack('<IIIHHqI', 0x0A0D0D0A, 28, 0x1A2B3C4D, 1, 0, -1, 28)
IDB     = struct.pack('<IIHHII',  0x00000001, 20, 1, 0, 0, 20)
UNKNOWN = struct.pack('<III', 28, 16, 0) + struct.pack('<I', 16)   # type 28, length 16

Extractor(<SHB + IDB + UNKNOWN>, nofile=True, store=True)
# builtins.ValueError in_library=False: 28 is not a valid BlockType

The traceback stops in the field layer, before any PCAP-NG code sees the value:

  File "pcapkit/protocols/schema/schema.py", line 860, in unpack
    value = field.unpack(byte, packet.copy())
  File "pcapkit/corekit/fields/field.py", line 528, in unpack
    return self.post_process(value, packet)
  File "pcapkit/corekit/fields/numbers.py", line 513, in post_process
    return self._namespace(value)
  File ".../aenum/_enum.py", line 2276, in __new__
    raise ValueError("%r is not a valid %s" % (value, cls.__name__))

Why this is a defect rather than a rejection working as intended

The library already has the answer and cannot reach it.
pcapkit/protocols/schema/misc/pcapng.py's BlockType schema declares

__default__ = lambda: UnknownBlock

and pcapkit/protocols/misc/pcapng.py carries a whole _read_block_unknown reader for it. Measured:
Schema_BlockType.registry.default_factory() is UnknownBlock. That default is unreachable for any
genuinely unassigned type — the enum rejects the value several frames earlier — so it only ever fires
for the Reserved_* ranges BlockType._missing_ auto-extends:

BlockType(0x0bad0bad) -> <BlockType.Reserved_0bad0bad: 195890093>    # in a Reserved range, extended
BlockType(0x1c)       -> ValueError: 28 is not a valid BlockType     # in range, unassigned
BlockType(0xffff)     -> ValueError: 65535 is not a valid BlockType

The format requires the opposite. The PCAP-NG specification is explicit that a reader must skip a
block whose type it does not recognise, which is the entire reason block length is repeated at both
ends. UnknownBlock is that skip; it is what the block type registry's default exists to select.

get()'s default contract does not rescue it. EnumField.post_process calls
self._namespace(value) — the enum constructor — rather than self._namespace.get(value, default),
so the generated get()'s except ValueError fallback is never in the path at all. The sibling
OptionEnumField.post_process in pcapkit/protocols/schema/misc/pcapng.py does go through
get():

    def post_process(self, value, packet):
        value = super(EnumField, self).post_process(value, packet)
        return self._namespace.get(value, namespace=self._opt_ns)

which is why an unassigned option type resolves to UnknownOption while an unassigned block type
raises. The asymmetry is the bug.

How often it is reachable

A 4,000-round bounded mutation fuzz over the committed examples/captures/dhcp.pcapng — word-aligned
length and type corruptions, each case capped at the original size and run under a 2 GiB
RLIMIT_AS — hits it 46 times on PR #699's tree, and it is present in the same fuzz on
f0999858e too, so it pre-dates that change. It is the largest remaining family of foreign exceptions
in that sweep; the only other is MemoryError, at exactly 30 in both trees, which is #593's 32-bit
band and a separate matter.

Scope, and what the fix has to decide

Not fixed in #699 because pcapkit/corekit/fields/numbers.py is shared by every protocol in the
package, so changing EnumField.post_process moves the behaviour of every enum-typed wire field at
once — every unassigned value that currently raises would start resolving to something. That wants its
own review, its own breaking argument, and a decision on two points:

1. Where the fallback goes. EnumField.post_process going through get() fixes it everywhere at
   once but changes every protocol. Overriding post_process for PCAP-NG's block type field alone
   fixes the reported case and leaves the asymmetry everywhere else.
2. What an unassigned value should become. get(value, default) needs a default, and for a
   registry with no "unknown" member the honest answer may be an extended member (Unassigned_28,
   as several registries already do) rather than a sentinel.

Either way the exception it does raise should be in-library: #647 documents that 113 of 117
generated _missing_ guards raise a bare ValueError, and this is the same leak reached through a
different door — aenum's own raise, from the return super()._missing_(value) that
BlockType._missing_ ends with, so it is not even pcapkit's text.

Related

• Every EOF-truncated PCAP-NG file raises an uncaught ValueError: pcapng_block_selector passes a negative __length__ to SchemaField #678 / fix(pcapng): stop a negative length reaching a read on a truncated capture (#678) #699 — the same consequence (one malformed block costs the whole extraction) from the
  negative-length side. This is what the fuzz found next, once that was closed.
• the generated enum guards raise a bare ValueError in 113 of 117 const modules, and tcp.flags.Flags has no guard at all #647 — the bare-ValueError census over the generated enum guards. Closed, and about the exception
  type; this is about a lookup that should not fail at all.
• fix(corekit): bound the total zero padding a parse may synthesise (#573) #593 / A 16-bit padding shortfall band is still unbudgeted after #593: a crafted capture amplifies 1,637x, indistinguishable from a truncated one at the field layer #594 — the amplification band, which is the other foreign family the same fuzz still reaches.