scapy engine dissects nothing: it imports scapy.sendrecv, which does not load scapy's layer registry #406

engine='scapy' returns every frame as Raw unless the calling program happens to have imported scapy.all itself. The engine therefore delivers none of the dissection it exists to provide, and does so silently.

Reproduction

pcapkit/foundation/engines/scapy.py imports only the sniffing module:

42:        import scapy.sendrecv
64:        from scapy import sendrecv as scapy  # isort:skip

scapy.sendrecv does not load scapy's layer registry. Two fresh processes, same capture, same call:

# fresh
ex = pcapkit.extract(fin='examples/captures/in.pcap', engine='scapy', store=True, nofile=True)
ex.frame[0]  ->  Raw
# WARNING: PcapReader: unknown LL type [1]/[0x1]. Using Raw packets

# with `import scapy.all` first
ex.frame[0]  ->  Ether / IPv6 / ICMPv6ND_NS / ICMPv6 Neighbor Discovery Option ...

Link type 1 is plain Ethernet — about as ordinary as a capture gets — and scapy parses it perfectly well once scapy.layers.l2 is registered. Without the registry, PcapReader cannot map the link type and falls back to Raw for every packet.

The same shows through follow_tcp_stream:

fresh process                : 0 streams
after `import scapy.all`     : 3 streams   (matches the default engine)

Why this went unnoticed

It is order-dependent in the test suite, which makes it look like flakiness rather than a defect:

$ pytest tests/interface/test_misc.py                        ->  7 passed
$ pytest tests/toolkit tests/interface/test_misc.py          ->  1 failed
      AssertionError: 3 != 0
      test_scapy_engine_finds_no_streams_for_this_capture

Something under tests/toolkit imports enough of scapy to populate the registry, so the later test then sees correct dissection and fails an assertion that expects the broken result. It passes in isolation and passes in the full CI selection, so CI never sees it.

And the assertion is backwards. #402 added test_scapy_engine_finds_no_streams_for_this_capture on the rationale that scapy genuinely cannot dissect this capture — "a documented capability gap". That rationale is wrong: 0 is the polluted answer produced by the missing registry, and 3 is the truthful one. I wrote that test, so the mistake is mine; the polluted value looking like the plausible one is exactly what made it convincing.

Suggested fix

Import scapy such that the layer registry is populated — import scapy.all is the conventional way, or explicitly import the layer modules the engine needs (scapy.layers.l2, .inet, .inet6, …) if the full import is too heavy. Note pcapkit/toolkit/scapy.py already imports scapy.layers.inet and .inet6 lazily inside functions, so the pieces are partly reachable already — but not before PcapReader decides how to parse the link layer, which is where it matters.

Two things to settle in the fix:

• Import cost. scapy.all is slow to import and pulls in a great deal, including a CryptographyDeprecationWarning from the TLS layer. If that is unacceptable, importing just the layer modules is the narrower option — but it needs verifying against captures with less common link types, not just Ethernet.
• Correct the test. test_scapy_engine_finds_no_streams_for_this_capture should assert the dissected result (3 streams, matching default) and be renamed accordingly. Better still, add an assertion that frame 0 is not Raw, which fails loudly for this root cause rather than for a stream count.

Worth checking whether engine='pyshark' or the other third-party engines have an equivalent partial-import problem.

Environment

main at 89201dff6, Python 3.14, scapy 2.7.0, examples/captures/in.pcap (link type 1, 6 frames).