interface: add the three missing engine constants, and guard dpkt/scapy tracing

[JarryShaw]

Two engine-plumbing defects.

Three of seven engines had no constant

interface/core.py exposed DPKT, Scapy, PCAPKit and PyShark, but not PyPCAP, PCAP_CT or PyPCAPFile — so three shipped engines could only be selected by a bare string while their siblings had a name.

All seven now resolve from pcapkit, pcapkit.interface and pcapkit.all:

pcapkit.DPKT = 'dpkt'          pcapkit.PyPCAP = 'pypcap'
pcapkit.Scapy = 'scapy'        pcapkit.PCAP_CT = 'pcap_ct'
pcapkit.PCAPKit = 'default'    pcapkit.PyPCAPFile = 'pypcapfile'
pcapkit.PyShark = 'pyshark'

That needed three files beyond interface/core.py, because the constants are re-exported by explicit name — pcapkit.PyPCAP would not have existed while pcapkit.DPKT did. Checked for shadowing: pcapkit.foundation does not re-export the engine classes, so the string macros collide with nothing.

docs/source/pcapkit/interface/core.rst claimed the newer engines were string-selected only, and never mentioned PCAP_CT. Now documents all three, and says only runtime-registered engines lack a constant.

The trace guard now covers dpkt and scapy

Its own NOTE said they were "deliberately not listed here… outside the scope of this change". That scope has passed.

Their flow-tracing adapters report each frame as a plain dict, which the PCAP trace dumper cannot re-serialise — it reaches for frame.packet:

engine='dpkt' trace_format=None
  File ".../pcapkit/dumpkit/pcap.py", line 139, in _append_value
      packet=value.packet,
  AttributeError: 'dict' object has no attribute 'packet'

None stays in the format tuple because TraceFlow.__init__ substitutes 'pcap' for it.

dpkt reproduced directly; scapy did not — and that is the interesting part. Its crash was masked by #406: the engine imported only scapy.sendrecv, so every frame dissected as Raw, no TCP layer was found, and the tracer was never fed. With the layer registry loaded (as #409 now does) it crashes identically for None, 'pcap' and 'cap'.

So the guard is written from what the adapters produce, not from which engines happen to crash today. Writing it the other way would have left scapy out and re-broken it the moment #409 landed.

follow_tcp_stream's workaround is kept, having measured both paths

	trace files	warning
follow_tcp_stream(engine='dpkt', format=None)	3	none
Extractor(engine='dpkt', trace_format=None)	3, byte-identical	FormatWarning naming trace_format=None
follow_tcp_stream(engine='dpkt', format='pcap')	—	warns, naming the engine's limitation

Same replacement format, same files. They differ in when they complain: the core guard warns for every substitution including the unset default, while follow_tcp_stream upgrades an unset format silently and warns only on an explicitly unusable one — a distinction already pinned by test_dpkt_unset_trace_format_is_upgraded_quietly.

Removing it would make follow_tcp_stream(engine='dpkt') warn about a default the caller never chose, with a message naming a trace_format= argument the function does not expose. Kept, with its stale NOTE corrected and a test pinning the difference.

Negative controls

A constant that silently falls back is worse than a missing one, so the tests were checked against the bug:

• PyPCAP = 'pypcap-typo' → registry-divergence test fails, and the selection test fails 'PCAP' != 'PyPCAP', catching exactly the silent-fallback case.
• Reverting the guard tuple → 12 subtests fail '.pcap' != '.json', and the end-to-end test fails with the real AttributeError.

Five constants get a real extraction asserting type(extractor.engine).__engine_name__; the other two use stand-in modules, since pcap/pcapfile aren't installed here and pypcapfile is version-gated off on 3.14.

dumpkit/pcap.py is unchanged — the fix belongs in the guard — but a dumpkit test now pins why (PCAPIO raises AttributeError on a mapping), so the guard's justification is verified rather than asserted in a comment.

Verification

Fixture-free CI selection: 660 passed, 8 skipped against a 644/8 baseline measured on the unmodified worktree — the delta is the new tests, no new skips, no failures.

Rebased onto merged main, and I checked the thing most likely to break: this branch predates #409's rename of the scapy test in tests/interface/test_misc.py, and a clean apply could have silently reverted it. It didn't — the new name is present and the old one absent, matching main.

[Copilot]

🟡 Changes recommended

The new engine-constant tests use stub modules for pcap-ct/pcapfile that aren’t marked as packages (missing __path__), which can break importlib.import_module('pcap._pcap') / import pcapfile.linklayer during selection tests.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR addresses two engine-selection/tracing plumbing gaps in pcapkit: it adds missing public engine constants for shipped engines, and extends the constructor-level trace-format guard so dict-frame engines (including DPKT/Scapy) don’t route to the PCAP trace dumper path that can’t serialize mappings.

Changes:

• Add engine constants for PyPCAP, PCAP_CT, and PyPCAPFile, and re-export them from pcapkit, pcapkit.interface, and pcapkit.all.
• Guard Extractor(..., trace_format in {None,'pcap','cap'}) for dict-frame engines (dpkt, scapy, pyshark, pypcapfile) by substituting a dict-capable format (JSON) and warning.
• Add/adjust tests and docs to pin constant reachability/selection and the trace-format substitution behavior (including an explicit unit pin for why follow_tcp_stream keeps its local upgrade behavior).

File summaries

File	Description
tests/interface/test_misc.py	Adds a regression test pinning why follow_tcp_stream keeps its local format upgrade semantics.
tests/interface/test_core.py	Adds tests asserting engine-constant completeness, re-exports, and that constants select the intended engine.
tests/foundation/test_extraction.py	Adds constructor-level and end-to-end tests for trace-format substitution on dict-frame engines.
tests/dumpkit/test_common_unit.py	Adds a unit test pinning that the PCAP dumper can’t serialize mapping-shaped frames.
pcapkit/interface/misc.py	Updates rationale/comments around the local trace-format upgrade behavior in follow_tcp_stream.
pcapkit/interface/core.py	Adds missing engine constants and clarifies invariant that constant values must match registry keys.
pcapkit/interface/init.py	Re-exports the new engine constants from pcapkit.interface.
pcapkit/foundation/extraction.py	Extends the dict-frame tracing guard to include dpkt and scapy.
pcapkit/all.py	Exposes new engine constants via pcapkit.all.__all__.
pcapkit/init.py	Exposes new engine constants via pcapkit.__all__.
docs/source/pcapkit/interface/core.rst	Documents the new engine constants and clarifies runtime-registered engines have no constant.

Review details

Suppressed comments (1)

tests/interface/test_core.py:350

• The pcapfile stub module needs to be marked as a package (set __path__) or import pcapfile.linklayer/savefile/structs inside PyPCAPFile.__init__ can raise ModuleNotFoundError: 'pcapfile' is not a package, even with the submodules pre-seeded in sys.modules.

        package = types.ModuleType('pcapfile')
        savefile = types.ModuleType('pcapfile.savefile')
        savefile.load_savefile = lambda *a, **kw: FakeSaveFile()  # type: ignore[attr-defined]
        linklayer = types.ModuleType('pcapfile.linklayer')
        linklayer.clookup = lambda linktype: FakeDecoded          # type: ignore[attr-defined]

• Files reviewed: 11/11 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[JarryShaw]

Following up on the review summary, which raises a second stub beyond the one in the inline thread — import pcapfile.linklayer alongside importlib.import_module('pcap._pcap'). Both were checked by running them, and neither needs __path__. No change made.

The import system resolves sys.modules before it consults a parent package, so a preregistered submodule is returned without any path search. That holds for the statement form as well as importlib:

import sys, types
pkg = types.ModuleType('pcapfile')            # deliberately no __path__
sub = types.ModuleType('pcapfile.linklayer')
pkg.linklayer = sub
sys.modules['pcapfile'] = pkg
sys.modules['pcapfile.linklayer'] = sub
import pcapfile.linklayer                     # works

Measured on 3.10, 3.12 and 3.14: ok, statement form works without __path__ on each. The pcap._pcap case was checked the same way across 3.10 through 3.14 and behaves identically.

Both stubs do register every submodule they need — tests/interface/test_core.py:361-362 for pcapfile and its three submodules, and the is_pcap_ct branch for pcap._pcap.

Worth noting that the test already documents the requirement that does exist here, which is a different one — the parent needs the submodules bound as attributes, because the engine reaches them through the package object it stored in _expkg rather than by importing them again:

# Both bindings are needed: ``import pcapfile.savefile`` is satisfied by
# sys.modules, but the engine then reaches the submodules as *attributes*
# of the package it stored in ``_expkg``.

That is why package.savefile, package.linklayer and package.structs are assigned. Adding __path__ would address a failure mode that cannot occur while leaving that real one — already handled — unremarked.

tests/interface/test_core.py: 13 passed, 7 subtests passed.