protocols: honour layer=/protocol= limits, and forward the packet context

[JarryShaw]

Closes #356. Closes #382.

(Filing both explicitly — a comma list only auto-closes the first, which bit #403.)

#356 — layer= and protocol= were completely inert

ProtocolBase.__init__ read the limits from _layer/_protocol, but no producer in the tree spells them that way: both engines, all four _import_next_layer implementations, and the public extract(layer=…, protocol=…) that the CLI feeds all pass them un-prefixed. They landed in **kwargs, were dropped, _sigterm stayed False, and the parse always ran to the top.

Measured on frame 4 of http6.cap, whose full chain is Ethernet:IPv6:TCP:HTTP/1.1:

argument	before	after
layer='link'	Ethernet:IPv6:TCP:HTTP/1.1	Ethernet:Internet_Protocol_version_6
layer='internet'	Ethernet:IPv6:TCP:HTTP/1.1	Ethernet:IPv6:TCP
layer='transport'	Ethernet:IPv6:TCP:HTTP/1.1	Ethernet:IPv6:TCP:Raw
protocol='TCP'	Ethernet:IPv6:TCP:HTTP/1.1	Ethernet:IPv6:TCP:Raw

PCAP-NG behaved the same way. CLI: pcapkit-cli -v -L internet went from the full chain to Ethernet:IPv6:TCP.

Fixed at ProtocolBase.__init__, which now accepts both spellings with the prefixed one winning — the single point every producer funnels through, so it repairs all six at once plus any third-party protocol. Guarded to the parse path (file is not None), because on the construction path protocol is a real make() argument (IPv4.make takes one), so consuming it there would drop the value being built. Also normalised the 'none'/'null' no-limit sentinels that Extractor.__init__ substitutes, so they are not compared against real protocol names on every protocol of every packet.

The CLI's forwarding was already correct — the core ignored it. But -L defaulted to the string 'None' and -P to 'null', working only because Extractor happened to lowercase the former into its sentinel; both now default to None. -L gained choices + type=str.lower, so -L bogus exits with usage rather than being silently ignored (-L Internet still works). -P is deliberately left unvalidated, since protocols can be registered dynamically.

#382 — the packet context never reached the schema

_import_next_layer hands the next protocol its context as packet=; the schema layer reads __packet__. Nothing bridged them, so every schema's unpack/post_process saw {}. IPv4.read/IPv6.read are the only reads declaring __packet__, and both always received None.

A second, chained bug had to be fixed for the value to survive. Schema.unpack cleared __updated__ before prepare ran post_process, and every field post_process assigns sets the flag again — so the schema was left dirty, the next bytes()/len() re-packed it, and Schema.pack called post_process a second time with a context rebuilt from the schema's own fields, overwriting exactly the context-derived values. OptionField.unpack measures each option with len(data), which tripped it one option later. Schema.pack already orders these correctly; the unpack path now matches.

Demonstrated with a hand-built IPv6 + HOPOPT carrying an MPL option whose Seed-ID RFC 7731 elides, so the seed is the outer source address:

	before	after
HOPOPT schema saw packet keys	[]	['dst', 'src']
MPLOption saw packet keys	[len, next, type]	['dst','len','next','src','type']
MPL seed_id	None	IPv6Address('2001:db8::1')

The context is passed as a shallow copy — Schema.unpack writes every field it reads plus __length__/__option_padding__ into it, and the IPv6 extension-header walk hands one dict to each header in turn, so sharing it would leak a sibling's fields into the next header's ConditionalField tests. An explicitly supplied __packet__ (the PCAP-NG engine's snaplen) is never overridden.

Verification

tier	baseline	after
fixture-free CI selection	575 passed, 14 skipped	594 passed, 14 skipped
tests/integration	70 passed, 3 skipped	85 passed, 2 skipped
*_runtime.py / *_regression.py	47 passed	47 passed

Zero failures. All 25 new assertions fail against the unmodified source — verified by reverting the three source files and re-running (25 failed / 42 passed), with a deliberate control (test_no_limit_parses_to_the_top_of_the_stack) that passes both ways so the suite cannot pass vacuously.

Un-skipped test_layer_and_protocol_limits_stop_the_parse, the only parse-limit skip, which now passes with its assertions unchanged. The two remaining integration skips are unrelated dictdumper blockers.

Parse benchmark (http.pcap, 1117 frames, best of 5): 1.0661s → 1.0266s. mypy: identical 9 pre-existing errors, only line numbers shifted.

Two scope notes

• pcapkit/utilities/decorators.py is touched, which was outside the original brief. The __updated__ ordering fix has to live there: post_process is invoked by prepare, after Schema.unpack returns, so schema.py cannot reach it.
• tests/integration/test_cli_subprocess.py had a blind spot, now fixed. It ran python -m pcapkit in a temp cwd, so the subprocess imported the installed editable package — the main checkout — rather than the tree under test. The new CLI assertions passed in-process and failed in subprocess until the collected tree went on the subprocess PYTHONPATH. Worth knowing generally: any prior CLI change made in a worktree was asserted against code it never ran.

Left alone, reported not fixed

ipv6_route.py's RPL consumer of __packet__ is still unreachable, confirming the caveat in #382: _read_data_type_rpl tests header.length % 16 and _read_data_type_src tests (header.length - 8) % 16, comparing the raw Hdr Ext Len octet — which counts 8-octet units — against octet counts. So a 1–3-address source route, or any RPL header without a multiple of 8 addresses, is rejected before the context matters. Separate defect, separate change.

[Copilot]

🟢 Approval recommended

The changes address well-scoped, demonstrated defects in core parsing/context propagation with strong unit + integration coverage and no issues found in review.

Pull request overview

This PR fixes two core parsing issues in PyPCAPKit’s protocol stack: (1) extraction limits (layer= / protocol=) were previously ignored due to keyword spelling mismatches, and (2) per-packet context was not reaching schema unpack/post-processing (packet= never bridged to __packet__). It also aligns schema “dirty” flag handling during unpack with the existing pack path to prevent unintended repacks and context-derived value clobbering.

Changes:

• Make ProtocolBase.__init__ honour both _layer/_protocol and layer/protocol (prefixed wins) and normalize “no limit” sentinels.
• Republish packet= as a shallow-copied __packet__ during parsing so schema unpack/post_process receives the enclosing-layer context.
• Fix prepare decorator to leave schemas clean after post_process, and harden/extend CLI + integration/unit test coverage for these behaviors.

File summaries

File	Description
pcapkit/protocols/protocol.py	Accept unprefixed parse-limit kwargs during parsing; normalize sentinels; bridge packet → __packet__ with a copy.
pcapkit/utilities/decorators.py	Clear schema __updated__ after post_process in the unpack path to prevent unintended repacks and value overwrites.
pcapkit/main.py	CLI: default -L/-P to None, validate -L with choices + case-folding, keep -P free-form.
tests/utilities/test_decorators.py	Add targeted regression tests for prepare leaving schemas clean and tolerating non-schema returns.
tests/protocols/test_protocol_base_unit.py	Add unit tests for parse-limit keyword spelling, sentinel normalization, and packet→__packet__ forwarding (including a real MPL/IPv6 seed-id consumer).
tests/integration/test_frame_iteration.py	Unskip/expand end-to-end assertions proving limits stop parsing at the correct layer/protocol and preserve raw payload verbatim above the stop.
tests/integration/test_cli_subprocess.py	Ensure subprocess CLI tests execute against the checkout under test (PYTHONPATH), and add CLI assertions for -L/-P behavior and -L validation.
tests/cli/test_main.py	Pin CLI forwarding contract for layer/protocol, and verify defaults + layer case-insensitivity + invalid layer rejection.

Review details

• Files reviewed: 8/8 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.