Make the legacy smoke scripts runnable again

[JarryShaw]

Twelve of the seventeen runnable demonstrations in examples/legacy_smoke/ worked; the rest had rotted against the current API. Nothing in CI runs them, so nothing noticed — testpaths = ["tests"] means these files are never collected.

API drift

• test_analyse.py passed tcp=True, reasm_strict=True but not reassembly=True, and Extractor.reassembly is gated on _flag_r. It parsed all 26 frames of its capture and then died with UnsupportedCall.
• test_api.py had the same omission latently: its ip/tcp/reasm_strict knobs did nothing, and it passed only because it never touched .reassembly.
• Two commented-out lines in test_engine.py referenced engine='pipeline' and engine='server', neither of which exists in Extractor.__engine__ any more.
• test_time.py had unused top-level import dpkt / import pyshark / import scapy.all that would kill the script on any host missing one.

I audited every extract() call in the directory against inspect.signature(pcapkit.extract): no unknown keywords remain, and every reassembly knob is now paired with reassembly=True.

Honest degradation, and a trap worth knowing

test_engine.py and test_time.py died partway through on pyshark, which asks for an implicit asyncio event loop that Python 3.14 no longer provides — pyshark's bug, not pcapkit's, and tshark is absent here too. A demo that dies partway demonstrates nothing, so engines that work still run and report, and an unavailable one is reported as skipped with its reason.

That needed more care than it looks. Extractor does not raise when an engine's package is missing — it warns and silently falls back to its own parser, so the naive version credited PCAP's work to dpkt. The demos now name the driver that actually ran. Verified by blocking dpkt at the import hook:

    dpkt: its package is not installed -- pcapkit fell back to PCAP; 6 frames -> ../captures/engines/dpkt.txt

Classification lives in a new _engine_support.py and covers exactly three cases — ImportError, a missing tshark, and the event-loop RuntimeError. Anything else re-raises, so a real pcapkit failure is still loud.

test_perf.py needs pyperf, which pcapkit does not depend on; it now prints what to install and points at test_time.py, which does the same engine comparison with nothing extra. I deliberately did not add pyperf to Pipfile/pyproject.toml — it is a benchmarking harness for one demo, and it would land in every contributor's install for something CI never runs.

test_profile.py was also broken and was not on the list — found by running it. cProfile will not create its output directory, and only make profile did mkdir -p temp, so python test_profile.py profiled the whole run and then died at the dump. It makes its own temp/ now.

Regenerating the committed fixtures is now one command

make fixtures in examples/legacy_smoke/, documented in its README:

TZ=UTC python test_extractor.py   # out.json, out.plist, out.txt  <- in.pcap
TZ=UTC python test_pcapng.py      # pcapng.txt                    <- dhcp.pcapng

Both inputs are committed, so it works on a fresh clone with no make samples.

Why TZ=UTC: these fixtures were unreproducible because pcapkit renders frame timestamps in the host's zone — and the four committed files disagreed with each other, the PCAP three having been generated at UTC−05:00 and pcapng.txt at UTC+08:00 (both reproduced exactly under those zones to confirm). No single zone reproduces all four. UTC is the only choice that is reproducible anywhere, self-consistent across the set, and — for PCAP-NG — yields the correct epoch. Regenerated here against current main, so they will move again when #365–#368 land.

Two corrections to what was originally reported: out.plist and out.txt were not stale, and pcapng.txt's drift was not purely cosmetic — its timestamp_epoch value changed, which is #361.

A footgun fixed

test_stream.py and test_stream_askpass.py passed buffer_path='../captures/stream.pcap' — a generated fixture the runtime tests pin byte-for-byte. Running either would have silently broken the suite. They write to a fresh tempfile.mkdtemp() now, printed at startup. The macOS-only en0 is a named INTERFACE constant with a note (Linux is usually eth0/wlan0). Neither script is executable here (they need sudo tcpdump), so both were byte-compiled rather than run.

Verification

17/17 runnable scripts exit 0 (was 12/17), executed against current main with the renamed ../captures/ paths. Library suite unchanged: 508 passed, 4 skipped, 315 subtests.

Reported, not fixed

Noisy [ERROR]-level logs on entirely normal input: MissingKeyError: <ExtensionHeader.IPv6_Frag: 44> ×12 during ordinary IPv6 fragment reassembly, and MissingKeyError: <OptionType.if_tsoffset: 14> ×4 on a capture that simply lacks an optional option. That is #362. Also [WARNING] not enough values to unpack 2–10 times per script from httpv1.py:278-280, which also emits thousands of DeprecationWarnings about positional maxsplit.

[Copilot]

🟡 Changes recommended

The new examples/legacy_smoke/_engine_support.py uses from __future__ import annotations, which breaks imports on Python 3.6 despite the repository declaring >=3.6 support.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Restores the “legacy smoke” demonstration scripts under examples/legacy_smoke/ so they run successfully against the current pcapkit.extract(...) API, including correct handling of reassembly flags and engine availability, and updates the committed output fixtures with a reproducible timezone.

Changes:

• Fix legacy smoke scripts to match current extractor/reassembly behavior and avoid hard failures on missing optional engine dependencies.
• Add shared engine availability / fallback detection helper (_engine_support.py) and wire it into engine/timing/benchmark demos.
• Document how to run demos and regenerate committed fixtures; update committed fixtures accordingly.

File summaries

File	Description
examples/legacy_smoke/_engine_support.py	New shared helper for engine preflight/unavailability and detecting silent fallback.
examples/legacy_smoke/test_engine.py	Runs each engine, skips unavailable ones, and reports the actual driver used.
examples/legacy_smoke/test_time.py	Adds engine preflight and skips unavailable engines before timing loops.
examples/legacy_smoke/test_perf.py	Makes pyperf optional and benchmarks only engines that pass preflight.
examples/legacy_smoke/test_profile.py	Ensures the cProfile output directory exists when run standalone.
examples/legacy_smoke/test_stream.py	Buffers live capture to a temp file to avoid overwriting pinned fixtures.
examples/legacy_smoke/test_stream_askpass.py	Same as test_stream.py, but supports sudo -A askpass flow.
examples/legacy_smoke/test_api.py	Aligns reassembly knobs with reassembly=True requirement.
examples/legacy_smoke/test_analyse.py	Aligns reassembly knobs with reassembly=True requirement.
examples/legacy_smoke/README.md	Expanded documentation for running demos, dependencies, and fixture regeneration.
examples/legacy_smoke/Makefile	Adds fixtures target with pinned TZ and configurable interpreter.
examples/captures/out.json	Regenerated committed fixture output (formatting + timestamp rendering).
examples/captures/out.plist	Regenerated committed fixture output (timestamp rendering).
examples/captures/out.txt	Regenerated committed fixture output (timestamp rendering).
examples/captures/pcapng.txt	Regenerated committed fixture output (timestamp_epoch + structural output changes).

Review details

• Files reviewed: 15/15 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🔵 Needs a closer look

The updated legacy_smoke README currently references a non-existent ../samples/make_samples.py path and should be corrected to the actual generator location before merge.

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

examples/legacy_smoke/test_time.py:41

• For timing, time.time_ns() is wall-clock time and can jump backwards/forwards (e.g., NTP adjustments), which can yield noisy or even negative deltas. Use the monotonic time.perf_counter_ns() for duration measurement instead.

        now = time.time_ns()

        extraction = pcapkit.extract(fin='../captures/in.pcap', store=False, nofile=True, verbose=False, engine=engine)  # type: ignore[arg-type]

        delta = time.time_ns() - now

examples/legacy_smoke/README.md:23

• The sample-generation command uses python ../samples/make_samples.py, but there is no examples/samples/ directory in this repo. The generator script lives under examples/generators/make_samples.py, so from examples/legacy_smoke/ the correct relative path is ../generators/make_samples.py.

• Files reviewed: 15/15 changed files
• Comments generated: 0 new
• Review effort level: Lite

[JarryShaw]

Both suppressed comments from the review body checked. One was real, one was stale.

Real — README.md:23. It said python ../samples/make_samples.py, left over from before examples/samplegen/ was renamed to examples/generators/. There is no examples/samples/, so the one command a fresh clone must run before the other scripts work was the one command that couldn't. Fixed to ../generators/make_samples.py, and verified by running it from examples/legacy_smoke/ — the directory the README tells you to be in.

Stale — test_time.py:41. Copilot filed this as "previously missed, in code that hasn't changed since the last review", but the file has used the monotonic time.perf_counter_ns() since 00e10390b, with a comment recording exactly the reason given:

# NOTE: perf_counter_ns is monotonic; time_ns is wall clock and can step
now = time.perf_counter_ns()
...
delta = time.perf_counter_ns() - now

No change needed there.