Skip to content

ci(deps): narrow the all extra to core addons, add a dev extra (#910) - #914

Merged
JarryShaw merged 1 commit into
mainfrom
ci/910-narrow-all-add-dev
Sep 29, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
ci/910-narrow-all-add-dev

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

Please follow the guide below

  • You will be asked some questions, please read them carefully and answer honestly

  • Put an x into all the boxes [ ] relevant to your pull request (like that [x])

  • Use Preview tab to see how your pull request will actually look like

  • Searched for similar pull requests

  • Followed the coding style (make isort verified clean; pcapkit/ itself is untouched, so make pylint/make mypy are unaffected)

  • make test passes, and a test case covers the change — ran tests/test_tier_guard.py (109) and tests/project/ (178) directly per instruction, not the full suite; two new tests cover the new all/dev shape and were shown to fail without this change

  • Added a changelog entry — N/A, changelog centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657

What is the purpose of your pull request?

  • ci — workflows or build tooling

Description of your pull request and other information

Implements the owner's ruling on #910: all narrows from 8 requirements to
3 core addons (cli, crypto, pycrate/NGAP); the four 3rd-party capture
engines it used to bundle (dpkt, scapy, pyshark, pypcapfile) become on-demand
extras, same as PyPCAP/PCAP_CT always were. pypcap/pcap-ct stay out
of everything.

A new dev extra carries what all used to for CI's own need: pylint,
mypy and Sphinx's autodoc resolve imports against what is installed, and
lint.yml's comment tracks exactly 8 import-error messages from the two
engines already absent by version marker. Measured with a real editable
install: that count holds at 8 (identical messages) with .[all,dev], and
the docs build holds at 42 warnings (byte-identical text) either side.

lint.yml, deploy-pages.yml and cron-conda.yml's conda-update job move
to .[all,dev]. cron-vendor.yml's existing .[all,vendor] already covers
what left all (verified). create-release.yml's and cron-conda.yml's
conda-dist job's bare .[all] installs are untouched — traced through to
confirm neither has a downstream consumer of the removed engines.

Updates the three test modules and two docs pages (+README) that described
the old all, and adds two new tests in tests/test_tier_guard.py proving
the new shape against a real pyproject.toml parse.

@JarryShaw JarryShaw added ci Pull requests that change CI or workflow configuration (ci: subject prefix) chore Maintenance work: tooling, repo hygiene, no library behaviour change breaking Breaks public-facing behaviour or API (apply alongside the type label) review: pending No verdict for the current head - never reviewed, or the head moved since the last one and removed chore Maintenance work: tooling, repo hygiene, no library behaviour change labels Sep 29, 2026
@JarryShaw
JarryShaw force-pushed the ci/910-narrow-all-add-dev branch from b1535e7 to a91c706 Compare September 29, 2026 06:44
@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO on a91c70658 — cross-review on fable (author's model unrecorded at dispatch, so I cannot assert it differed; stated rather than glossed). One comment fix applied by me since; extras resolve byte-identically.

The claim that mattered most — does narrowing all break anything at runtime — is falsified as a risk. No unguarded module-scope import of the four engines exists; the one module-scope case, pcapkit/toolkit/scapy.py:49-52, is a try/except ModuleNotFoundError. Extractor.run calls import_test, warns "engine … is not installed; using default engine instead", and sets self._exnam = 'default'. So a [all]-only user gets graceful degradation, and README.md:41 and docs/source/index.rst:311-316 now say so outright. That was the one genuine user-visible consequence of the ruling and it is deliberate.

Independently reproduced: import-error 8 → 8, byte-identical (pcap×2, pcapfile×6) — the reviewer built a scratch venv and ran pylint rather than trusting the count. The two bare .[all] sites are safe: both sit in conda-deployment jobs whose only downstream steps read the static conda/requirements.txt and run an isolated conda build .. And the conda-dist.py fix is causal, not incidental — my brief doubted it because appdirs is in neither extra, but it arrives transitively via pyshark ∈ dev, verified with pip show appdirs.

One comment was wrong and this PR was duplicating it, so I fixed both copies. They said installing pypcapfile on 3.12+ is "actively harmful, since engine='pypcapfile' then raises ModuleNotFoundError". It does not raise: Extractor.run consults PyPCAPFile.unsupported_reason() before the import test (extraction.py:545-551) and degrades with a warning. The guard exists precisely because a bare import pcapfile succeeds on 3.12+ — so the comments described the reason the guard is there as though it were current behaviour. One copy was pre-existing on the PyPCAPFile extra, the other newly copied into dev.

Sequencing, and my own brief was wrong here. I told the reviewer #912's pep.rst change was a pure rename that would merge cleanly. It is R098 — a rename with a content edit at old line 492, immediately adjacent to this PR's rewrite at @@ -490,8. git merge-tree confirms CONFLICT (content): Merge conflict in docs/source/contributing/pep.rst. README.md and index.rst auto-merge. Resolution is trivial — keep this PR's rewritten sentence plus #912's /-prefixed :doc: path — but merge #912 first and I will rebase this one, rather than discovering it at the button.

Unverified, stated as such: the base-tree Sphinx count (the PR tree measures 42 on a fresh build with and without PCAPKIT_DEVMODE, so DEVMODE is falsified as the cause of the 42-vs-58 gap I had proposed); and the origin of #912's 58 remains unidentified.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: pending No verdict for the current head - never reviewed, or the head moved since the last one labels Sep 29, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Resolve conflicts.

The owner's ruling on #910: `all` should carry only the core addons that
make the library work at full functionality (`cli`, `crypto`, `pycrate` for
NGAP), not every 3rd-party capture engine an end user might want. Drops
dpkt, scapy, pyshark, pypcapfile, requests and beautifulsoup4 from `all` (8
requirements to 3); each moves to its own on-demand extra, same as
PyPCAP/PCAP_CT always were. `pypcap`/`pcap-ct` stay out of everything,
unchanged.

Adds a `dev` extra carrying what `all` used to, for CI's different need:
pylint/mypy/Sphinx resolve imports against what is installed, so narrowing
`all` would otherwise have grown lint.yml's tracked 8 `import-error`
messages. Measured with a real editable install before and after: pylint's
import-error count holds at 8 (identical messages), and the docs build
holds at 42 warnings (identical text).

lint.yml, deploy-pages.yml and cron-conda.yml's conda-update job move to
`.[all,dev]`. cron-vendor.yml's `.[all,vendor]` already covers what left
`all`, verified with a real resolve. create-release.yml's and
cron-conda.yml's conda-dist job's bare `.[all]` are untouched -- neither has
a downstream consumer of the removed engines (`conda/requirements.txt` is a
separately pinned list). Updates the three test modules and two docs pages
(plus README) that described the old `all`, and adds two tests proving the
new shape against a real pyproject.toml parse.

Build: 109 tests in tests/test_tier_guard.py and 178 in tests/project/ pass.

Corrected two comments that asserted `engine='pypcapfile'` raises
ModuleNotFoundError on Python 3.12+. It does not: `Extractor.run` consults
`PyPCAPFile.unsupported_reason()` *before* the import test
(`pcapkit/foundation/extraction.py:545-551`) and degrades to the default engine
with a warning. That guard exists because a bare `import pcapfile` succeeds on
3.12+, so an import-only check would let the error escape -- which is what the
comments were describing, but as current behaviour rather than as the reason the
guard is there. One copy was pre-existing on the `PyPCAPFile` extra; the other
this change had duplicated into `dev`. Comments only: extras resolve byte-identically
and `tests/test_tier_guard.py` still passes 109 tests.
@JarryShaw
JarryShaw force-pushed the ci/910-narrow-all-add-dev branch from a91c706 to 23d2c13 Compare September 29, 2026 13:09
@JarryShaw

Copy link
Copy Markdown
Owner Author

Rebased onto main after #912 merged — 23d2c13b6. This one had a real content conflict and I resolved it; the resolution is the only judgement call, so here it is explicitly.

The clash was the paragraph in pep.rst describing the engine extras. #912 had rewritten its :doc: path to root-relative; this PR had rewritten the surrounding prose because #910 narrowed all:

HEAD (main, post-#912):  ``PyPCAPFile``, which ``all`` includes, and
                         ``PyPCAP``, which it deliberately does not
                         :doc:`/pcapkit/foundation/engines/index`

this PR:                 ``PyPCAPFile`` and ``PyPCAP``, neither of which
                         ``all`` includes -- ``PyPCAP`` for the installability
                         reason below, and ``PyPCAPFile`` because GitHub issue
                         #910 narrowed ``all`` to core addons only
                         :doc:`pcapkit/foundation/engines/index`

Kept this PR's prose and main's leading slash. The prose because main's version is now factually wrong — all no longer includes PyPCAPFile, which is the whole point of this change — and the slash because the file lives under contributing/ and a relative :doc: there would resolve against the wrong directory.

Verified the resolution introduced nothing else. Comparing a91c70658 to 23d2c13b6: seven of the ten files are blob-identical, including pyproject.toml, all four workflows, and both test files. The three that differ do so only by #912's own edits — pep.rst by exactly eleven leading-slash additions, README.md by the contributing/testing.html URL, index.rst by the split toctree. My resolved paragraph is byte-identical to the version you reviewed.

Extras confirmed unchanged after the rebase: all = ['emoji', 'cryptography>=3.4', 'pycrate'], dev carrying the four engines plus requests[socks]/beautifulsoup4[html5lib]. tests/test_tier_guard.py passes 109/109. Every :doc: in pep.rst is now root-relative — checked, none left bare.

review: good-to-go carries, since the only delta from the reviewed head is other people's merged work.

@JarryShaw
JarryShaw merged commit 3766c3c into main Sep 29, 2026
63 checks passed
@JarryShaw
JarryShaw deleted the ci/910-narrow-all-add-dev branch September 29, 2026 13:56
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Sep 29, 2026
@JarryShaw JarryShaw added this to the 1.5 milestone Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Breaks public-facing behaviour or API (apply alongside the type label) ci Pull requests that change CI or workflow configuration (ci: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant