ci(unit-tests): stop the integration job re-running the unit tier - #727
Conversation
- Replace the `integration` job's bare `pytest -q` (the whole suite) with a positive selection: `tests/integration`, every `*_runtime.py`/ `*_regression.py` file, plus the exact test methods that skip a generated capture in the `test` job and only run for real once fixtures exist. Computed at run time via a new `tests._tiers.fixture_tier_paths()`, not spelled out as literal flags, so the workflow cannot drift from it. - Add `tests._tiers.skip_idiom_modules()`/`skip_idiom_test_ids()`, which find those skip-idiom call sites via the AST machinery the guard already has, and resolve each to a precise pytest node ID so pulling in one skip-idiom test does not also pull in its module's unrelated ~100 other tests. - Extend `WorkflowAgreementTests` (`tests/test_tier_guard.py`) so the guard also polices the `integration` job's positive selection, not just the `test` job's `--ignore`/`--ignore-glob` flags; add `EnclosingScopeTests` and `FixtureTierSelectionTests` (including a grep-based check textually independent of the AST machinery) to pin the new machinery, plus a canary for a capture name it cannot see statically. - Leave the `gate` job's full-suite run untouched: it is a deliberate independent re-verification for the release path, not part of the per-PR-push duplication this change targets. Measured: the skip-idiom set is exactly 3 methods across 2 files (`test_dpkt_unit.py:444,812`, `test_pcapng_unit.py:3058`). Under the exact `.[test,Scapy]` venv the `integration`/`gate` jobs install, the new selection collects 173 tests and runs 142 passed/31 skipped, 0 failed -- 2 of the 3 recovered methods still skip there (pre-existing, unrelated: `dpkt` sits behind its own extra that job never installed) but no test runs twice and nothing regresses. `tests/test_tier_guard.py`: 37 passed, 57 subtests.
2dddb1c to
3afdfcd
Compare
|
✅ GOOD TO MERGE @ Cross-review ran on opus (no substitution needed) and returned GOOD TO GO. Posted here by the coordinator: that reviewer was a read-only helper with no GitHub write grant, so its verdict existed only in a hand-back report. Independently verified before posting:
Two corrections the reviewer asked for were applied after it reported, so its GOOD TO GO covers the pre-amend state; the delta is exactly those two fixes (corrected 161→142 passed / 31 skipped, plus the second textually-independent guard test at Known conflict: #725 also edits this job's run step. Merge order and resolution are in the PR body. ✅ GOOD TO MERGE @ |
Please follow the guide below
You will be asked some questions, please read them carefully and answer honestly
Put an
xinto all the boxes [ ] relevant to your pull request (like that [x])Use Preview tab to see how your pull request will actually look like
Searched for similar pull requests
Followed the coding style (
make pylint,make mypy,make isort) -- N/A: those targets coverpcapkit/only. Ranisort --check-only(clean) andpy_compileby hand on both changed.pyfiles.make testpasses, and a test case covers the change -- not run in full.tests/test_tier_guard.py: 36 passed, 57 subtests. New selection, run for real in a venv built from the exactpip install -e '.[test,Scapy]'theintegration/gatejobs use: 142 passed, 31 skipped, 0 failed (see caveat below on the skip count). Unit job's own selection over the 2 affected files: 131 passed, 3 skipped.Added a changelog entry -- N/A — changelog centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657
What is the purpose of your pull request?
ci— workflows or build toolingDescription
Part of #715.
integrationran the whole suite on every matrix version despitetestalready running the unit tier for real on the same commit (~130 min/PR push wasted). Fix:integrationpytest -q(whole suite)tests._tiers.fixture_tier_paths()gatepytest -qtests/integration+ every*_runtime.py/*_regression.pyfile + precise node IDs for tests that only skip (not read) a generated capture intest.test_dpkt_unit.py:444,812,test_pcapng_unit.py:3058. Resolved to node IDs, not whole modules, so they don't drag in ~95 unrelated tests.test_dpkt_unit.py, whose classes carry@unittest.skipUnless(HAS_DPKT, ...).dpktsits behind the separateDPKTextra, which neitherintegrationnorgateinstalls (.[test,Scapy]) — true before this PR too, so not a regression, but it means only thetest_pcapng_unit.pynode ID actually executes for real in CI today; the 2dpktones stay skipped there exactly as before, and only run for real in a fuller venv (or once/ifDPKTis added to that install line, a separate, out-of-scope change). The selection is still correct and forward-compatible either way.tests/_tiers.pyat CI run time, not hand-copied into YAML — can't drift.WorkflowAgreementTestsnow polices this structurally; verified it fails against a deliberately regressed workflow.timeout-minutes: 45untouched (CI: raise timeout-minutes on the three pytest jobs from 30 to 45 #713/ci(unit-tests): raise pytest job timeouts from 30 to 45 minutes #716's territory) — with this partitionintegrationdrops to ~2 min/version, so 45 is generous headroom, not a tight fit.Composes with #725 (open,
ci-xdist-drop-py315-matrix): both editintegration's "Run full test suite" step. On merge, keep this PR's guard-check structure and add-n auto --dist loadto the finalpytest -q $selectionline. #725's "Report available parallelism" step is untouched and stays put.Not closing #715 — item 1 (
purge_modulesto class scope) and item 2 (stopgatere-running on main pushes) remain open.