apt-get in the engine-tests install step has no timeout and no retries, so a
single mirror stall consumes the whole 30-minute job budget and surfaces as
cancelled — which correctly fails Required checks passed and blocks the PR.
Hit twice within forty minutes, on #971 and #972, both docs-only:
run 36887906077 Engines Python 3.13 (PCAP_CT) cancelled 15:57:45 .. 16:28:00 (30m15s)
run 36888646396 Engines Python 3.10 (PCAP_CT) cancelled 16:01:29 .. 16:31:47 (30m18s)
The log pins it precisely. apt-get update fetched InRelease metadata, then stalled:
15:58:06 Get:5 https://archive.ubuntu.com/ubuntu noble-security InRelease [126 kB]
16:27:58 ##[error]The operation was canceled.
29m52s of silence, then the timeout-minutes: 30 at unit-tests.yml:327 killed the
job. Not version-specific (3.13 and 3.10) and not leg-specific: on the same run the
other five PCAP_CT legs finished in 39–81 seconds.
Nothing in either diff can affect an engine's system-package install, so this is
purely infrastructural — but it is expensive, because one stalled leg of six turns a
green PR red and the 30-minute burn delays everything behind it.
Fix is on the apt invocations at unit-tests.yml:403-417: -o Acquire::Retries=3 -o Acquire::http::Timeout=30 so a stalled mirror fails fast and retries, plus a
step-level timeout-minutes well under the job's 30 so a hang cannot eat the budget.
Worth noting Required checks passed behaved exactly right here — refusing to treat
cancelled as a pass is the property #966 and #970 added tests for.
apt-getin theengine-testsinstall step has no timeout and no retries, so asingle mirror stall consumes the whole 30-minute job budget and surfaces as
cancelled— which correctly failsRequired checks passedand blocks the PR.Hit twice within forty minutes, on #971 and #972, both docs-only:
The log pins it precisely.
apt-get updatefetched InRelease metadata, then stalled:29m52s of silence, then the
timeout-minutes: 30atunit-tests.yml:327killed thejob. Not version-specific (3.13 and 3.10) and not leg-specific: on the same run the
other five
PCAP_CTlegs finished in 39–81 seconds.Nothing in either diff can affect an engine's system-package install, so this is
purely infrastructural — but it is expensive, because one stalled leg of six turns a
green PR red and the 30-minute burn delays everything behind it.
Fix is on the apt invocations at
unit-tests.yml:403-417:-o Acquire::Retries=3 -o Acquire::http::Timeout=30so a stalled mirror fails fast and retries, plus astep-level
timeout-minuteswell under the job's 30 so a hang cannot eat the budget.Worth noting
Required checks passedbehaved exactly right here — refusing to treatcancelledas a pass is the property #966 and #970 added tests for.