Skip to content

Keep registry stalls and slow device loads from failing the Maestro tests - #153

Draft
rjhuijsman wants to merge 2 commits into
mainfrom
rjh.maestro-npm-stall-hardening
Draft

Keep registry stalls and slow device loads from failing the Maestro tests#153
rjhuijsman wants to merge 2 commits into
mainfrom
rjh.maestro-npm-stall-hardening

Conversation

@rjhuijsman

Copy link
Copy Markdown
Contributor

The Test React Native mobile app (Maestro) check failed three times
in a row while #151 was in the merge queue, on a commit that changed
nothing it tests — and the same failures showed up on an unrelated
branch. Two independent weaknesses in the harness turned a transient
runner-network problem into red checks.

npm stalls ate the test budget

npm fetches from inside the devcontainer began stalling on the
16_core_GitHub_hosted pool between 2026-09-03 20:45Z and
2026-09-04 00:21Z. npm's default fetch-timeout is five minutes, so a
single stall cost the test that long before it even retried. Both
npm-dependent setup phases were affected:

phase before during
rbt dev run → serving :9991 10–16s 213–616s
mobile npm install 47–77s 128–321s
uv sync 0.9/1.8/0.24s 1.2/2.1/0.24s
emulator boot 29–30s 29–30s
Metro bundle 17–31s 20–31s
container build 6–8 min 6–8 min

The slow phases cluster at ~310s and ~615s, i.e. one or two whole
fetch-timeouts. A large test gets 900s, so one stalled phase was
enough: bank-pydantic spent 616s waiting for rbt dev run, then had
its npm install killed at the 900s limit without ever booting an
emulator.

This branch warms the devcontainer image's npm cache with the three
@bufbuild packages rbt generate installs into .rbt, and runs npm
in the harness with prefer-offline so it serves them from that cache
rather than revalidating them against the registry. fetch-timeout
drops to one minute and fetch-retries rises to five, so a stalled
request fails quickly and is retried instead of sitting for five
minutes.

The first wait covered more than it was sized for

Separately, chat-room failed with Assertion is false: id: message-input is visible. The maestro-debug artifact showed Expo Go
still on its splash screen, reading Loading from 172.17.0.2:8081,
with no React Native view tree at all and not one host.exp.exponent
line in 30s of logcat.

The harness starts the flow as soon as Metro logs Android Bundled,
which reports that Metro finished building the bundle. The device
then has to fetch it over the emulator's user-mode NAT, parse ~900
modules in Hermes and mount them — all inside the flow's first
extendedWaitUntil. That assertion completes in 0.7s against an idle
emulator, but these runs boot the AVD cold from a google_apis image
under swiftshader_indirect, so Play Services' first-run work competes
for the same software-rendered CPU. Thirty seconds was never a budget
the slow case could meet, so this widens just that first wait to three
minutes. Every later timeout stays tight, so a genuine regression in
the app still fails fast.

Testing

maestro_test is manual/local/exclusive and needs /dev/kvm, so
it does not run in this repo's PR checks; the Maestro job that exercises
it lives in the mono dev-containers workflow. What I verified here:
both harness scripts pass bash -n, both flows parse and their first
step reads back as timeout: 180000, and npm cache add accepts the
scoped, pinned specs the Dockerfile uses.

Worth a reviewer's judgement: prefer-offline also applies to the
app's own npm install, where the cache is cold in CI and so should be
a no-op — but it does change resolution semantics for anyone running
these tests locally with a populated cache.

Follow-up not taken here

The sturdier fix is for the harness to wait on actual app readiness
rather than on Metro's log line, and to serve the bundle over adb reverse tcp:8081 tcp:8081 instead of the emulator's user-mode NAT.
Both are larger changes to how the harness drives the device; this PR
deliberately stays with the two narrow ones.

🤖 Generated with Claude Code

https://claude.ai/code/session_016qY16VqAe1y1dQuGEnqv72

rjhuijsman and others added 2 commits September 4, 2026 13:43
The `Test React Native mobile app (Maestro)` check failed three times
in a row on 2026-09-04 on a PR that changed nothing it tests, and the
same failure appeared on an unrelated branch. npm fetches from inside
the devcontainer were stalling on the `16_core_GitHub_hosted` runner
pool, and npm's default `fetch-timeout` is five minutes, so one stall
cost the test that long before it even retried.

Both npm-dependent setup phases hit it. The `.rbt` install that `rbt
dev run` performs went from 10-16s to 213-616s, and the app's own
`npm install` from 47-77s to 128-321s; the phase timings clustered at
~310s and ~615s, i.e. one or two whole `fetch-timeout`s. Everything
else was unchanged: `uv sync`, the emulator boot, Metro's bundling and
the flows themselves all took what they always take. A `large` test
gets 900s, so one stalled phase was enough to end the run, either by
timing out outright or by leaving the flow too little time to finish.

This commit keeps a degraded registry off that critical path. The
devcontainer image warms the npm cache with the three `@bufbuild`
packages `rbt generate` installs, and the harness runs npm with
`prefer-offline` so it serves them from that cache rather than
revalidating them against the registry. `fetch-timeout` drops to one
minute and `fetch-retries` rises to five, so a stalled request fails
quickly and is retried instead of sitting for five minutes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016qY16VqAe1y1dQuGEnqv72
The chat-room Maestro flow failed with `Assertion is false: id:
message-input is visible` on a run where nothing about the app had
changed. The `maestro-debug` artifact showed what the device was
actually doing: Expo Go sat on its splash screen reading `Loading from
172.17.0.2:8081`, its 30s of logcat carried no lines from
`host.exp.exponent` at all, and the view hierarchy held a splash image
and a single `TextView`, so `message-input` had never existed.

The harness starts the flow as soon as Metro logs `Android Bundled`,
which reports that Metro finished building the bundle. The device then
has to fetch it over the emulator's user-mode NAT, parse ~900 modules
in Hermes and mount them, and all of that has to fit inside the flow's
first `extendedWaitUntil`. That assertion completes in 0.7s against an
idle emulator, but these runs boot the AVD cold from a `google_apis`
image under `swiftshader_indirect`, so Play Services' first-run work
competes for the same software-rendered CPU and the load takes minutes.
Thirty seconds was never a budget the slow case could meet.

Widen just that first wait, in both flows, to three minutes. Every
later timeout stays tight, so a genuine regression in the app still
fails fast rather than hanging.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016qY16VqAe1y1dQuGEnqv72
@aviator-app

aviator-app Bot commented Sep 4, 2026

Copy link
Copy Markdown

Current Aviator status

Aviator will automatically update this comment as the status of the PR changes.
Comment /aviator refresh to force Aviator to re-examine your PR (or learn about other /aviator commands).

This pull request is currently open (not queued).

How to merge

To merge this PR, comment /aviator merge or add the mergequeue-ready label.


See the real-time status of this PR on the Aviator webapp.
Use the Aviator Chrome Extension to see the status of your PR within GitHub.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant