Keep registry stalls and slow device loads from failing the Maestro tests - #153
Draft
rjhuijsman wants to merge 2 commits into
Draft
Keep registry stalls and slow device loads from failing the Maestro tests#153rjhuijsman wants to merge 2 commits into
rjhuijsman wants to merge 2 commits into
Conversation
The `Test React Native mobile app (Maestro)` check failed three times in a row on 2026-09-04 on a PR that changed nothing it tests, and the same failure appeared on an unrelated branch. npm fetches from inside the devcontainer were stalling on the `16_core_GitHub_hosted` runner pool, and npm's default `fetch-timeout` is five minutes, so one stall cost the test that long before it even retried. Both npm-dependent setup phases hit it. The `.rbt` install that `rbt dev run` performs went from 10-16s to 213-616s, and the app's own `npm install` from 47-77s to 128-321s; the phase timings clustered at ~310s and ~615s, i.e. one or two whole `fetch-timeout`s. Everything else was unchanged: `uv sync`, the emulator boot, Metro's bundling and the flows themselves all took what they always take. A `large` test gets 900s, so one stalled phase was enough to end the run, either by timing out outright or by leaving the flow too little time to finish. This commit keeps a degraded registry off that critical path. The devcontainer image warms the npm cache with the three `@bufbuild` packages `rbt generate` installs, and the harness runs npm with `prefer-offline` so it serves them from that cache rather than revalidating them against the registry. `fetch-timeout` drops to one minute and `fetch-retries` rises to five, so a stalled request fails quickly and is retried instead of sitting for five minutes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016qY16VqAe1y1dQuGEnqv72
The chat-room Maestro flow failed with `Assertion is false: id: message-input is visible` on a run where nothing about the app had changed. The `maestro-debug` artifact showed what the device was actually doing: Expo Go sat on its splash screen reading `Loading from 172.17.0.2:8081`, its 30s of logcat carried no lines from `host.exp.exponent` at all, and the view hierarchy held a splash image and a single `TextView`, so `message-input` had never existed. The harness starts the flow as soon as Metro logs `Android Bundled`, which reports that Metro finished building the bundle. The device then has to fetch it over the emulator's user-mode NAT, parse ~900 modules in Hermes and mount them, and all of that has to fit inside the flow's first `extendedWaitUntil`. That assertion completes in 0.7s against an idle emulator, but these runs boot the AVD cold from a `google_apis` image under `swiftshader_indirect`, so Play Services' first-run work competes for the same software-rendered CPU and the load takes minutes. Thirty seconds was never a budget the slow case could meet. Widen just that first wait, in both flows, to three minutes. Every later timeout stays tight, so a genuine regression in the app still fails fast rather than hanging. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016qY16VqAe1y1dQuGEnqv72
Current Aviator status
This pull request is currently open (not queued). How to mergeTo merge this PR, comment
See the real-time status of this PR on the
Aviator webapp.
Use the Aviator Chrome Extension
to see the status of your PR within GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
Test React Native mobile app (Maestro)check failed three timesin a row while #151 was in the merge queue, on a commit that changed
nothing it tests — and the same failures showed up on an unrelated
branch. Two independent weaknesses in the harness turned a transient
runner-network problem into red checks.
npm stalls ate the test budget
npm fetches from inside the devcontainer began stalling on the
16_core_GitHub_hostedpool between 2026-09-03 20:45Z and2026-09-04 00:21Z. npm's default
fetch-timeoutis five minutes, so asingle stall cost the test that long before it even retried. Both
npm-dependent setup phases were affected:
rbt dev run→ serving:9991npm installuv syncThe slow phases cluster at ~310s and ~615s, i.e. one or two whole
fetch-timeouts. Alargetest gets 900s, so one stalled phase wasenough:
bank-pydanticspent 616s waiting forrbt dev run, then hadits
npm installkilled at the 900s limit without ever booting anemulator.
This branch warms the devcontainer image's npm cache with the three
@bufbuildpackagesrbt generateinstalls into.rbt, and runs npmin the harness with
prefer-offlineso it serves them from that cacherather than revalidating them against the registry.
fetch-timeoutdrops to one minute and
fetch-retriesrises to five, so a stalledrequest fails quickly and is retried instead of sitting for five
minutes.
The first wait covered more than it was sized for
Separately,
chat-roomfailed withAssertion is false: id: message-input is visible. Themaestro-debugartifact showed Expo Gostill on its splash screen, reading
Loading from 172.17.0.2:8081,with no React Native view tree at all and not one
host.exp.exponentline in 30s of logcat.
The harness starts the flow as soon as Metro logs
Android Bundled,which reports that Metro finished building the bundle. The device
then has to fetch it over the emulator's user-mode NAT, parse ~900
modules in Hermes and mount them — all inside the flow's first
extendedWaitUntil. That assertion completes in 0.7s against an idleemulator, but these runs boot the AVD cold from a
google_apisimageunder
swiftshader_indirect, so Play Services' first-run work competesfor the same software-rendered CPU. Thirty seconds was never a budget
the slow case could meet, so this widens just that first wait to three
minutes. Every later timeout stays tight, so a genuine regression in
the app still fails fast.
Testing
maestro_testismanual/local/exclusiveand needs/dev/kvm, soit does not run in this repo's PR checks; the Maestro job that exercises
it lives in the
monodev-containers workflow. What I verified here:both harness scripts pass
bash -n, both flows parse and their firststep reads back as
timeout: 180000, andnpm cache addaccepts thescoped, pinned specs the
Dockerfileuses.Worth a reviewer's judgement:
prefer-offlinealso applies to theapp's own
npm install, where the cache is cold in CI and so should bea no-op — but it does change resolution semantics for anyone running
these tests locally with a populated cache.
Follow-up not taken here
The sturdier fix is for the harness to wait on actual app readiness
rather than on Metro's log line, and to serve the bundle over
adb reverse tcp:8081 tcp:8081instead of the emulator's user-mode NAT.Both are larger changes to how the harness drives the device; this PR
deliberately stays with the two narrow ones.
🤖 Generated with Claude Code
https://claude.ai/code/session_016qY16VqAe1y1dQuGEnqv72