Skip to content

fix(android): read device ownership off the device, not off adb - #2604

Merged
thymikee merged 12 commits into
mainfrom
fix/android-helper-ownership-release-2553
Sep 15, 2026
Merged

thymikee merged 12 commits into
mainfrom
fix/android-helper-ownership-release-2553

Conversation

@thymikee

@thymikee thymikee commented Sep 14, 2026

Copy link
Copy Markdown
Member

Summary

Snapshot helper retirement treated the outcome of an am force-stop call as the fact it was supposed to measure. On a loaded host that adb round trip exceeds its budget while the helper process is already gone, so a completed interaction failed in its own teardown and the following quarantine refused the next command with "could not confirm release of device automation ownership" on a device holding nothing. That is also how a settled press --settle turned into the stale-coordinate failures reported in the issue.

Ownership is a device fact, so it is read as one now: the probe asks the device shell for pidof <helperPackage> and tells it to echo a marker when nothing matched. A process id is occupied, the bare marker with a silent stderr is released, and everything else — error: closed, cannot connect to daemon, device offline, an adb client killed by a signal before it wrote anything — is unknown. A release is then claimable only through an answer the transport cannot produce about itself: a client that ran out of budget, was SIGKILLed, or lost the connection prints nothing at all. No exit status is consulted either, because adb shell answers 0 for a device command that failed (measured on API 35) and an adb that dies by signal leaves the executor inventing an exit code it never saw. Listing adb's failure texts was not the fix either: the list is long, version-dependent, and every missed entry fails open. Only a device that keeps naming a live helper process may refuse a command (android_snapshot_helper_runtime_occupied), and it refuses after a second read, so a helper still inside Android's exit path costs nothing. Teardown records what it could not prove for the next acquire and never decides what the command it is finishing reports.

A session that reaches ready settles a pending release, because am instrument for the helper package force-stops whatever is already instrumenting that package — the ready helper is the only helper process the device has. That mechanism is named in the lifecycle comment, the settleAndroidSnapshotHelperRetirement docblock, the CHANGELOG and the test, so a change to the helper's ready order or to its package breaks the settle loudly instead of silently. A helper start that fails is answered for by the one-shot transport and not spawned again until a backoff scaled to how long it spent failing has run out (10–60 s per capture identity), which is what had roughly doubled command time on hosts where the helper never starts. The readiness wait takes half the helper-command budget the capture was built with — 15 s today — instead of a fixed 10 s; the CLI's --timeout reaches that wait as its deadline aborting it, not as the number.

Two cleanups ride along, named because they are not ownership fixes: the system-surface capture case no longer emits an android_snapshot_helper_system_surface diagnostic (no consumer read it, and the typed systemSurfaceOnly metadata already travels with the response and becomes the user-facing disclosure), and a cached session whose instrumentation process has already exited is dropped rather than written to. Four comments also repeated the "Android permits one UiAutomation owner" story this review rejected; they name the instrument takeover now, in the lifecycle module docstring, the session-scope type, the touch piggyback comment and the fill-verification header. The runtime reset also shares the one best-effort am force-stop the retirement path owns, one fake answers pidof for every session test, and the helper device key and the occupied-device predicate stay module-owned.

Closes #2553

Validation

Tested at 3de2027997.

  • pnpm check:affected --run: all runnable checks passed, incl. check:fallow and check:layering; typecheck, lint and format clean; 718 platform-android tests. All 17 GitHub checks pass at 18c6651.
  • Live on a Pixel 7 CI emulator (API 35), original repro: an adb wrapper delaying am instrument 12s and am force-stop 6s made snapshot --force-full fail with the ownership error before the change while adb shell pidof showed no helper process; after it, consecutive commands each return 37 nodes. Healthy-host snapshot (p50 296ms) and press --settle unaffected.
  • Live, a pidof read killed by a signal while the helper ran: the shim answers shell pidof by SIGKILLing itself before writing anything, am force-stop as a no-op, and refuses the one-shot fallback so the request log keeps the diagnostic. Request log 2ed92c31c6c3c924.ndjson:
    {"ts":"2026-09-15T15:36:28.151Z","level":"warn","phase":"android_snapshot_helper_retirement_pending","requestId":"2ed92c31c6c3c924","command":"snapshot","data":{"deviceKey":"android:emulator-5554","packageName":"com.callstack.agentdevice.snapshothelper","release":"unknown"}}
    with helper pid 27363 read directly afterwards by real adb shell pidof. Under the exit-code rule that shape was released and the entry was cleared. The next command's calls are am force-stoppidof … || echo AGENT_DEVICE_NO_HELPERam instrument: the pending retirement was retried, an unreadable read refused nothing, and the command answered from a fresh session (helper pid 27409; the instrument takeover ended 27363, the same mechanism the settle rests on).
  • Live, fail-closed read with the helper running: an adb shim answering shell pidof with error: closed (a fault no classifier lists) and am force-stop as a no-op, with the session socket pulled mid-command. Request log 5477a2983b05082a.ndjson:
    {"ts":"2026-09-15T13:41:55.173Z","level":"warn","phase":"android_snapshot_helper_retirement_pending","requestId":"5477a2983b05082a","command":"snapshot","data":{"deviceKey":"android:emulator-5554","packageName":"com.callstack.agentdevice.snapshothelper","release":"unknown"}}
    adb shell pidof read directly afterwards still returned the helper (pid 25334), so the unknown came from a live helper and an unrecognized adb error — the shape that used to read released and clear the entry. The next command's shim log order is am force-stoppidofam instrument -e sessionPort: the entry stayed pending, the stop attempt precedes any spawn, and the command answered from the session with one spawn and a clean quit, no second process coexisting.
  • Live, failed-start backoff: with am instrument -e sessionPort delayed 45s and --timeout 40000 three times, commands took 19 s, 0 s, 1 s with one sessionPort spawn across all three; before the backoff every command paid the failed start.
  • Live, readiness budget: on a host taking 12s to bring the helper up, the command used to answer with the one-shot transport (9 s, helperTransport: "instrumentation") and now answers from the session (15 s, helperTransport: "persistent-session").
  • Probe checked against a real session: pidof returns a pid while the instrumentation runs, empty after quit.

No provider-integration or coverage obligations selected locally; GitHub stays authoritative.

@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
Installed (including dependencies) 4.58 MB 4.58 MB +776 B
Package (unpacked) 4.58 MB 4.58 MB +776 B
Package (download) 1.36 MB 1.36 MB +316 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 27.0 ms 27.4 ms +0.4 ms
CLI --help 76.9 ms 78.1 ms +1.3 ms

@thymikee

Copy link
Copy Markdown
Member Author

Reviewed at 595f562. Reading ownership off the device is the right direction, but two paths can still clear or break ownership without proof.

A transport failure reads as released. The probe at snapshot-helper-retirement.ts:164 runs adb shell pidof with allowFailure: true, so "device offline" or "device unauthorized" returns exit 1 with stderr instead of throwing. The check (non-zero exit or no pid) treats that as released; unknown covers only timeouts and thrown errors. A real "released" answer is also exit 1 with empty stdout, so the tests cannot tell the two apart. With a flapping adb, a pending retirement clears without proof, a second instrumentation can start while the old helper still holds UiAutomation, and that failed start can then disable the persistent path for that identity. Could released require a trustworthy result (empty stderr, or classifyAndroidAdbFailure(stderr) undefined) and everything else map to unknown, with a fixture for exit 1 plus "error: device offline"?

An unknown read can later force-stop a live session. On main a pending retirement always threw quarantine. Now snapshot-helper-retirement.ts:71 leaves the entry pending and lets the command continue. A successful start clears disabledSessionIdentities but not pendingRetirements, so the next acquire (or prepareAndroidTouchHelper) still sees the old entry, force-stops the helper package, and kills the session this daemon just started. Under the load from #2553, where pidof can miss its budget, this can drop a session mid-command. Should a ready session clear the pending retirement for its device? A test for unreadable probe, then acquire, then acquire, with no force-stop against the live session, would pin it.

The tests "unproven release stays pending until an acquire reads the device" and "unreadable device leaves the retirement pending" (snapshot-helper-retirement.test.ts:75) call recover twice but assert nothing about adb calls or the entry afterwards, so they pass even if the first call consumes the entry. Can they assert the call counts?

Two smaller points. SESSION_READY_TIMEOUT_REASON (snapshot-helper-session-protocol.ts:111) has no reader in production or tests; drop it or use it? And resetAndroidSnapshotHelperSessions moved from Promise.allSettled to Promise.all, so one rejected stop skips clearing the disabled identities, retirements and probes. Could it keep allSettled and clear in a finally?

A design question before more state goes in: AndroidSnapshotHelperRetryState backs two maps with different meanings and is only a retryAtMs? field. Could it fold into the disabled-identity map, with pendingRetirements a plain map, and could one rule owned by the session lifecycle ("a ready session or a proven-released pidof clears the pending entry") close the second gap without new state? If you considered this and rejected it, what would have to change first?

The changed teardown and acquire path also needs a live Android run under a slow or offline adb, including one session reused across two commands after an unknown probe.

Coverage fails in scripts/__tests__/eager-closure-budgets.test.ts: packages/platform-android/src/mechanics.ts now evaluates 178 modules on import, one more than the merge-base. That looks related to the new module this PR adds, not to the Apple runner issue the PR body names. The Android smoke job was still running at review time.

@thymikee

Copy link
Copy Markdown
Member Author

Reviewed at 595f562. Reading ownership from the device is the right fact to use, but a few cases still look wrong.

snapshot-helper-retirement.ts#L166: the pidof probe runs with allowFailure: true. A host adb failure ("device offline", "device not found", "error: closed") seems to come back as exit code 1 with empty stdout, not as a throw. The exitCode !== 0 || no digits check maps that to released, the same as a device-side pidof: not found. On the unhealthy-transport host from #2553, the pending retirement would then clear while the helper may still own UiAutomation, and the next acquire starts a second instrumentation. Can released come only from a device that answered, with other results mapped to unknown? classifyAndroidAdbFailure may already do this classification.

snapshot-helper-retirement.ts#L90: unknown keeps the pending retirement but lets the command proceed, and startAndroidSnapshotHelperSession never clears it. So the next acquire, one-shot capture or prepareAndroidTouchHelper runs recover first, which force-stops the helper this daemon just started. If pidof keeps returning unknown, this repeats on every command, which is the #2553 churn again. Could a session that reaches ready clear the pending retirement, or recover skip a live session? A test would help: record unknown, start a session, acquire again, assert no force-stop.

snapshot-helper-session.fixtures.ts#L299 models an unreadable device as a thrown Error('device offline'), but the executor in the same file returns { exitCode: 1, stderr: 'error: device offline' } for force-stop. So the unknown tests take a path the real transport does not take. Can the fixture return the non-zero result instead, and assert the actual calls?

snapshot-helper-retry-state.ts is a new generic module with one predicate, and PendingRetirement never sets retryAtMs. Its static import is what moves mechanics.ts from 177 to 178 modules and fails Coverage (eager-closure-budgets). Could it go, with pendingRetirements as Map<string, { packageName, cause }> and the { identity, retryAtMs? } check inline in snapshot-helper-session-lifecycle.ts?

snapshot-helper-retirement.ts#L160 is a third pidof parser next to findPidToken in perf-native-process.ts and isAndroidPackageProcessRunning in app-lifecycle.ts. The latter also waits for a stable "gone" read after force-stop; this probe reads right away, so a helper that is still exiting could read as occupied and refuse the command. Could it reuse findPidToken and re-read before refusing?

snapshot-helper-session-lifecycle.ts#L481: the switch from allSettled to Promise.all means one throwing stop skips disabledSessionIdentities.clear(), resetAndroidSnapshotHelperRetirements() and the protocol-probe reset. Could the clears run in a finally? SESSION_READY_TIMEOUT_REASON is also written but never read.

The error reason changes from android_snapshot_helper_retirement_unconfirmed to android_snapshot_helper_runtime_occupied, which breaks scripts that match the old reason. Please add a CHANGELOG line.

Would a smaller change do? Keep the old two-map shape and change only the evidence source: a three-state pidof read that treats transport errors as unknown, inline retry times in the lifecycle, and a successful session start clearing the pending retirement. Nothing has to change first; the retry-state module and its test would go away.

The emulator run in the description covers the slow-transport repro. Could you also run a pidof read during a real adb transport failure with the helper still running, and a pending unknown followed by a started session and a second command?

The Coverage failure is from this change, not pre-existing: the new module adds the extra module to the mechanics.ts closure. There are no conflicts.

@thymikee

Copy link
Copy Markdown
Member Author

Reviewed at eed1d0c. The Coverage failure from 595f562 is gone because the delta deletes snapshot-helper-retry-state.ts, but the main problem from the last review is still open.

snapshot-helper-retirement.ts#L166: the pidof probe still runs with allowFailure: true. Doesn't a host adb failure ("error: device offline", "device not found") come back as a non-zero exit with empty stdout instead of a throw? If so, exitCode !== 0 || no pid reads it as released, and recover clears the pending retirement with nothing proving the old helper let go. On the unstable transport from #2553 the daemon can then start a second instrumentation while the old helper still holds UiAutomation. Can released require an empty or unclassified stderr, with every other result mapped to unknown? The fixture at snapshot-helper-session.fixtures.ts#L299 still models an unreadable device as a throw; returning { exitCode: 1, stderr: 'error: device offline' } there would test the real shape.

snapshot-helper-session-lifecycle.ts#L220: after an unknown probe the entry stays in pendingRetirements, and nothing clears it when a later session reaches ready. The next acquire, one-shot capture or touch prep calls recoverAndroidSnapshotHelperRetirement, which force-stops the helper package and kills the session this daemon just started. While pidof keeps returning unknown, that repeats on every command, which is the #2553 symptom again. Can a session reaching ready clear the pending retirement, or can recover skip force-stop while sessions.get(deviceKey) is live? A test that records unknown, starts a session, acquires again and asserts no force-stop would cover it.

snapshot-helper-session-lifecycle.ts#L145: the delta removes the start backoff that main and 595f562 had. When the helper never starts, each command now spawns, waits up to its timeout, force-stops and probes before it falls back to one-shot, which on the slow hosts from #2553 could roughly double command time. Could a short in-memory backoff per identity come back, with a test that counts spawns across several commands on a device that always times out?

The retirement tests at snapshot-helper-retirement.test.ts#L76 still assert only "no adb calls and no state change", as noted at 595f562, so they pass even if recover silently consumes the entry. Recording the calls and asserting that a second recover still sends force-stop and pidof would guard the two problems above. The error reason also changes from android_snapshot_helper_retirement_unconfirmed to android_snapshot_helper_runtime_occupied, and failed starts now retry on every command; please add a CHANGELOG line for both.

A smaller shape may cover all of this: keep main's two maps and change only the evidence source, so pidof has three states, a ready session or a proven released read clears the pending retirement, recover never force-stops a live session, and a failed start keeps a short backoff. Would that work, or is there a reason the current shape is needed?

This device-facing change still needs a live Android run on the fixed head covering a pidof read during a real adb transport failure while the helper runs, a pending unknown followed by a started session and a second command, and repeated commands on a device whose helper start times out, with latency recorded. All checks pass on eed1d0c, and there are no conflicts. The next step is to fix the first two problems with tests, then add that live run.

@thymikee
thymikee force-pushed the fix/android-helper-ownership-release-2553 branch from eed1d0c to ffae528 Compare September 15, 2026 12:44
@thymikee

Copy link
Copy Markdown
Member Author

Fixed at 18c6651 (four commits on top of eed1d0c). Rebase onto current main was clean; no conflicts.

pidof truthfulness. readAndroidSnapshotHelperRuntimeRelease (snapshot-helper-retirement.ts:173) now takes the process token from findPidToken (perf-native-process.ts) instead of its own regex, and when nothing is named it decides released vs unknown with classifyAndroidAdbFailure(stderr, stdout): a device/transport fault adb recognises is unknown, while an unclassified stderr (an older image printing pidof: not found) is the device's own answer and is released. The fixture models the real shape now — { exitCode: 1, stderr: 'error: device offline' } rather than a thrown error — so the unknown tests walk the transport they claim to cover. A refusal also re-reads 250 ms later, so a helper still inside Android's exit path no longer refuses a command.

A pending unknown no longer force-stops a live session. A session reaching ready calls settleAndroidSnapshotHelperRetirement(deviceKey) (snapshot-helper-session-lifecycle.ts:297): Android hands UiAutomation to one connection, so a helper that just announced itself owns it and whatever held it before does not. a session that reaches ready settles a release the device could not confirm records unknown, starts a session, and asserts the next command sends no am force-stop. I took your first option and did not add the reverse "recover skips a live session" rule, because retirement.ts reading the session map would be a back-import into the module that owns it; what that leaves open is a one-shot cancellation recording a pending release while a session is live, which still force-stops on the next acquire.

Failed-start backoff. failedStarts is one inline Map<identity, retryAtMs> in the lifecycle (snapshot-helper-session-lifecycle.ts:97, :207), no module and no disabled-identity state. The window is how long that start spent failing, clamped to 10 s..60 s, so a start that burned half a minute failing is not re-paid on the next command, while a start that failed instantly is not written off for a minute. a helper that never starts is not spawned again on every command counts spawns: three commands, one spawn, and a different capture identity spawns again.

The retirement tests assert calls now. unproven release stays pending… pins force-stop + pidof on the record, force-stop + pidof + pidof for the refusal, force-stop + pidof once the device says it is gone, and nothing on the acquire after that. a device that cannot be read… pins the repeated force-stop + pidof while the device stays unreadable. Added a shell that has no pidof still answers for its own processes.

Smaller shape. Mostly taken: the retry-state module and its test are gone, pendingRetirements is a plain { packageName, cause } map, and the retry time is inline in the lifecycle. What I did not bring back is the disabled-identity map — once the backoff exists it has no job the backoff does not already do.

Smaller points. SESSION_READY_TIMEOUT_REASON is read now: the start-failed diagnostic carries error.details.reason instead of an error message. resetAndroidSnapshotHelperSessions is back on allSettled with the clears in a finally. CHANGELOG carries both lines — the old reason is in released v0.21.3, so the rename is user-visible. eager-closure-budgets passes locally (614 tests); the extra module in the mechanics.ts closure went away with the retry-state module.

Two cleanups found on the way: the system-surface case no longer emits a diagnostic duplicating the typed systemSurfaceOnly disclosure that travels with the response, and a cached session whose instrumentation process already exited is dropped instead of written to.

Found while verifying on device, and fixed: the readiness wait was still capped at 5 s. resolvePersistentSessionCaptureOptions clamps the persistent path's commandTimeoutMs to capture (2 s) + overhead (3 s), so feeding that into the wait reproduced the original 10 s problem in miniature — a host needing ~12 s to bring the helper up was pushed onto the one-shot transport even at --timeout 60000. The wait now gets half the caller's command budget, floored at what one session command is worth (resolveAndroidSnapshotHelperStartBudgetMs, unit-tested at 60 s→30 s, 30 s→15 s, 6 s→5 s, 1 s→1 s). A/B on the same device under the same shim: 9 s / instrumentation before, 15 s / persistent-session after.

Live runs (emulator-5554, fixed head, real adb behind a shim that logs every call):

  • pidof during a real transport failure with the helper running — adb shell pidof answering error: device offline / exit 1: the request log records android_snapshot_helper_retirement_pending { release: "unknown" } after am force-stoppidof. The old read called that released and cleared the entry with nothing proving the hand-off.
  • pending unknown, then a started session, then a second command — with the fault left in place the next command force-stops, reads unknown, starts a session, reaches ready and answers helperTransport: "persistent-session"; the command after that answers from the session as well. The no-force-stop-against-a-live-session claim is asserted directly in the unit test.
  • repeated commands on a device whose helper start times out (am instrument -e sessionPort delayed 45 s), --timeout 40000 three times: 19 s, 0 s, 1 s, with one sessionPort spawn across all three. Before the backoff, every one of those commands paid the failed start.
  • real adb, no shim: snapshot --force-full → success, helperTransport: "persistent-session", 3 s, and no helper process left on the device afterwards.

Gate: check:affected --run, typecheck, lint, format, check:fallow, check:layering green; 717 platform-android tests.

@thymikee

Copy link
Copy Markdown
Member Author

Reviewed at 18c6651, as a follow-up to the review at eed1d0c. The ready-settle and the start backoff are back, but the pidof read can still fail open.

snapshot-helper-retirement.ts#L188 reads every stderr that classifyAndroidAdbFailure does not recognize as released. Don't other adb transport failures, such as error: closed, cannot connect to daemon, failed to get feature set or protocol fault (couldn't read status), also come back as a non-zero exit with empty stdout? If so, recover clears the pending retirement with nothing proving the old helper let go, which is the #2553 case. Can released require the device shell's own no-process answer, with every other non-pid result mapped to unknown? A fixture with stderr error: closed that expects unknown would pin it.

snapshot-helper-session-lifecycle.ts#L381: resolveAndroidSnapshotHelperStartBudgetMs is documented as half of --timeout, but its only production caller, buildAndroidSnapshotHelperCaptureOptions in snapshot-helper-capture.ts:136, always passes 30000. The start wait is therefore always 15 s, whatever --timeout says. Should the command's real timeout reach the builder, or should the comments and CHANGELOG say the wait is fixed?

snapshot-helper-session-lifecycle.ts#L294 says a ready helper holds the device's one UiAutomation connection. SnapshotInstrumentation.java sends sessionReady right after it binds the socket, before any getUiAutomation() call. The settle holds because starting instrumentation for the same target package force-stops the earlier process. Can the comment, the snapshot-helper-retirement.ts docblock, the CHANGELOG line and the lifecycle test comment name that mechanism, so a later change to the target package or the ready order does not break it silently?

A smaller point: snapshot.ts drops the android_snapshot_helper_system_surface diagnostic (L487), which is not part of the ownership fix. No consumer reads it, but can the PR body name it as an intended cleanup, or can it move to its own PR?

All 17 checks pass and there are no conflicts.

The live runs in the PR are author-reported and use error: device offline, the one string the classifier already knows. After the pidof fix, this device-facing change still needs a live run where pidof gets an adb error the classifier does not list (for example error: closed from a shim) while the helper runs. It should show android_snapshot_helper_retirement_pending with release: 'unknown' and no second session spawn while the old process is alive, with the request-log excerpt attached.

Next: make pidof fail closed on unrecognized adb errors, fix the --timeout claim, then attach that run.

@thymikee
thymikee force-pushed the fix/android-helper-ownership-release-2553 branch from 18c6651 to f4a514b Compare September 15, 2026 13:46
@thymikee

Copy link
Copy Markdown
Member Author

Fixed at f4a514bf59. Rebased onto current main; no conflicts.

pidof fails closed now, and not by list. readAndroidSnapshotHelperRuntimeRelease (snapshot-helper-retirement.ts:172) keeps only the positive form of your rule: occupied when findPidToken finds a token, released only when the shell itself said there is no process — non-zero exit, nothing on stdout, nothing on stderr — and unknown for every other result. classifyAndroidAdbFailure is no longer imported here at all, which was the actual problem: the answer's trustworthiness cannot depend on how complete a list of adb's complaints happens to be. error: closed, adb: cannot connect to daemon, failed to get feature set: device offline, pidof: not found on stderr, and a zero exit naming nobody are all unknown, pinned in a read that names no process is released only when the shell itself said so, next to a device that answers with nothing at all is read as released so the one released-earning shape stays pinned too. snapshot-helper-session.fixtures.ts answers pidof with error: closed under a closed variant (next to the classifier-known unreadable), and a session that reaches ready settles a release the device could not confirm now runs the whole lifecycle on that unrecognized fault.

The --timeout claim is gone rather than the budget. You are right that the only production caller passes the constant: buildAndroidSnapshotHelperCaptureOptions builds commandTimeoutMs from ANDROID_SNAPSHOT_HELPER_COMMAND_TIMEOUT_MS (30 s), so the wait is 15 s whatever the flag says. I kept the formula — it does honor an explicit commandTimeoutMs, which is what the budget test drives — and fixed the three places that claimed otherwise: the function docblock now says production builds the budget from that constant and names 15 s, the call-site comment says the caller's deadline reaches the wait as an abort on options.signal, and the CHANGELOG says the same instead of "half of --timeout". I did not thread the command deadline into the builder: it would cross from the daemon's request budget into platform capture options for a wait that is already bounded by the abort, and on the device the abort is what ended that wait at the command's own 6 s deadline under --timeout 6000. Say the word if you would rather have the number reach it.

The settle's mechanism is named where it can break. am instrument force-stops whatever is already instrumenting the helper package, and SnapshotInstrumentation reports sessionReady straight after binding the socket, before any getUiAutomation(). That is in the ready-path comment (snapshot-helper-session-lifecycle.ts:293), in the settleAndroidSnapshotHelperRetirement docblock — which also states what would break it, a shared package or a ready report moved after the UiAutomation acquire — in the CHANGELOG line, and in the lifecycle test's comment. The one UiAutomation-connection story is out.

The diagnostic removal is named in the PR body as one of two cleanups that are not ownership fixes, with why: nothing consumed android_snapshot_helper_system_surface, and systemSurfaceOnly already travels on the capture and becomes the user-facing disclosure. It stayed rather than moving out because removing it is what keeps the content branch a single return.

Live run with an unrecognized adb error while the helper runs (Pixel 7 CI emulator, API 35): an adb shim answers shell pidof with error: closed / exit 1 and am force-stop as a no-op success, and the session forward is pulled out from under a live capture. Request log 5477a2983b05082a.ndjson:

{"ts":"2026-09-15T13:41:55.173Z","level":"warn","phase":"android_snapshot_helper_retirement_pending","session":"cwd:51b329c31054f9af:default","requestId":"5477a2983b05082a","command":"snapshot","data":{"deviceKey":"android:emulator-5554","packageName":"com.callstack.agentdevice.snapshothelper","release":"unknown"}}

adb shell pidof read outside the CLI immediately afterwards returned 25334, so that unknown came from a live helper plus an error no classifier lists — exactly the read that used to be released and clear the entry. The next command's shim log is am force-stoppidofam instrument -e sessionPort: the entry stayed pending, the stop attempt still precedes any spawn, and the command answered helperTransport: "persistent-session" with one spawn and a clean quit. There is never a moment with two helper processes, for the reason named above: starting instrumentation for the package takes the runtime from the process that had it.

Gate: check:affected --run, typecheck, lint, format, check:fallow, check:layering green; 718 platform-android tests.

@thymikee

Copy link
Copy Markdown
Member Author

Reviewed at f4a514b, as a follow-up to the review at 18c6651. Replacing the classifier with one positive rule makes this smaller, but a signal-killed pidof read may still count as released.

The rule at snapshot-helper-retirement.ts#L191 is a non-zero exit with empty stdout and stderr. The local executor sets exitCode to code ?? 1 (exec.ts#L199), and ExecResult has no signal field. Doesn't an adb client killed before it writes anything, for example by pkill adb, a host OOM kill or a concurrent kill-server, then look the same as the shell's own no-process answer? A timeout rejects and reads as unknown, but a signal death resolves and reads as released. On the loaded hosts #2553 describes, that would clear a pending retirement while the old helper may still run, so the next acquire can skip its force-stop and start a second instrumentation.

Can the exec result carry the signal, or keep code null, and can released require a real exit code? A fixture with a signal-killed result and empty streams that expects unknown would pin it. If you think the gap is acceptable, can the PR say why?

All 17 checks pass and there are no conflicts. Next: decide the signal case and pin it with a fixture.

@thymikee
thymikee force-pushed the fix/android-helper-ownership-release-2553 branch from f4a514b to 3de2027 Compare September 15, 2026 15:42
@thymikee

Copy link
Copy Markdown
Member Author

Fixed at 3de2027997, rebased onto main (four commits in, including #2599); no conflicts.

Neither signal plumbing nor an accepted gap: the probe stopped reading exit status. readAndroidSnapshotHelperRuntimeRelease now asks the device shell to answer the question itself — ['shell','pidof',pkg,'||','echo','AGENT_DEVICE_NO_HELPER'] — and reads only stdout: a pid token is occupied, stdout that is exactly the marker with nothing on stderr is released, everything else is unknown. A killed adb cannot produce the marker because the marker is printed by the shell the client never reached: a client SIGKILLed, OOMed or raced by kill-server prints nothing, and so does one whose transport closed. exitCode is not consulted at all, which also retires the assumption that a host exit status is the device's — on the API 35 emulator adb shell pidof nosuchthing || echo MARK answers 0, so the old rule was reading something unsound in both directions. ANDROID_SNAPSHOT_HELPER_NO_HELPER_ANSWER is exported so the fake device answers with the real string rather than a copy.

I did not add a signal to ExecResult or keep code null. That puts a field on every host command in the repo to serve one probe, across exec.tsHostCommandResultAndroidAdbExecutorResult, and main rewrote exactly that exec settlement path today in #2599exitCode: code ?? 1 would have been a bad file to be standing in. The requirement you wrote, "released requires a real answer", is now satisfied without any layer knowing about signals.

The stderr clause stays for a different reason than before: a device shell with no pidof exits 127, || fires, and the marker prints anyway — the complaint on stderr is what distinguishes "there is no such process" from "there is no such command", and the table pins that entry alongside error: closed, cannot connect to daemon, failed to get feature set, and a marker with trailing text.

Pinned with fixtures, as asked. snapshot-helper-session.fixtures.ts gains a signalled variant returning the executor's own report of a SIGKILLed child (exitCode: 1, empty stdout, empty stderr), and an adb killed before it answers is not a release, whatever exit code it left behind asserts that reads unknown, naming the consequence: clearing here lets the next acquire skip its force-stop beside a running helper.

Same scenario on the emulator (Pixel 7, API 35): an adb shim answers shell pidof by SIGKILLing itself before writing, am force-stop as a no-op, and refuses the one-shot fallback so the command fails and keeps its diagnostics. Warm daemon-scoped session (helper 27363), session forward pulled out from under the next command:

{"ts":"2026-09-15T15:36:28.151Z","level":"warn","phase":"android_snapshot_helper_retirement_pending","session":"cwd:51b329c31054f9af:default","requestId":"2ed92c31c6c3c924","command":"snapshot","data":{"deviceKey":"android:emulator-5554","packageName":"com.callstack.agentdevice.snapshothelper","release":"unknown"}}

adb shell pidof outside the CLI read 27363 immediately after that request, so the entry stayed pending over a live helper — the shape that read released and cleared until this commit. The next command's adb calls were am force-stoppidof … || echo AGENT_DEVICE_NO_HELPERam instrument: the pending retirement was retried, an unreadable read refused nothing, and the command answered from a fresh session (helper 27409, which is the instrument takeover ending 27363 — the mechanism the settle rests on).

Holistic pass over the claim you corrected. Four other comments still repeated "Android permits ONE UiAutomation owner" as the reason for device-exclusivity: the snapshot-helper-session-lifecycle.ts module docstring, the AndroidHelperSessionScope docblock, the touch-piggyback comment in snapshot-helper-session.ts, and the fill-verification.test.ts header. All four name the am instrument force-stop now, so the corrected mechanism is stated once per place it is load-bearing rather than being one wrong sentence repeated. That is the whole second commit; the first is the probe, its fixtures, and the CHANGELOG.

Gate: check:affected --run, 729 platform-android tests, typecheck, format, lint, check:fallow, check:layering green.

@thymikee

Copy link
Copy Markdown
Member Author

Reviewed at 3de2027, as a follow-up to the review at f4a514b. The signal case is fixed. A release is now read only from the marker with an empty stderr, so a signal-killed read with empty streams is unknown, and the new test fails under the old exit-code rule. Keeping the fix inside snapshot-helper-retirement.ts is smaller than carrying a signal through ExecResult. The live runs in the PR body at 3de2027 reach the marker probe through a real adb shell, including a signal-killed pidof.

One question: remote or cloud Android executors were not checked. If one quotes || per argument, the probe reads unknown on every call. That fails closed, but a pending retirement would then clear only when the next helper session is ready. Is that acceptable?

All 5 checks pass. The PR now conflicts with main. Next: rebase onto main and let CI run on the new head.

The snapshot helper retirement treated the outcome of an `am force-stop` call as the
fact it was supposed to measure. On a loaded host the adb round trip exceeds its
budget while the helper process is already gone, so a completed interaction failed
its own teardown, and the quarantine that followed refused the next command with
"could not confirm release of device automation ownership" on a device that had
nothing holding UiAutomation.

Ownership is a device fact, so it is now read as one: `adb shell pidof` answers
released, occupied, or unknown, and only a device that names a live helper process
may refuse a command. Teardown records what it could not prove for the next acquire
and never decides what the command it is finishing reports, which is what let a
settled `press` turn into a stale-coordinate failure in #2553.

A start that only ran out of time or lost its transport no longer excludes the
helper identity for the rest of the daemon's life; it retries after a cooldown. Only
a helper that ran and exited before announcing readiness proves the identity
unusable.
…swer

resetAndroidSnapshotHelperRuntime carried a second best-effort `am force-stop` with the
same 2s budget the retirement path already owned, and two session fakes answered `pidof`
between them. One stop serves both callers, one fake answers the probe for every test that
steers it, and the release read now looks for a pid token rather than any digit.
A disabled session identity stored an epoch time under a duration's name, and the pending
retirement next to it asked the same question with its own shape. One retry state now answers
both: a value plus when trying again is worth it, standing until then or until the device
settles it.

The helper device key and the occupied-device predicate stay module-owned: nothing outside
platform-android reads them, so they leave the mechanics boundary.
… the caller does

A helper start that failed was answered twice: the caller fell back to the one-shot transport, and
the identity was excluded anyway — for the daemon's whole life in the worst case, which is how a
transient slow start becomes a permanent state. The fallback covers the outcome, so the next
command starts again and nothing stands in front of it.

The wait for readiness also stops being a guess at how long Android needs. It gets the budget the
caller allowed one helper command, which is how `--timeout` reaches it, so a slow device is not
pushed out of the persistent path while the transport it fell back to had room for the same start.

A helper capture's content branches answer one question — is this worth another call — so they
return once, disclosing a system-surface tree instead of giving it its own branch.
@thymikee
thymikee force-pushed the fix/android-helper-ownership-release-2553 branch from 3de2027 to 788944f Compare September 15, 2026 18:11
@thymikee

Copy link
Copy Markdown
Member Author

Rebased onto main as 788944f. The only conflict was CHANGELOG.md, and both sides are kept. The range-diff shows every commit unchanged apart from CHANGELOG context. The 731 platform-android tests and pnpm check:affected --run pass on 788944f. The code verdict from the 3de2027 review still holds, and there are no conflicts now.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Sep 15, 2026
@thymikee
thymikee merged commit 0400701 into main Sep 15, 2026
17 checks passed
@thymikee
thymikee deleted the fix/android-helper-ownership-release-2553 branch September 15, 2026 18:21
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-15 18:21 UTC

@thymikee

Copy link
Copy Markdown
Member Author

Checked at 788944f. The rebase onto main is identical to 3de2027 apart from CHANGELOG.md, where both entries are kept, so the code verdict from the 3de2027 review still holds and the live runs in the PR body still cover the changed path.

All checks pass on 788944f, and there are no conflicts. The earlier question about remote or cloud Android executors that quote || per argument is still open for a human reviewer to accept or reject.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Android: roughly one call in three fails with "snapshot helper could not confirm release of device automation ownership"

1 participant