Skip to content

fix(android): report the clip an Android recording really captured - #2566

Merged
thymikee merged 4 commits into
mainfrom
t3code/fix-2551-quality-refactor
Sep 14, 2026
Merged

thymikee merged 4 commits into
mainfrom
t3code/fix-2551-quality-refactor

Conversation

@thymikee

@thymikee thymikee commented Sep 13, 2026

Copy link
Copy Markdown
Member

Summary

Android screenrecord encodes a frame only when the screen changes, so a window ending on an unchanged screen returns a video far shorter than the requested duration while record stop reported only host wall-clock durationMs. record stop now also reports capturedDurationMs, measured from the pulled MP4 timelines, and warns with the clip length against the window the host bracketed around the recorder: Date.now() immediately before the recorder launches and immediately before the stop signal. A host that slept mid-recording reports a shorter window, which costs the warning rather than inventing one. iOS is unchanged.

$ agent-device record stop --json
{ "durationMs": 15402, "capturedDurationMs": 6736,
  "warning": "…it covers 6.7s of the 13.0s recording window." }

Closes #2551. 30 files, inside the recording family plus its contracts, MCP schema, help and docs. Measuring the timeline needed an ISO-BMFF box walk, which replaces the private atom scan MP4 container detection used to do.

Validation

Rebased onto 04052fdcd2; conflicts were keep-both merges over #2564's probeRunningWriters op and its manifest-retirement help. Human review findings 1 and 2 are in d4c0ad2adb: the device clock is gone and the window module owns the clock arithmetic.

  • Live on emulator-5554 at d4c0ad2adb: swipe then 10s idle → capturedDurationMs: 6736 against ffprobe duration=6.736144, warning above, recorder and session gone. Earlier heads measured 13102/13.101478 and 9733/9.732689.
  • pnpm check:affected --run at d4c0ad2adb: every gate passed except mutation-model, which fails identically at 04052fdcd2 (a kernel is owned by tests that reach it indirectly) and is not this PR's. Validation is therefore not fully green.
  • Coverage and Integration Tests passed at 625dc00bab; both rerun on this head.

@github-actions

github-actions Bot commented Sep 13, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-14 11:54 UTC

@github-actions

github-actions Bot commented Sep 13, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
Installed (including dependencies) 4.53 MB 4.53 MB +1.9 kB
Package (unpacked) 4.53 MB 4.53 MB +1.9 kB
Package (download) 1.34 MB 1.34 MB +720 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 25.6 ms 25.0 ms -0.6 ms
CLI --help 69.4 ms 71.3 ms +1.8 ms

@thymikee

Copy link
Copy Markdown
Member Author

No blocking findings at e590e85. The measured duration reaches the response and recovery metadata, and the short-clip warning reaches normal CLI output. The reported Android run matches ffprobe and current checks are green. The shared MP4 reader replaces the existing scan and reasonably accounts for the size increase; ready for human review.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Sep 13, 2026
Android `screenrecord` encodes a frame only when the screen changes, so a window that ends on an
unchanged screen returns a video far shorter than the requested duration, and `record stop` had
nothing to say about it: the reported `durationMs` is host wall clock from `record start` until the
export finished, which is not the length of the file that was just pulled.

`record stop` now measures the pulled MP4 timelines and reports them as `capturedDurationMs`, and
warns with the clip length against the window when the video is two or more seconds short. The
window is measured on the device's own elapsed clock, read before the stop signal and at launch,
because host wall clock drifts against the clock the encoder timestamps frames with; an unreadable
clock or a chunk that answers no duration costs the caller the claim, never the recording.

Measuring the timeline needed an ISO-BMFF box walk, which now lives in
`@agent-device/capture-kit/recording-mp4-duration` and replaces the private top-level atom scan
that MP4 container detection was doing. Stop replay through daemon recovery carries the field too,
so a completion read back from the session resource reports the same numbers it did live.
…n holds

`@agent-device/capture-kit/recording-mp4-duration` and its fixture sibling are new declared package
subpaths, so the boundary enumeration that holds every exported workspace subpath has to name them
for the layering scan to accept the Android recorder's read of a pulled clip's timeline.
…ated

The Coverage job's ADR-0019 eager-closure probe failed: `recording/video.ts` evaluated 25 modules on
import where the merge-base evaluated 24, because the MP4 container gate statically imported the box
walk it now shares with the clip-duration read, and `recording/overlay.ts` grew by the same module.
An entry the merge-base already carries gets no growth budget, so the edge moves behind a
function-scoped `await import`: the scan is something recording completion asks for, and importing
this module for `waitForStableFile` or WebM detection should not evaluate a box walker.

The alternative the probe offered -- hosting the walker in a module both growing entries already
evaluate -- would have put an ISO-BMFF walk in `swift-cache.ts` or `video-webm.ts`, or made the
duration read import the Swift validator machinery that sits behind `video.ts`.
@thymikee

Copy link
Copy Markdown
Member Author

Human review at 625dc00. The MP4 side is fine: the walker is a fair replacement for the old top-level scan since it has to descend into moov, and the sum-over-chunks policy and warning text are right. The weight of this PR is elsewhere, and two of the three parts are avoidable.

1. The device clock is not needed; use host time.
elapsedUptimeMs on the transport, the 1.5 s probe budget, device-clock.ts + test, and two extra adb round trips per recording (one on the start path) all exist to defend against host-vs-encoder clock drift. Quartz drift is tens of ppm; over a 30-minute chunked recording that is under 100 ms against a 2 s threshold. Date.now() taken at the same two points (before startChunkAt, before the stop signal) gives the same window with zero device I/O and no contract change. The one case where the clocks really diverge is a host that sleeps mid-recording, and there the host window comes out shorter, so the warning is missed rather than invented. That is the safe direction. This removes roughly a third of the production diff.

2. Merge device-clock.ts into captured-window.ts.
Whatever clock code survives is two tiny functions about the concern the window module already owns.

3. The function-scoped await import('./mp4-atoms.ts') is a gate artifact.
The old scan was inline in video.ts (closure = 1 module). Splitting it out made it 2, and the eager-closure rule for existing entries is no-growth with no approval path, so the gate punishes splitting a file and rewards a fake-lazy import of a 99-line dependency-free module. Keep the dynamic import for this PR so it lands, but I will file a follow-up to let an existing mechanics-surface entry grow within its category ceiling. Once that is in, video.ts imports the walker statically and readMp4DurationMs can live on the existing recording-video subpath instead of a new one.

4. Not for this PR: capturedDurationMs touched seven sites (two contracts, MCP schema, response builder, metadata encoder, and two hand-written validators in different packages checking the same shape). A single codec for ScreenRecordingCompletion in contracts, shared by the daemon store and the Android manifest, would make the next field a two-site change. Separate PR.

Requested changes: 1 and 2.

Human review of #2566: the device-clock read defended against host-vs-encoder drift that does not
matter at this threshold. Quartz drifts by tens of ppm, so a 30-minute chunked recording moves the
window under 100 ms against a 2s warning threshold, while the read cost a transport operation, its
own probe budget, and two adb round trips per recording. The window is now the host elapsed time
between `Date.now()` immediately before the recorder launches and `Date.now()` immediately before the
stop signal, so the contract change and the extra device I/O are gone, and the one case where the
clocks genuinely diverge -- a host that sleeps mid-recording -- reports a shorter window and misses
the warning rather than inventing one.

The surviving clock arithmetic is one subtraction, so it lives in the window module that already owns
that concern instead of a module of its own. A stop recovered through daemon recovery now passes the
manifest's own start instant, which is the first host timestamp the recording ever had, so a recovered
stop gets the same comparison a live stop gets.
@thymikee

Copy link
Copy Markdown
Member Author

Addressed in d4c0ad2adb.

1. Device clock removed. elapsedUptimeMs is off the transport contract and out of the adb host, the 1.5s probe budget and both round trips went with it, and AndroidScreenRecordingTransport is back to the shape it had at the merge-base. The window is now Date.now() taken immediately before startChunkAt and immediately before the stop signal, which keeps the one property the read existed for — export latency is not attributed to an unchanged screen — at zero device I/O. A stop recovered through daemon recovery now passes the manifest's own startedAt, the first host timestamp the recording ever had, so a recovered stop gets the comparison it did not have before.

2. device-clock.ts is gone. What survived is one subtraction, and it lives in captured-window.ts, which already owned the comparison; measureCapturedWindow takes startedAtMs/stoppedAtMs and the callers take no part in the arithmetic.

Live at d4c0ad2adb (emulator-5554, one swipe then 10s idle): capturedDurationMs: 6736 against ffprobe duration=6.736144, warning naming 6.7s of the 13.0s recording window, and durationMs: 15402 — the 2.4s between the host window and the reported duration is the export, exactly where it should be. pnpm check:affected --run at this head passes every gate except mutation-model, which fails identically at 04052fdcd2.

3. Left the function-scoped await import as you asked. When the ceiling change lands, the two-site follow-up is video.ts importing the walker statically and readMp4DurationMs moving onto recording-video; the recording-mp4-duration and recording-mp4-fixtures subpaths added here would fold back at the same time.

4. Agreed, and the count is worse than the comment says for the next field: the daemon metadata encoder, both hand-written validators, the stop-recovery reader, the response builder, two contracts and the MCP schema. A ScreenRecordingCompletion codec in contracts shared by the session resource and the Android manifest is the fix; leaving it out here.

@thymikee
thymikee merged commit 6db1a42 into main Sep 14, 2026
20 checks passed
@thymikee
thymikee deleted the t3code/fix-2551-quality-refactor branch September 14, 2026 11:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Android screen recording stops capturing about one second after the last input, regardless of how long record stop waits

1 participant