Skip to content

clean: reclaim finished agent checkouts hourly - #41

Merged
kixelated merged 1 commit into
mainfrom
agent-disk-budget
Nov 6, 2026
Merged

kixelated merged 1 commit into
mainfrom
agent-disk-budget

Conversation

@kixelated

Copy link
Copy Markdown
Owner

Problem

On 2026-11-04 the desktop root disk filled three times, down to 244 MB and then to 5-14 GB, while parallel agent builds ran. Managed mbx targets were about 430 GiB across roughly 420 checkouts. Almost all of that sat on ~/worktrees/moq/t3-* checkouts. Cleaning targets that had been idle for more than six hours freed about 150 GiB, and only 2 of 742 T3 worktrees were at a merged pull-request head.

The cleaner did not scan ~/worktrees, and after the 15-minute timer was removed it ran only when someone started it. kixelated/dotfiles#36 keeps a target until its checkout is gone. This change is what removes a finished checkout, or its target, so that retention rule can release the disk.

Approach

The hourly worktree-clean.timer runs the existing oneshot. It is persistent, with a ten-minute randomized delay. A 15-minute timer spent more time fetching and calling GitHub than the disk needed. An hour matches how these targets accumulate.

~/worktrees is a scan root, next to ~/work, ~/.codex/worktrees, ~/.worktrees, and /tmp.

A checkout is finished only when an owner signal says so:

  • T3: every thread on the resolved checkout path is finished. A thread is finished when it is archived or deleted, or its runs are all terminal (completed, failed, cancelled, interrupted, rolled_back) and every session is stopped or absent. One non-terminal run or any other session status keeps the path. A path with no terminal run and no archived thread is still being created.
  • Quest: phase is released and runs is empty. Any other phase, a supervised run, or an unreadable registry keeps a .worktrees checkout.
  • Claude Code: a project transcript (*.jsonl) exists for that .claude/worktrees path. No transcript falls through to the git rules.

An unknown t3-* directory under a worktrees path stays, including when the T3 database cannot be read. A database or registry error is not treated as finished. Idle time, mtime, and T3's worktreeAfterDays are not signals.

Git completion still applies to ordinary worktrees. A closed pull request's head is abandoned work, so its history qualifies. Commits after that head do not. A detached HEAD still needs a surviving local branch or tag, including when the agent signal says the checkout is finished. Branches are not deleted.

A finished, clean, idle checkout is removed with git worktree remove, which already runs mbx cache remove. A finished checkout that is dirty or untracked stays, and mbx cache remove runs for its managed workspace targets after a fresh process check. A process in the checkout or the target keeps both.

/tmp is still scanned only for registered worktrees. Age does not delete test scratch such as /tmp/moq-test-*. On 2026-11-05 /tmp is its own filesystem (/dev/nvme0n1p1, 627G free of 954G). A du of /tmp showed about 159G, of which /tmp/moq-test-1000 was 78G of failed-test evidence. mbx attributed 0 managed-target bytes to /tmp.

Impact

Read-only classification on 2026-11-05, using this gate and an mbx project snapshot from the same day. This did not delete anything.

  • Checkouts one level under ~/worktrees/* and ~/work/*: 873 finished, 15 busy, 295 untracked. Finished paths accounted for about 76.6 GiB of managed targets. Busy paths accounted for about 3.6 GiB.
  • The busy set includes the checkouts still in use: ~/worktrees/moq/t3-cced8ddf (1.2 GiB, run in progress) and ~/worktrees/moq.pro/t3-00f61769 (2.4 GiB, run in progress). This session's checkout t3-825df977 is busy and had no managed target in the snapshot.
  • The largest finished managed targets were same-day completed threads: t3-fb8bc7d8 35.5 GiB, t3-384cf58d 12.8 GiB, t3-762e4a05 12.7 GiB, t3-e9c7666c 11.4 GiB.
  • Claude checkouts under ~/work/*/.claude/worktrees: 279 directories, 33 with a transcript and no managed-target bytes, 246 with no transcript. The 5.10 GiB checkout agent-a42ff81308526f267 has no transcript, so it stays unless git can prove completion.
  • Quest checkouts under ~/work/*/.worktrees: 7 released, 4 busy, 9 untracked, and no managed-target bytes in the snapshot.
  • Root disk at classification time: 210G free of 1.9T (89% used).

Action-cache figures overlap across checkouts, so they are not added to these totals. The 76.6 GiB figure is the managed-target upper bound the gate would allow. The cleaner still keeps a dirty tree and anything a process has open. The timer has not run for a day on this host because this PR does not install over the live dotfiles.

python3 -B -m unittest discover -s tests -v passed, 92 tests. systemd-analyze --user verify accepted the service and the timer.

Alternatives

Routing T3 checkouts through quest worktree would give one supervisor, but T3 owns ~/worktrees and recreates a missing worktree. Quest only supervises .worktrees/ checkouts it created. Extending this cleaner reclaims the checkouts T3 already has.

Reclaiming on idle time, or on T3's one-day worktreeAfterDays, would delete a checkout whose owner is between turns. The same-day targets above would also sit until the next day.

Leaving cleanup manual keeps the disk full until someone notices. A 15-minute timer is more GitHub and fetch load than this disk needs.

Changing target.keep belongs to #36. This PR does not edit mbx/config.toml. An explicit mbx cache remove after a finished signal is the removal #36's retention allows.

Follow-ups

  • Install this branch, or merge it, before the hourly timer can reclaim the finished targets above. Do not mass-delete them from an agent session.
  • Claude checkouts that lost their project transcript keep their targets until a merged or closed pull request covers the commit. That is the safe side of the retention rule.
  • fix(mbx): retain targets until their checkouts are removed #36 should still land so a target whose checkout is already gone can be collected by mbx itself.

Choices made without asking

  • Extend the dotfiles cleaner instead of sending T3 checkouts through quest worktree.
  • Hourly timer (OnCalendar=hourly, Persistent=true, RandomizedDelaySec=10min), not 15 minutes and not idle-time reclaim.
  • Finished means a terminal T3 run with a stopped or absent session, an archived or deleted T3 thread, a released quest checkout with no runs, or a Claude project transcript. Unknown t3-* paths stay.
  • A finished dirty checkout keeps the git checkout and drops only the managed target, after a process check.
  • A closed pull-request head is completion of that head's history. An open unmerged head is not.
  • Do not age-sweep /tmp and do not change mbx target.keep here.

(Written by Grok 4.7 with high effort)

kixelated added a commit to moq-dev/moq that referenced this pull request Nov 5, 2026
Finished T3 and Claude Code checkouts reclaim their managed targets from
the dotfiles cleaner on an hourly timer. That work is
kixelated/dotfiles#41. Delete this quest.

Co-Authored-By: Grok 4.7 <noreply@x.ai>
Scan ~/worktrees and release a managed target when its T3 thread, quest
checkout, or Claude Code transcript is finished. A live session, a
non-terminal run, an unknown t3-* path, or a process in the target keeps
it. Idle time is not a signal. Closed pull-request heads count as
abandoned work. The hourly timer replaces the manual-only service.

Co-Authored-By: Grok 4.7 <noreply@x.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant