fix(daemon): stop capping how many projects can enrol - #718
fix(daemon): stop capping how many projects can enrol#718ScriptedAlchemy wants to merge 5 commits into
Conversation
Enrolling a fourth project failed with 'graph capacity budget exhausted (limit 8)', and once exhausted even a read-only projects list failed. Two graph owners per mounted project plus two profile-wide ones is exactly 8, so three projects filled the budget. Raises both ceilings to runaway guards instead of population bounds, and records why LRU cannot rescue this today: a mounted project holds its owner attachment for the life of the mount, so the eviction candidate search can never select it. Genuine elasticity needs idle-project hibernation, which is follow-up work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two ceilings raised alongside the project ceiling were picked as round numbers rather than derived, and both were wrong in a way the advertised capacity hid. The graph registry slot ceiling was two slots short. Every mounted project holds its own graph owners on top of the two profile-wide ones, so 8,192 slots covered only 4,095 projects: the 4,096th failed inside GraphDbRegistry with a capacity budget error before ever reaching MAX_RETAINED_PROJECT_RUNTIME_OWNERS. It is now written as arithmetic over the project ceiling, the profile-wide owner count, and the graph owners one project can hold, so the two cannot drift apart again. The per-project figure is 3, not 2: RecoveryRequired retains the outgoing session owner alongside the candidate and Faulted retains both the retained and the faulted one, and none of them is reclaimable while the project stays mounted. The remote node owner ceiling disagreed with credential registration. Raising it to 4,096 left DaemonRemoteCredentialAuthorityV1 refusing register_storage at 128, and those two checks refuse at different points in the mount: admit_remote_node_owner refuses before anything is published, while register_storage refuses after mount_remote_node_storage has already published the runtime owner and provisioned the node's remote.db. Because mount_registered_remote_nodes remounts every discovered remote.db at daemon start with `?`, a rejected 129th provisioning left residue that failed every later start -- worse than the refusal it replaced. The mount ceiling now derives from the credential registry's MAX_REGISTERED_REMOTE_NODES, so the earlier, residue-free refusal is authoritative by construction. Unifying upward was rejected: the remote fleet size is a Remote Brain decision unrelated to how many local projects a profile enrols, and this PR raises only the latter. Rolling the mount back instead was also rejected. The in-memory owner is the easy half; the residue that wedges startup is the provisioned remote.db, and deleting a just-provisioned enrollment database is a destructive path this change does not need. Guards cover both, and each fails on the exact defect it names. With the shipped values they report "graph slot ceiling 8192 cannot admit 4096 projects: they need 12290 slots", "project 4096 is refused: 12287 of 8192 graph slots are already taken", and "remote mount ceiling 4096 exceeds the credential registry ceiling 128". A behavioural test in tracedecay-graph-db grounds the arithmetic in real registry behaviour: at the derived ceiling every owner attaches, and one slot short the final owner is refused with a capacity budget error, because a held attachment is never evictable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…plan-reopened' into HEAD
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b6f6223a7b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| /// This is the *only* declared project-population ceiling. Every other ceiling | ||
| /// that has to admit the same projects is derived from it below, so the | ||
| /// advertised capacity cannot silently become unreachable again. | ||
| const MAX_RETAINED_PROJECT_RUNTIME_OWNERS: usize = 4_096; |
There was a problem hiding this comment.
When a long-lived daemon successively touches many projects, every Ready entry remains in project_owners and its graph attachment is not evictable, so increasing this ceiling 512× permits resident memory to grow until the process or system OOMs instead of returning the previous typed capacity refusal. Add resident-memory admission plus idle-owner hibernation or eviction before increasing this bound; the commit description itself identifies that as the required safe implementation.
AGENTS.md reference: AGENTS.md:L126-L128
Useful? React with 👍 / 👎.
| assert!( | ||
| MAX_RETAINED_GRAPH_DB_OWNERS >= required, |
There was a problem hiding this comment.
If the production mount path begins retaining another owner, changes admission ordering, or leaves residue after a late refusal, this assertion still passes because MAX_RETAINED_GRAPH_DB_OWNERS is defined from the same constants used to compute required; the sibling project and remote guards repeat the same source-shape check. Exercise bounded daemon mounts and the refusal/cleanup behavior instead so the tests can detect the production regressions they claim to prevent.
AGENTS.md reference: AGENTS.md:L85-L90
Useful? React with 👍 / 👎.
|
Closing as evidence-only and unsafe to merge. Raising retained project ceilings without resident-memory admission or idle-owner eviction trades a typed refusal for OOM risk, and the arithmetic-only guards do not exercise daemon mount behavior. |
1 participant
Reopened against
codex/tracedecay-total-redesign-plan-reopenedfor visibility, not for merge as-is.A prior version was reverted as "unsafe unmetered capacity expansion" and that bar is right: raising ceilings without resident-memory admission trades a typed refusal for possible OOM.
What this branch has that is worth keeping is the arithmetic and the structural finding: two graph owners per mounted project plus two profile-wide is exactly 8, so three projects fill the budget; and
reserve_capacity_evictioncan never reclaim a live project because its owner attachment is held for the life of the mount, so nothing bounds residency today.The owner's requirement is "no cap on projects that we can enroll". The version that satisfies both is resident-memory-admitted elasticity, not this constant bump.