Repository navigation
Decisions proposed during --replay look like live ones #36
Description
What happens
cloudfall agent investigate --replay <recording> (#34) plays recorded check output back to a live model. Decisions the model proposes during a replay are saved in --decisions like real ones, and nothing in the decision record says their check output was played back; only the investigation record names the replay.
An operator could approve one of these as if its check had run on the host.
Expected
A decision proposed during a replay can't be mistaken for a live one. Options raised in the review of #34 (#34 (comment)):
- Mark replayed decision records (for example a
replayfield naming the recording), and refuse or warn on approving one - Write replayed decisions to a separate default folder
- Leave it as is
Activity
shipmill-romamo commentedon Oct 8, 2026
Triage: implement, once the design below is settled. Waits on your decision in the next comment.
Confirmed on
main(9cb777c):agent investigate --replay(sdk/src/cloudfall/app.py,agent_investigate) swaps the check runner for the recording, but--decisionskeeps its default, so decisions proposed in a replay are saved in the same folder as live onesreplay.decisions list,whyanddecisions approvecan't tell them apart--replay decisionsand the default--decisions decisions, and the replay writes its new records into the recording it is playing back.Recording.loadkeeps the first run of each call, so this mostly doesn't change later replays, but a call the recording didn't hold is saved there with exit 3 and its "replay: ... holds no run" outputDesign. This changes stored state (the decision record) and maybe a default, so it goes through the design gate. No decisions log exists yet. The decision record schema is also changed by #37 (
approval.via) and maybe #26 (status values), so whichever lands second rebases on the other.Compatibility for every option: records written before the change carry no replay marker and read as live, which they were.
shipmill-romamo commentedon Oct 8, 2026
@romamo Decision needed: how should a decision proposed during
agent investigate --replaybe kept apart from a live one?replayfield naming the recording,decisions listandwhyshow it, anddecisions approverefuses it withPRECONDITION(its check output was played back, not run).--replayalso refuses a--decisionsfolder that is the recording itself. Additive schema field--replay,--decisionsdefaults to something liketmp/replay/decisionsinstead, and records are not marked. Nothing in the schema changes, but a record copied or pointed at by hand looks live againReply here with a number or your own answer; shipmill takes this up on the tick after your reply.
romamo commentedon Oct 9, 2026
1 — mark it with a
replayfield and havedecisions approverefuse it withPRECONDITION;--replayalso refuses a--decisionsfolder that is the recording itself(decision by @romamo, given in chat; posted by Claude on his behalf)
shipmill-romamo commentedon Oct 9, 2026
Triage: implement, taking up the answer above (option 1), recorded as D-2 in #51.
Plan: a decision proposed during
agent investigate --replaygets an optionalreplayfield naming the recording (additive in the v1 schema; older records have none and read as live);decisions listandwhyshow it;decisions approverefuses it withPRECONDITION;--replayrefuses a--decisionsfolder that resolves to the recording itself.#26 also changes
operation-decision.schema.json; the same implementer builds both on separate branches, and whichever merges second rebases on the other.shipmill-romamo commentedon Oct 9, 2026
PR: #55. A decision proposed during
--replaycarriesreplay(the path as given to--replay, the same value the investigation record stores);decisions listandwhyshow it;decisions approverefuses it withPRECONDITION/decision_replayedbefore asking for the id;--replayrefuses a--decisionsfolder that resolves to the recording. #53 (#26) touches the same files, so the second to merge gets rebased.romamo commentedon Oct 9, 2026
Decision by @romamo, given in chat; posted by Claude on his behalf.
1. Mark it and refuse to approve it: the decision record gets a
replayfield naming the recording,decisions listandwhyshow it,decisions approverefuses it withPRECONDITION, and--replayrefuses a--decisionsfolder that is the recording itself.Mark decisions proposed during a replay and refuse to approve them
shipmill-romamo commentedon Oct 9, 2026
Released in v0.6.0rc9.