Repository navigation
Add durable goals with scheduled and event-driven execution - #19
josephschorr wants to merge 17 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
a712433 to
48f371e
Compare
b1910e0 to
4982a52
Compare
05242e4 to
52b5f73
Compare
52b5f73 to
9157768
Compare
samkim
left a comment
There was a problem hiding this comment.
Review focused on design and on whether this preserves the existing security and authorization principles. Two inline comments cover the main issues: reconcile errors on finished goal sessions, and CLI-asserted goal consent.
Minor findings. None of these block merge. Each was checked against the current tree by an independent verifier.
- No operator-side plan check for manual goals (
pkg/web/goals/server.go:126-150).report_execution_resultandprepare_replydon't check that an approved plan phase exists. Today only the runner's enforcing plan gate (RequirePlan, no auto-approval) keeps the "each run asks you to approve a plan" promise. That is the same trust every plan gate places in the runner, butstanding_privatealready re-checks on the operator, so manual mode could too. - The discovery-proposal consent card hides most of what it grants (
pkg/web/goals/discovery.go:259-265). It shows only Monitor/Until/Limits. It is the only execution consent in this flow, because the policy card says "This does not authorize execution", but it grants the template's operations, action approval and destination unchanged. It should show the same terms asconsentEntry. Monitor is untrusted observation text (up to 256 chars) and should be shown clearly as quoted data. - Reply sources aren't rechecked before delivery (
pkg/controllers/goals/reply.go:124).processReplyre-validates the goal but noto.Reply.Sources. The window is short and the recipient is the owner, so the impact is low, but recheckingReadGoalon the reply's sources beforeAcceptis cheap. resolve()writes to SpiceDB on every call, including reads (pkg/web/goals/server.go:460-463). The writes repeat the existing grant, butSlotGrantExpiry(time.Now(), 0)makes each one a real SpiceDB write. It runs several times per request and from 5 places on each discovery tick. Consider doing the grant once at session start, or skipping it for read operations.standing_privatebehavior contradicts the consent card (pkg/web/goals/standing.go:66-74vsconsent.go:289,308). Extra actions get a hard deny, but the card says "Other actions require separate approval". Either fall back torequestPhaseApprovalfor extra actions or change the card text. Denying on operator errors is intentional and tested, so that part should stay.- One unreadable run blocks the whole run list (
pkg/agent/goals/occurrence.go:148-160). This is intentional and pinned byexecution_test.go:529, but it differs fromList, which skips goals the owner can no longer read. Consider leaving out only the affected runs so the rest of the history stays visible.
Two findings were checked and refuted: the p.K8s == nil fail-open in interaction_decision.go:63, which only tests can hit (refusing there would still be tidier), and the claim that ConstrainGoalSettings drops plan-gate limits. PlanGate.Limits isn't read anywhere.
| if sess.Spec.GoalExecution != nil { | ||
| if r.GoalValidator == nil { | ||
| return ctrl.Result{}, fmt.Errorf("goal session validator unavailable") | ||
| } |
There was a problem hiding this comment.
Goal sessions get stuck in reconcile errors once their run finishes
This check runs on every reconcile that isn't a deletion, before any status work. ValidateGoalSession returns ErrDenied in any of these cases:
- the occurrence isn't
Running Outcome != nil- the session has outlived
Bounds.DurationSeconds(pkg/controllers/goals/dispatcher.go:373,383)
The dispatcher records an outcome as soon as the runner pods succeed or the phase becomes Succeeded/Failed (dispatcher.go:177-199). It then keeps the session for the inspection window (retainCompleted, dispatcher.go:472-485). For that whole window, every reconcile returns goal session activation refused: denied and backs off. As a result:
- the phase transition and
auditChainHeadsstamping (controller.go:1290) never run for a finished goal session, sooap audit verifyloses the tail anchor - the error repeats in operator logs until the session is deleted
- deletion still works, because
finalizeruns first
No agentsession test covers GoalValidator, which is why this got through.
Suggested fix: limit the check to activation. One way is to validate only on the path that creates the runner pod or first moves the session out of Pending. Another is to split the validator so that a terminal occurrence with an outcome still allows status and completion reconciles but refuses to (re)start a pod. Please add a test that records an outcome on a retained session and asserts that reconcile still completes and stamps auditChainHeads.
There was a problem hiding this comment.
Fixed in b947590. A new GoalSessionConcluded check routes a finished goal session (attached to its occurrence, run ended) through the terminal reconciles: chain heads, reap, GC. It returns before the AgentClass resolve, so it never reads credentials, forks, wakes, or starts a pod. A session the occurrence never attached still gets the full refusal. Tests are in goal_concluded_envtest_test.go: it stamps auditChainHeads, never creates a runner pod, and an unbound session is still refused.
| if err := channelevents.PublishIn(publishFn, b.Namespace, sessionName, | ||
| dispatch.decisionKind, pl); err != nil { | ||
| // The CLI credentials authorize this component-only subject. Runner | ||
| // credentials cannot publish here, which protects included goal consents. |
There was a problem hiding this comment.
The component subject proves a component sent the message, not that a human decided
The comment here says runner creds can't publish on this subject, "which protects included goal consents." That part holds: the runner's per-session grant (pkg/controllers/agentsession/controller.go:4163-4171) doesn't include it. But validateGoalConsent then treats arrival on this subject as proof that the owner decided. The only identity check is that the decider's canonical principal equals Goal.Domain.Owner (pkg/channels/channelsd/pipeline/goal_consent.go:111). On the CLI path, that principal is cliApprover, a self-declared --approver-email with Kind: "cli" (approve.go:128-133).
So anyone holding spicebox-cli-nats-creds can approve any owner's goal consent with oap session approve <session> goalconsent-request-<digest> --approver-email <owner>. That includes unattended standing_private goals. channelsd signs the decision witness as system:channelsd, and the operator writes the approved tuple.
Some context on how much this matters:
- These creds already have
PubAllow: ap.>(installcmd/install.go:3607-3611), so the holder can already forge most bus traffic. - Self-declared
--approver-emailfor tool approval already exists onmain. oap agent chatalso loads these creds, so in practice they are not admin-only.
What's new is that goal consent, which is supposed to be the human's exact-terms approval of unattended execution, now accepts this path.
We should pick one of these:
- Refuse CLI-sourced deciders for goal consent.
validateGoalConsentwould rejectKind == "cli"(or anything other than a verified channel identity), so consent can only come from a channel where the transport authenticates the human. - Require a verified CLI identity. Bind the CLI decision to an authenticated principal, for example a signed token from the kubeconfig user, instead of an email the caller types.
- Accept it and document it. Holding the CLI NATS creds would mean being able to consent as any owner. This should be stated next to the
standing_privatedocs and in this comment.
I'd lean toward (1) for now. Goal consent is the one decision where the "a human decided" claim is the whole point.
There was a problem hiding this comment.
Went with (1) in b947590. A new ChannelDecisionOnly category flag is set on goal consent. With it, channelsd rejects a cli decider both directly and when the consent is included in a plan card, and oap session approve refuses it up front. The flag doc says this blocks the CLI path but is not a bus boundary, since the CLI NATS creds can still forge a non-CLI decider. Option (2) would be the follow-up.
|
Minor findings, addressed in b947590:
|
|
Follow-up on b947590. That commit stopped the reconcile errors on concluded goal sessions, but chain heads were still not anchored on the most common way a goal run ends. Pushed 0174a75 to fix that. The gap. Goal sessions have an input channel, so a run normally ends with the runner's Idle yield ( The fix (0174a75):
Tests:
Verification: every package under The remaining minor items from the second review pass are still open: the source check before |
Introduce user- and class-scoped goals with revisions, idempotent mutations, source dependencies and retained audit events. Add management tools, CLI commands and HTTP endpoints backed by durable relational stores. Keep desired outcomes separate from permission to execute work, and check current ownership and source access on reads and changes.
Require exact consent to a pinned agent class, private destination, launch window and execution limits. Dispatch through a durable occurrence ledger with worker leases, fencing and deterministic session identities. Recheck consent and account authority at activation, constrain delegated execution, and expose only the operations allowed for the bounded run.
Record run proposals separately from delivery receipts and terminal outcomes so retained history does not claim effects that were not verified. Reconcile uncertain private deliveries through a durable idempotent receiver. Add generic asynchronous-session summaries with inspectable instructions, and replay interaction cards, decisions and plan snapshots consistently after refresh across conversation transports.
Accumulate provider-reported tool spend alongside model usage without double-counting buckets. Retain per-run estimates after session cleanup and distinguish final accounting from incomplete lower bounds.
Select a ready running service pod when establishing local connections, ignoring retained terminal pods and terminating rollout replicas instead of forwarding to the first listed pod. Cover that selection in CLI tests.
Prepare exact reminder consent before freezing an action plan and decide it together with the phase approval. Retain the consent and its outcome for replay and reject preparation in bounded or delegated sessions. Present compact consent details, alert on goal-created browser sessions, and explain how to begin when no agent is installed.
Introduce a reusable scheduled-session model for once, interval, daily and weekly runs. Resolve finite occurrence windows in an explicit timezone and defer quiet-hour runs while coalescing collisions. Retain schedule identity through retries, skip missed windows and recheck goal revision, consent and capacity before each launch.
Separate launch consent from action approval. Support explicit standing consent for unattended private delivery within reviewed operations, recipient, schedule and execution bounds. Derive approval only for the exact frozen plan and phase under live authority; ordinary sessions and manual consent retain human approval.
Add a reusable private observation ledger with source incarnation pins, dependencies, witnesses and publisher checkpoints. Register source adapters and validate ingress independently of agent-provided content. Persist finite predicates, admission decisions and launch requests so event redelivery and worker turnover cannot create duplicate sessions.
Connect finite event subscriptions to bounded goal occurrences. Recheck source access and execution consent before materializing an event launch. Allow separately authorized discovery policies to propose private monitors with retained evidence, expiry and question limits. Require exact consent for each proposed monitor before creating its goal and watch.
Bind consent and audit recovery to verified actors, revisions and signed records. Enforce temporal, operation and destination bounds through revocation, cleanup and worker recovery. Keep event scans fair under failing work and bound pending admissions. Serialize approval timeout decisions and restore the original reviewed card and recorded outcome instead of synthesizing a new approval.
Return server time, conservative class-capped bounds and actionable validation errors during goal setup. Discover configured event feeds through registered source adapters with current access checks. Exercise scheduled and event-driven workflows through the full local runner, including private delivery, standing approval and recovery.
Allow explicitly authorized runs to attach scoped observations to their retained results. Publish through an idempotent outbox with source dependencies, live authority checks and audited permanent rejection. Expose stable goal-produced feeds for downstream event watches. Release execution capacity after verified runner termination while retaining the conversation for inspection, so dependent runs can proceed promptly.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Let a goal session whose run has ended reach its terminal reconciles. Activation validation refuses such a session by design, which blocked chain-head anchoring, reap and GC for the whole retention window. The reconciler now routes a concluded session through those and returns before anything that reads credentials or starts a runner. Refuse CLI-asserted deciders for goal consent. A new ChannelDecisionOnly category flag makes channelsd reject a "cli" decider, directly or through a plan card that includes the consent, and oap session approve refuses it up front. Also: recheck reply sources before delivery, show the full terms on the discovery proposal card, stop rewriting the scope binding on every reauthorization, make the standing_private card match its enforcement, and leave unreadable runs out of the run list instead of failing it.
A goal run normally ends with the runner's Idle yield and the pod exiting, which the dispatcher records as RunSessionEnded and retains. The concluded-session early return skipped every sweep that moves an Idle session on, so these sessions stayed Idle until the dispatcher deleted them and never had their audit chain heads anchored. GoalSessionConcluded now also reports whether the occurrence is retained, which the store admits only after a clean end with every runner pod succeeded. For a retained run the reconciler appends a new GoalRunConcluded lifecycle event and marks the session Succeeded ahead of chain-head anchoring. Unlike ArchiveSweep it does not park the session as wakeable. A SessionHold defers the transition, as it does for the archive sweep. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H2YWxVxmFNinVPenAbuBTn
0174a75 to
2cfedc1
Compare
Motivation
Agents currently perform work within individual sessions. Users also need to describe an outcome once, retain it across sessions, and have approved work resume when it is due or when relevant information changes.
Summary of changes
Adds opt-in, private goals that persist across sessions, with scheduled and event-driven execution. Goals can run once or on a finite recurring schedule, respect quiet hours, and react to changes from accessible information sources.
Execution stays within reviewed limits. Users can approve each run or explicitly authorize unattended private reporting for a defined period. Agents can also request permission to suggest new monitoring goals; each suggestion requires its own approval.
Results, delivery evidence, approvals, and cost estimates remain available after a run ends. Goal-created sessions alert the user, show a concise summary, and keep their exact instructions inspectable. Refreshing a conversation preserves its recorded approval state.
How it is used
Enable goals for an agent, then create and manage them through conversation or the CLI. Review the proposed schedule or event conditions, recipient, duration, and execution limits before approving. Pause, update, or cancel the goal as needs change, and inspect its retained run history to see what happened.
Scheduled goals can publish private observations that other goals use to react to changes. Execution is limited to approved private reporting and notifications.
Review structure
The commit series separates durable goal management, bounded execution, retained conversation state, scheduling, event handling, and recovery. Generated outputs and regression coverage accompany the changes they support.
This is a draft stacked on #16 so its diff contains only the goals feature and supporting changes.
Validation
mage test:unitmage test:integrationmage test:e2eBrowser tests and both TypeScript projects passed. Storage-dependent browser tests were rerun with the local Node 26 compatibility setting.
API schemas, install manifests, and CLI/CRD references were regenerated. Documentation checks, tests, type checks, production build, and search indexing passed. The documentation build used webpack with the installed local dependencies.