Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/fast-restart-closeout-unrecoverable.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@roomote/web': patch
---

A Fast turn whose durable admission write failed before it ran no longer ends in silence when a restart cuts it off. Every other turn already resumes on the next process; this one had no row to resume from, so the turn disappeared and the Session stayed marked responding. The turn is now admitted late at the moment of interruption and handed straight to the queue, which resumes it the same way. Only if that late admission also fails does the turn post "Roomote restarted while working on this request. Please send it again." Queue-delivered follow-ups stay quiet because the queue re-runs them itself, and platform events keep their existing handling.
9 changes: 8 additions & 1 deletion apps/api/src/handlers/discord/fast-agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import {
import {
admitFastAgentHumanFollowUp,
createFastAgentConversationArtifact,
handOffFastAgentInterruptedTurn,
persistFastAgentInlineHumanTurn,
recordFastAgentConversationMessageBestEffort,
resolveUserMcpServerConfigs,
Expand Down Expand Up @@ -374,7 +375,13 @@ export async function processDiscordFastAgentMessage(
retryAt,
),
}
: {}),
: {
requestLateDurableAdmission: () =>
handOffFastAgentInterruptedTurn({
parent: { sessionId: session.id, conversation },
event: humanFollowUpEvent,
}),
}),
resolveMcpServerConfigs: () =>
resolveUserMcpServerConfigs({
userId: input.senderUserId,
Expand Down
15 changes: 12 additions & 3 deletions apps/api/src/handlers/slack/events/fast-agent-reaction.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import {
import {
buildFastAgentArtifactCreator,
findFastAgentSessionForProviderMessage,
handOffFastAgentInterruptedTurn,
persistFastAgentInlineHumanTurn,
recordFastAgentConversationMessageBestEffort,
resolveUserMcpServerConfigs,
Expand Down Expand Up @@ -93,7 +94,9 @@ async function processFastAgentReaction(params: {
// claim before it runs, so an interruption hands it to the queue, which
// resumes it with the same reaction input instead of asking the user to
// react again.
const durableTurn = await persistFastAgentInlineHumanTurn({
const reactionAdmission: Parameters<
typeof persistFastAgentInlineHumanTurn
>[0] = {
parent: { sessionId: session.id, conversation },
event: {
type: 'human_follow_up',
Expand All @@ -107,7 +110,10 @@ async function processFastAgentReaction(params: {
: {}),
input: { type: 'reaction', externalInput: reactionInput },
},
}).catch((error) => {
};
const durableTurn = await persistFastAgentInlineHumanTurn(
reactionAdmission,
).catch((error) => {
console.error(
`[SlackWebhook] Failed to persist Fast reaction turn admission: ${error instanceof Error ? error.message : String(error)}`,
);
Expand Down Expand Up @@ -159,7 +165,10 @@ async function processFastAgentReaction(params: {
retryAt,
),
}
: {}),
: {
requestLateDurableAdmission: () =>
handOffFastAgentInterruptedTurn(reactionAdmission),
}),
createArtifact: buildFastAgentArtifactCreator(session.id),
activity: createFastAgentSlackSessionActivity({
slack: context.slack,
Expand Down
9 changes: 8 additions & 1 deletion apps/api/src/handlers/slack/events/fast-agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import { appendAttachmentTextsToPromptText } from '@roomote/cloud-agents';
import {
admitFastAgentHumanFollowUp,
createFastAgentConversationArtifact,
handOffFastAgentInterruptedTurn,
persistFastAgentInlineHumanTurn,
wakeFastAgentParentEventAt,
wakeFastAgentParentEventNow,
Expand Down Expand Up @@ -357,7 +358,13 @@ export async function processFastAgentMessage(params: {
retryAt,
),
}
: {}),
: {
requestLateDurableAdmission: () =>
handOffFastAgentInterruptedTurn({
parent: { sessionId: session.id, conversation },
event: humanFollowUpEvent,
}),
}),
activity: createFastAgentSlackSessionActivity({
slack,
workspaceId: teamId,
Expand Down
2 changes: 1 addition & 1 deletion apps/docs/fast-sessions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ for a separate turn under that participant's identity. This keeps shared
conversations ordered without losing instructions that arrive during active
work.

Fast turns also survive API or worker interruptions. Roomote durably admits the turn before acknowledging it, records every action the turn takes as it happens, and resumes unfinished work on another process. The resumed run receives the transcript of its earlier attempt, up to the point it was cut off, so it continues from there instead of repeating actions or asking you to send the request again. If the cut lands on the final reply itself, the resumed run finishes from the transcript: a reply that was recorded is not posted again, and a reply the process died while posting goes out once more without another model request. This covers turns started by a typed message, by an emoji reaction, and by platform events such as a setup kickoff. Provider-side retry waits are honored the same way: short waits keep the turn in place, and longer waits park it durably until the scheduled time.
Fast turns also survive API or worker interruptions. Roomote durably admits the turn before acknowledging it, records every action the turn takes as it happens, and resumes unfinished work on another process. The resumed run receives the transcript of its earlier attempt, up to the point it was cut off, so it continues from there instead of repeating actions or asking you to send the request again. If the cut lands on the final reply itself, the resumed run finishes from the transcript: a reply that was recorded is not posted again, and a reply the process died while posting goes out once more without another model request. This covers turns started by a typed message, by an emoji reaction, and by platform events such as a setup kickoff. Provider-side retry waits are honored the same way: short waits keep the turn in place, and longer waits park it durably until the scheduled time. A turn that could not be recorded durably before it started is recorded at the moment a restart interrupts it and resumed the same way; only if that also fails does Roomote ask you to send the request again.

### Message Suggestions

Expand Down
31 changes: 23 additions & 8 deletions apps/web/src/trpc/commands/fast-sessions/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import {
buildFastAgentArtifactCreator,
buildFastAgentSurfaceReplyDelivery,
createFastAgentSessionArtifact,
handOffFastAgentInterruptedTurn,
persistFastAgentInlineHumanTurn,
resolveUserMcpServerConfigs,
wakeFastAgentParentEventAt,
Expand Down Expand Up @@ -138,8 +139,10 @@ type WebFastAgentTurnInput = {
/** Deterministic turn ID override. Canonical event IDs derive from it, so a
* fixed value lets a turn be claimed idempotently across retries. */
currentMessageId?: string;
/** Fast conversation id for durable admission of a human turn. Platform
* turns (kickoffs, artifact builds) omit it and stay non-replayable. */
/** Fast conversation id for durable admission. Every turn passes it;
* only setup-session turns (adapter extensions, setup snapshot) are still
* process-bound, and their scheduler re-runs them when they never
* completed. */
durableSessionId?: string;
/** Skip the turn if this exact canonical event row already exists when the
* turn acquires its lock. This is the atomic claim for the setup kickoff:
Expand Down Expand Up @@ -258,9 +261,11 @@ async function runWebFastAgentTurn({
// events ride the same row with their framing recorded; the ones that
// need adapter extensions or a setup snapshot cannot be rebuilt by the
// queue and stay process-bound.
const durableTurn =
const durableAdmissionRequest:
| Parameters<typeof persistFastAgentInlineHumanTurn>[0]
| null =
durableSessionId && !adapterExtensions && !setupSnapshot
? await persistFastAgentInlineHumanTurn({
? {
parent: { sessionId: durableSessionId, conversation },
event: {
type: 'human_follow_up',
Expand All @@ -281,13 +286,18 @@ async function runWebFastAgentTurn({
: {}),
...(setupSession ? { setupSession: true } : {}),
},
}).catch((error) => {
}
: null;
const durableTurn = durableAdmissionRequest
? await persistFastAgentInlineHumanTurn(durableAdmissionRequest).catch(
(error) => {
console.error(
`[Fast Web] Failed to persist turn admission: ${formatErrorForLog(error)}`,
);
return null;
})
: null;
},
)
: null;
if (durableTurn && durableSessionId) {
release.durableRowId = durableTurn.id;
release.durableResume = () =>
Expand Down Expand Up @@ -345,7 +355,12 @@ async function runWebFastAgentTurn({
retryAt,
),
}
: {}),
: durableAdmissionRequest
? {
requestLateDurableAdmission: () =>
handOffFastAgentInterruptedTurn(durableAdmissionRequest),
}
: {}),
...delivery.adapter,
...adapterExtensions,
},
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,14 @@ export type FastAgentMessageUpsertResult = {
initialHumanTurn: boolean;
};

/**
* Closeout for a turn a restart cut off with no way to resume it: the turn
* had no durable row (its admission write failed), so nothing will re-run
* it and the user has to ask again. Every admitted turn resumes instead.
*/
export const RESTARTED_ACTIVE_TURN_MESSAGE =
'Roomote restarted while working on this request. Please send it again.';

export const INTERRUPTED_INFERENCE_RETRY_MESSAGE =
'The inference retry was interrupted before it completed. Please send the request again.';

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,14 @@ export type FastAgentTurnAdapter = {
* wait for the next sweep. Best effort.
*/
requestDurableResume?: () => Promise<void>;
/**
* Called when a shutdown interrupts a turn that has no durable row (its
* admission write failed before it ran). Persists the row now and hands
* it to the queue so the turn resumes on the next process instead of
* asking the user to send it again. Resolves true when the hand-off
* landed; false means nothing will re-run the turn.
*/
requestLateDurableAdmission?: () => Promise<boolean>;
/**
* Called when a turn has parked itself for a durable inference
* retry; schedules the queue wakeup for `retryAt` so the retry does not
Expand Down
Loading
Loading