fix(tables): keep table cells working through lost database connections by waleedlatif1 · Pull Request #9705 · simstudioai/sim

waleedlatif1 · 2026-11-05T05:14:56Z

Summary

A reset database socket no longer crashes a table cell. Before, a read outside the task's error handling failed the run, and with no later sweep the cell stayed Queued for good, along with the row's remaining groups.
Retry cell reads in process with withDatabaseReadRetry, bounded by attempts and elapsed time.
Add withIdempotentWriteRetry for pickup and terminal state writes. It retries connection loss only, never lock, deadlock, or capacity errors.
A lost connection leaves a write's outcome unknown, so writeWorkflowGroupStateDurably checks what the last attempt did before repeating it. A landed commit only re-emits its event, so an edit made since survives and a cleared cell stays cleared. Cancel stamps have no guard and are never repeated.
A failure before a terminal state now marks the cell as an error so the user can re-run it. If a Stop or newer run owns the cell, the carrier ends instead of starting the next group. If that write fails too, the run still fails and alerts.
If a read between groups runs out of retries, fail every unclaimed marker on the row while still holding its lock, then fail the run. Other carriers bail on that lock expecting its owner to drain their markers.
A final write that committed despite reporting failure keeps its result instead of being overwritten with an error.
On a retry, a write that follows pickup skips a cell the user cleared or a marker from a newer run.
Stop storing driver text, SQL, and bound values as the visible cell error. Block errors that are already safe keep their message and block name.
No Trigger.dev retry: it replays from the payload's group and would run a billed workflow again for any group already finished.

Type of Change

Bug fix

Testing

cell-write.integration.ts: real schema, updateRow, and SQL guards through a proxy that forwards COMMIT, withholds the reply, and sends a real TCP reset. Covers a landed commit, a later user edit, a cleared cell, a Stop or newer run during backoff, a repeated cancel stamp, and which row markers get failed.
Fails any fault-injection test whose reset never fired.
connection-loss.integration.ts: real resets against postgres.js for reads and transactional writes, including a full pool of reset sockets.
workflow-column-execution-connection-loss.test.ts, read-retry.test.ts, and sibling cell-write, dispatcher, and resume suites.
Reverted each guard separately and watched its named test go red, then restored it.
Ran the two integration suites six times locally on disposable Postgres via test:integration, plus scoped Biome. Type-check, check:audits, and the full test suite run in CI.

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing (new tests pass the test-audit authoring gate)
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2026-11-05T05:14:58Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Nov 5, 2026 9:24am UTC

cubic-dev-ai

All reported issues were addressed across 9 files

_{Turn on auto-fix | Re-trigger cubic}

greptile-apps · 2026-11-05T05:18:59Z

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issue was found.

Summary

Keeps table cells moving through lost database connections without replaying workflows.

Table-cell carriers retry database reads and finish row work after repeated failures.
Cell state writes survive lost database replies without reviving newer work.
Cell error messages hide database details and identify lost connections.

All previous threads were rechecked against the supplied current code; no original finding remains outstanding. None had a supplied Comment number.

waleedlatif1 previously deferred sibling-marker cleanup; the current code now covers those markers.
waleedlatif1 accepted that an unclaimed, status-only error badge could return after a clear. Writes after pickup now skip cleared cells.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Run cell under row lock] --> B[Retry database operation]
  B --> C{Write reply lost?}
  C -->|Yes| D{State already committed?}
  D -->|Yes| E[Send stored outputs]
  D -->|No| F[Retry guarded write]
  A --> G{Cascade throws?}
  G -->|Yes| H[Fail unclaimed markers]
  H --> I[Tell open tables to refetch]
  I --> J[Release lock and fail task]

_{Reviews (8) · Last reviewed commit: "fix(tables): refresh open tables when a ..." · Reviewed by Greptile}

… cell unfinished

waleedlatif1 · 2026-11-05T05:27:25Z

@greptile

waleedlatif1 · 2026-11-05T05:27:27Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-05T05:27:33Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

waleedlatif1 · 2026-11-05T05:33:43Z

@greptile

waleedlatif1 · 2026-11-05T05:33:44Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-05T05:33:52Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 13 files

_{Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic}

…mitted results

waleedlatif1 · 2026-11-05T05:48:46Z

@greptile

waleedlatif1 · 2026-11-05T05:48:47Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-05T05:48:53Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 13 files

_{Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic}

waleedlatif1 · 2026-11-05T06:03:55Z

@greptile

waleedlatif1 · 2026-11-05T06:03:56Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-05T06:04:03Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 13 files

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

waleedlatif1 · 2026-11-05T06:15:20Z

@greptile

waleedlatif1 · 2026-11-05T06:15:21Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-05T06:15:27Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 13 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

Track whether a carrier's own pickup landed instead of inferring it from the write. After a pickup, an unclaimed marker belongs to a newer run, so the write is dropped and the cascade drains it. Before a pickup, an error over a marker leaves the execution id null so a newer carrier can still claim it.

waleedlatif1 · 2026-11-05T09:07:26Z

@greptile

waleedlatif1 · 2026-11-05T09:07:28Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-05T09:07:35Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 13 files

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

A repeat after a lost reply finds nothing left to update, so it cannot name the cells the lost attempt failed. The cleanup now owns its retry and signals a refetch. Correct the retry budget comment: maxElapsedMs only decides whether another attempt starts.

waleedlatif1 · 2026-11-05T09:24:59Z

@greptile

waleedlatif1 · 2026-11-05T09:25:01Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-05T09:25:07Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 13 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

fix(tables): keep table cells working through lost database connections

58caadb

vercel Bot temporarily deployed to Preview November 5, 2026 05:14 Inactive

cubic-dev-ai Bot reviewed Nov 5, 2026

View reviewed changes

Comment thread apps/sim/background/workflow-column-execution.ts Resolved

Comment thread apps/sim/lib/db/read-retry.ts Resolved

Comment thread apps/sim/lib/db/connection-loss.integration.ts Resolved

greptile-apps Bot reviewed Nov 5, 2026

View reviewed changes

fix(tables): use the central test mocks in the cell connection tests

8deed03

vercel Bot temporarily deployed to Preview November 5, 2026 05:20 Inactive

fix(tables): close the remaining ways a lost connection could leave a…

4caa1db

… cell unfinished

vercel Bot temporarily deployed to Preview November 5, 2026 05:25 Inactive

greptile-apps Bot reviewed Nov 5, 2026

View reviewed changes

Comment thread apps/sim/lib/table/cell-write.ts Resolved

fix(tables): keep a cleared failed cell cleared after a lost connection

d6d2e6c

vercel Bot temporarily deployed to Preview November 5, 2026 05:33 Inactive

cubic-dev-ai Bot reviewed Nov 5, 2026

View reviewed changes

Comment thread apps/sim/lib/table/cell-write.integration.ts Resolved

Comment thread apps/sim/background/workflow-column-execution.ts Resolved

Comment thread apps/sim/background/workflow-column-execution.ts Resolved

Comment thread apps/sim/lib/db/connection-loss.integration.ts Resolved

fix(tables): finish every unclaimed cell on a failed row and keep com…

8cedba3

…mitted results

vercel Bot temporarily deployed to Preview November 5, 2026 05:48 Inactive

greptile-apps Bot reviewed Nov 5, 2026

View reviewed changes

Comment thread apps/sim/background/workflow-column-execution.ts Resolved

cubic-dev-ai Bot reviewed Nov 5, 2026

View reviewed changes

Comment thread apps/sim/lib/table/cell-write.ts Resolved

Comment thread apps/sim/background/workflow-column-execution.ts Resolved

Comment thread apps/sim/lib/table/cell-write.integration.ts Resolved

fix(tables): clean up a failed row under its lock and respect newer runs

45b0094

vercel Bot temporarily deployed to Preview November 5, 2026 06:03 Inactive

cubic-dev-ai Bot reviewed Nov 5, 2026

View reviewed changes

Comment thread apps/sim/background/workflow-column-execution.ts Resolved

fix(tables): keep driver text out of wrapped cell errors

1765841

vercel Bot temporarily deployed to Preview November 5, 2026 06:15 Inactive

cubic-dev-ai Bot reviewed Nov 5, 2026

View reviewed changes

vercel Bot temporarily deployed to Preview November 5, 2026 09:07 Inactive

cubic-dev-ai Bot reviewed Nov 5, 2026

View reviewed changes

Comment thread apps/sim/lib/table/cell-write.ts Resolved

Comment thread apps/sim/lib/table/constants.ts Resolved

vercel Bot temporarily deployed to Preview November 5, 2026 09:24 Inactive

cubic-dev-ai Bot reviewed Nov 5, 2026

View reviewed changes

Conversation

waleedlatif1 commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Diagram

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Nov 5, 2026

Uh oh!

waleedlatif1 commented Nov 5, 2026

Uh oh!

cubic-dev-ai Bot commented Nov 5, 2026

Uh oh!

Uh oh!

waleedlatif1 commented Nov 5, 2026

Uh oh!

waleedlatif1 commented Nov 5, 2026

Uh oh!

cubic-dev-ai Bot commented Nov 5, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Nov 5, 2026

Uh oh!

waleedlatif1 commented Nov 5, 2026

Uh oh!

cubic-dev-ai Bot commented Nov 5, 2026

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Nov 5, 2026

Uh oh!

waleedlatif1 commented Nov 5, 2026

Uh oh!

cubic-dev-ai Bot commented Nov 5, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

waleedlatif1 commented Nov 5, 2026

Uh oh!

waleedlatif1 commented Nov 5, 2026

Uh oh!

waleedlatif1 commented Nov 5, 2026 •

edited

Loading

vercel Bot commented Nov 5, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

greptile-apps Bot commented Nov 5, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading