fix(traces): make trace backfills bounded and resumable

[TheodoreSpeaks]

Summary

• Make the trace backfill fail fast, scan bounded chronological pages, and resume from checkpoints that preserve timestamp precision and the scan cutoff.
• Use the current workflow owner for file metadata while preserving the original execution user in the stored payload.
• Support up to 512 workers with a shared payload byte budget, graceful shutdown, and throughput, RSS, and stage timing logs.
• Route execution-log reads, writes, and reference transactions through the execution pool, matching the completion path.

Type of Change

• Bug fix

Testing

• 112 targeted tests pass across the backfill, trace store, and payload store.
• All 46 repository audits, block registry checks, docs manifest check, changed-file Biome checks, and git diff --check pass.
• Full repository bun run lint:check passes. All GitHub CI checks pass, including Lint and Test, the app build, both PostgreSQL integration jobs, and all test shards.
• Built and tested the companion image against local PostgreSQL and S3-compatible fixtures: metadata ownership, original execution identity, durable references, and idempotent reruns pass.
• Companion image and task template: simstudioai/infra#375. No database migration or backfill execution is triggered by this PR.

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Sep 18, 2026 7:20pm UTC

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge; no new actionable defects or outstanding previous findings remain.

Summary

This PR makes trace backfills bounded, resumable, and fail-fast.

• Adds chronological keyset pagination with precision-preserving checkpoint tokens and a fixed scan cutoff.
• Introduces bounded worker concurrency, a shared payload-byte budget, graceful shutdown, and progress/timing metrics.
• Uses workflow ownership for durable-file metadata while preserving the execution identity in stored payloads.
• Routes execution-log and large-value operations through the execution database pool.
• Improves strict storage error propagation and redacted fallback logging.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Parse options or checkpoint] --> B[Preflight database schema and permissions]
    B --> C[Scan bounded metadata page by startedAt and id]
    C --> D[Read eligible payload sizes]
    D --> E[Reserve shared byte budget]
    E --> F[Workers read payloads through execution pool]
    F --> G[Strip costs and externalize trace data]
    G --> H[Transactionally update execution log and references]
    H --> I{Entire page completed?}
    I -->|Yes| J[Emit resumable checkpoint]
    I -->|No, failure or shutdown| K[Keep previous checkpoint]
    J --> C

Loading

_{Reviews (2) · Last reviewed commit: "fix(traces): use execution pool througho..."}

[cubic-dev-ai]

All reported issues were addressed across 6 files

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[TheodoreSpeaks]

@greptile

[TheodoreSpeaks]

@cubic-dev-ai review this PR

[cubic-dev-ai]

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

[cubic-dev-ai]

No issues found across 6 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

_{Re-trigger cubic}