Skip to content

Recover pipeline failures at durable boundaries - #187

Merged
Stephen Belanger (Qard) merged 4 commits into
mainfrom
pipeline/06-recovery-transitions
Oct 19, 2026
Merged

Stephen Belanger (Qard) merged 4 commits into
mainfrom
pipeline/06-recovery-transitions

Conversation

@Qard

@Qard Stephen Belanger (Qard) commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

A failed route enters recovery at its earliest unresolved SpanOp. Persisted typed conditions control retries: plugin bytes/chain changes, fresh credential resolution, relevant configuration changes, or capped transport/storage delays. Restart preserves these conditions; routing retries do not replay translation.

Normal delivery and recovery share flush/completion handling. Rejected batches, accepted batches awaiting ledger completion, and pending head completions retain distinct retry points. Source publication retains built chunks through append/commit failures. Status and doctor expose the stage, unresolved position, reason, and retry criterion.

Preserves #184's journaled birth provenance for every owned span. Main's append-only ledger retains pending acknowledgements until a successful write; lock, refresh, and append failures after sink acceptance are all classified for completion-only retry. Production panic assumptions are replaced with explicit handling, including recovery-evidence encoding failures.

New regressions cover all three ledger failure points, retained pending acknowledgements, successful repair, and route-head recovery without resubmitting accepted work. Existing tests cover rejected batches, credentials, plugin-chain edits, route isolation, publication failures, and restart recovery.

Based on main at f7b7683, including merged #184. Lifecycle consolidation and InputPort isolation remain in later layers. Inherits the landed checked-delivery SDK pin from #174.

Validation at 5aee065: all 533 tests pass in the locked all-feature Rust suite without exclusions or environment overrides; five live-agent tests remain normally ignored. Strict all-target/all-feature Clippy, formatting, and diff checks pass.

Rebased at 8b9e7a8: all four commits retain identical patches (git range-diff); diff checks pass. Fresh CI passes on Linux, macOS, and Windows at 8b9e7a8 (run 39305768223, attempt 3).

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T19:26:33.577720Z 066c394 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 066c39457e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

Comment thread bt-daemon/src/pipeline/recovery.rs
Comment thread bt-daemon/src/pipeline/consumer.rs
Base automatically changed from pipeline/05-journal-boundary to main October 19, 2026 18:44
@Qard
Stephen Belanger (Qard) force-pushed the pipeline/06-recovery-transitions branch from 5aee065 to 8b9e7a8 Compare October 19, 2026 18:49
@Qard
Stephen Belanger (Qard) merged commit c6affdc into main Oct 19, 2026
40 of 56 checks passed
@Qard
Stephen Belanger (Qard) deleted the pipeline/06-recovery-transitions branch October 19, 2026 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants