Skip to content

fix(submitqueue): repair terminal request status on retry - #1039

Draft
sbalabanov wants to merge 2 commits into
wt/sqc010from
wt/sqc009
Draft

sbalabanov wants to merge 2 commits into
wt/sqc010from
wt/sqc009

Conversation

@sbalabanov

@sbalabanov sbalabanov commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

TL;DR: Rejected or cancelled requests reach their final displayed status even when publishing the terminal log initially fails.

Summary

A rejected or cancelled request can be finished internally but still look unfinished if its final status update fails to send. A retry now resends the final status without repeating validation, cancellation, or a state change. Repairs explicitly say that the original terminal details were not retained; the first attempt can still include its available reason.

Repro

  1. Submit a request that is rejected during validation, or cancel a request before it joins a batch.
  2. Let the final state write succeed, fail the final status publication, then retry the same queued message after publication works again.
  3. Before this change, the retry skips the update and the request still looks unfinished. With this change, it resends the final status and reports that the original details were not retained, without changing the request state or version again.

Test Plan

  • Existing validation, conflict-check, and cancellation tests fail the first final-status publication and retry the message. They check the final status and request version, the explicit unavailable-details diagnostic, and no repeated state write or validator/provider work.
  • Shared termination tests cover same-state replay and leaving a different terminal winner untouched. First-attempt error and cancellation details remain covered.

Six focused core/controller test targets and four affected libraries passed locally; formatting, lint, Gazelle, and module checks passed. No storage integration run is needed: this PR no longer changes storage or schema.

Stack

Based on #1037 (wt/sqc010); this is the current branch base, not a required runtime dependency.

Rollout

No schema migration is required. Replace old consumers so matching terminal retries reach status repair.

Revert Plan

Revert this PR to restore the earlier guard behavior; terminal retries may again leave the displayed status unfinished.

Issue Links

None; repository audit work, no external ticket.

Summary:
Persist the terminal timestamp, error, and metadata in the winning request snapshot. Retry matching terminal outcomes without another transition or provider call, and prevent incomplete legacy repairs from erasing known diagnostics.

Reproduction:
1. Reject or cancel a request while the terminal state write succeeds and its log publication fails.
2. Redeliver the original validation, conflict-signal, or cancellation message.
3. Confirm the retry publishes the original terminal facts without another terminal write; a different terminal winner remains unchanged.

Test Plan:
The existing Process regression tests exercise failed publication followed by successful replay and compare the original diagnostics, timestamp, and request version. TestTerminateRequest_ReplaysWinningFactsAfterPublishFailure also covers changed retry diagnostics and duplicate delivery.

TestRequestTerminalDetailsMigration demonstrates the explicit additive migration, readable legacy NULL details, preserved new diagnostics, and rejection of malformed non-NULL details against MySQL. Materializer tests verify history/mapping repair without overwriting known diagnostics.

Verified with ./tool/bazel test //submitqueue/... --test_output=errors (59 targets), the MySQL storage integration target, focused coverage, and repository formatting/lint/module/build/mock checks.

Revert Plan:
Revert the code while retaining the additive terminal_details column and its persisted facts. Retain failed deliveries for replay; reverting terminal Process repair reopens the missing-log failure.
Summary:
Keep terminal-status repair on validation, conflict-signal, and cancellation retries without retaining the original log content. A replay explicitly reports that original terminal details were not retained, preserves the current request version, and uses the same publication identity.

Remove the terminal-details entity fields, database schema and migration, storage serialization, gateway legacy protection, and their retention-only tests. Existing termination still publishes the available first-attempt diagnostic; matching-state retries do not repeat the state transition or validation.

Test Plan:
Six focused request-core and controller targets pass, including the existing three failed-publication/retry regressions. These check status/version stability, explicit unavailable diagnostics, and no repeated state write/provider/validator work. Four affected libraries build; formatting, lint, Gazelle, and module checks pass.

Revert Plan:
Revert this follow-up commit to restore the original larger draft; reverting the whole PR reopens terminal status repair failures.
@sbalabanov sbalabanov changed the title fix(submitqueue): replay persisted terminal request logs fix(submitqueue): repair terminal request status on retry Nov 10, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants