Repository navigation
Fix skipped mutable side effect replay recovery - #3164
Draft
Quinn-With-Two-Ns wants to merge 1 commit into
Draft
Quinn-With-Two-Ns wants to merge 1 commit into
Quinn-With-Two-Ns wants to merge 1 commit into
Conversation
Resolve pending mutable side effect invocations at the original workflow-task boundary without changing legacy replay timing. Gate recovery behind a first-release-disabled SDK flag and register opt-in use at the mutableSideEffect call site. Add cache-eviction, callback-time, and nondeterminism detection replay coverage for issue #3163.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed?
MUTABLE_SIDE_EFFECT_REPLAY_RECOVERY, disabled by default for its first release. Register opt-in usage at themutableSideEffectcall site and honor recorded flags during replay.Skip-count calculation and marker matching are unchanged.
Why?
Refs #3163.
A skipped mutable side effect has no corresponding marker event. During full replay, its internal placeholder can remain pending when there is no later command event to trigger cleanup, and an empty
RecordMarkercommand can reach the server.Unconditionally notifying the pending invocation at the next task start changes the workflow time observed by its continuation and can break replay. The added timing regression covers that case; flagged cleanup instead runs at the completed task's boundary, before the next task advances time or delivers signals.
Breaking changes?
None to public APIs. The new SDK flag remains disabled by default (
Enabled: pending) for rollout compatibility; tests opt in explicitly. Unflagged histories retain their existing replay behavior.Server PR
None required.
Validation
./gradlew --offline spotlessApplyadf0fc8) on the test and real servers. Baseline replay passed first, followed by 40 patched replay cases comparing exact callback values, order, and workflow times with opt-in off/on. The bulk corpus and generator are intentionally not included in this PR.