diff --git a/docs/migration-1.x-to-2.x.md b/docs/migration-1.x-to-2.x.md new file mode 100644 index 00000000..e150c3b3 --- /dev/null +++ b/docs/migration-1.x-to-2.x.md @@ -0,0 +1,152 @@ +# Migrating from 1.x to 2.x + +`2.x` is a breaking major release. Every change is a bug fix or brings Python to +parity with the JavaScript and Java SDKs. The two changes most likely to touch +your code are the typed, per-operation **error hierarchy** and the +**serialize/deserialize round trip on the first run**. + +There is no compatibility shim: removed names (for example `CallableRuntimeError`) +are gone with no alias. If you are not ready to migrate, stay on `1.x`. + +## What Changed and What to Do + +| Change | What you must do | +| --- | --- | +| `CallableRuntimeError`, `UserlandError`, `CallableRuntimeErrorSerializableDetails` removed; typed per-operation errors added | Catch `StepError`, `InvokeError`, `ChildContextError`, or `WaitForConditionError` (or the base `DurableOperationError`) instead of `CallableRuntimeError`. | +| `CallbackError` moved out of the termination tree; graded subtypes added | Remove any `termination_reason == TerminationReason.CALLBACK_ERROR` check (the enum member is gone). Optionally catch `CallbackTimeoutError` / `CallbackExternalError` / `CallbackSubmitterError`. | +| `BatchResult.throw_if_error()` now raises a typed error | Catch `ChildContextError` instead of `CallableRuntimeError`. | +| First-run serialize/deserialize round trip for `step`, child contexts, `map`/`parallel`, and `wait_for_condition` | Make custom `SerDes` round-trip safe: `deserialize(serialize(x)) == x`. Ensure `wait_for_condition` `initial_state` is serializable by the configured serdes. For a transient serdes failure, raise the new `RetryableSerDesError` (retries) instead of `SerDesError` (permanent). | +| `InvokeConfig.timeout` and `InvokeConfig.timeout_seconds` removed | Remove them. Enforce any timeout inside the invoked function or as a separate timer. | +| Removed `ItemBatcher`, `ItemsPerBatchUnit`, `BatchedInput`, `TerminationMode`, `StepFuture`, `MapConfig.item_batcher`, `ChildConfig.item_serdes` | Remove all uses. Replace `ChildConfig.item_serdes` with `ChildConfig.serdes`. | +| `MapConfig` / `ParallelConfig` / `CompletionConfig` now validate at construction | Wrap construction in `try/except ValidationError` if you build configs from external input. | +| `CompletionConfig.all_completed()` now actually tolerates all failures | If you hand-built the old all-`None` config, use the factory instead. | +| `WaitDecision` removed; `WaitStrategyConfig.timeout` / `timeout_seconds` removed | Use `WaitForConditionDecision` (`stop_polling()` / `continue_waiting(delay)`). | +| `wait_for_condition` raises `WaitForConditionError` when it exhausts `max_attempts` | Catch `WaitForConditionError` instead of inspecting the returned state. | + +Find affected code before upgrading: + +```bash +rg -n "CallableRuntimeError|UserlandError|CallableRuntimeErrorSerializableDetails" . +rg -n "CallbackError|CALLBACK_ERROR" . +rg -n "InvokeConfig\(|\.timeout_seconds" . +rg -n "WaitDecision|WaitStrategyConfig\(|item_batcher|ItemBatcher|ItemsPerBatchUnit" . +rg -n "TerminationMode|BatchedInput|StepFuture|ChildConfig\(" . +``` + +## Error Handling (the biggest change) + +In `1.x` nearly every user-land failure surfaced as one `CallableRuntimeError`, +so a failed step was indistinguishable from a failed invoke or child branch. `2.x` +raises a specific type per operation, all under a new base `DurableOperationError`, +and preserves the original error as `__cause__` (on replay, `__cause__` is +reconstructed from the checkpointed wire fields `error_type`/`message`/`data`/`stack_trace`). + +```python +# 1.x +from aws_durable_execution_sdk_python.exceptions import CallableRuntimeError +try: + result = context.step(charge_card, name="charge") +except CallableRuntimeError as e: + context.logger.error("something failed: %s", e.message) + +# 2.x +from aws_durable_execution_sdk_python import StepError, DurableOperationError +try: + result = context.step(charge_card, name="charge") +except StepError as e: # or `except DurableOperationError` to catch any operation + context.logger.error("charge step failed: %s", e.message) +``` + +New types, all exported from the package root: `DurableOperationError` (base), +`StepError`, `InvokeError`, `ChildContextError`, `WaitForConditionError`, +`CallbackError` (+ `CallbackExternalError`, `CallbackTimeoutError`, +`CallbackSubmitterError`), plus `SerDesError` (now exported) and +`RetryableSerDesError`. `SerDesError` stays a direct child of +`DurableExecutionsError`; `RetryableSerDesError` is a retryable `InvocationError`. + +### Callbacks + +`context.wait_for_callback(...)` returns the payload directly and raises the +callback error from the call itself (there is no `callback.result()`): + +```python +from aws_durable_execution_sdk_python import ( + CallbackError, CallbackTimeoutError, CallbackSubmitterError, +) +try: + payload = context.wait_for_callback(submit_approval, name="approval") +except CallbackTimeoutError: + ... # timeout / heartbeat expiry +except CallbackSubmitterError: + ... # the submitter step failed +except CallbackError as e: # external + internal + context.logger.error("callback failed: %s", e.message) +``` + +### map / parallel + +```python +result = context.map(items, process_item) +try: + result.throw_if_error() # raises ChildContextError for the first failure +except ChildContextError: + for err in result.get_errors(): # every failed item's ErrorObject + context.logger.error("%s: %s", err.type, err.message) +``` + +## Serialize/Deserialize Round Trip + +`1.x` returned the raw in-memory result on the first run but the deserialized +result on replay, so a non-identity custom `SerDes` produced different values. +`2.x` round-trips (`serialize` then `deserialize`) on the first run for `step`, +child contexts, `map`/`parallel`, and `wait_for_condition` (which also feeds the +deserialized state to the wait strategy). No API change, but a `SerDes` that is +not round-trip safe now surfaces the discrepancy (and any serialization bug) on +the first run. Fix it so `deserialize(serialize(x)) == x`. Async operations +(`invoke`, `wait_for_callback`, `wait`) are unaffected. + +`wait_for_condition` also round-trips `initial_state` through the serdes before +the first check, so `initial_state` must now be serializable by the configured +serdes. + +## New in 2.x: Custom Completion Predicate (Optional) + +`2.x` adds a `should_complete` predicate to `CompletionConfig`, giving `map` and +`parallel` full control over when a batch completes early. This is a new feature, +not a breaking change - no action is required unless you adopt it. + +```python +from aws_durable_execution_sdk_python import complete_batch, continue_batch + +config = CompletionConfig( + should_complete=lambda status: ( + complete_batch() if status.success_count >= 2 else continue_batch() + ) +) +``` + +The predicate receives a `CompletionStatus` snapshot (counts plus per-item +statuses) and returns a `CompletionDecision` - `continue_batch()` or +`complete_batch(outcome)`. The outcome reports `CUSTOM_COMPLETION_SUCCEEDED` or +`CUSTOM_COMPLETION_FAILED`; a failed custom completion surfaces through +`throw_if_error()` as a `ChildContextError`, so there is still no separate +batch-completion error type to catch. Notes: + +- It cannot be combined with `min_successful` or the `tolerated_failure_*` + fields; doing so raises `ValidationError` at construction. +- The predicate must be deterministic and side-effect-free. Replay uses the + checkpointed decision and never re-invokes it. +- New exports: `complete_batch`, `continue_batch`, `CompletionStatus`, + `CompletionDecision`, `CompletionOutcome`, `CompletionItemStatus`, + `BatchItemStatus`. + +## Recommended Validation After Upgrading + +1. Build and run your test suite against `2.x`, and grep for the removed names above. +2. Trigger a failure in a `step`, an `invoke`, and a `map`/`parallel` branch; + confirm you catch `StepError`, `InvokeError`, and `ChildContextError`. +3. Exercise a `wait_for_callback` timeout and a submitter-step failure + (`CallbackTimeoutError`, `CallbackSubmitterError`). +4. Exercise a `wait_for_condition` that exhausts its attempts (`WaitForConditionError`). +5. If you use a custom `SerDes`, run a workflow that checkpoints both a result and + an error payload and confirm first-run output equals replay output. diff --git a/packages/aws-durable-execution-sdk-python/src/aws_durable_execution_sdk_python/waits.py b/packages/aws-durable-execution-sdk-python/src/aws_durable_execution_sdk_python/waits.py index 3753a1be..cfb9a99f 100644 --- a/packages/aws-durable-execution-sdk-python/src/aws_durable_execution_sdk_python/waits.py +++ b/packages/aws-durable-execution-sdk-python/src/aws_durable_execution_sdk_python/waits.py @@ -99,7 +99,17 @@ def wait_strategy(result: T, attempts_made: int) -> WaitForConditionDecision: @dataclass(frozen=True) class WaitForConditionConfig(Generic[T]): - """Configuration for wait_for_condition.""" + """Configuration for wait_for_condition. + + Attributes: + wait_strategy: Called after each poll with (state, attempts_made) and + returns a WaitForConditionDecision (continue_waiting or stop_polling). + initial_state: State passed to the first poll. It is round-tripped + through serdes (serialize then deserialize) before the first check, + so it must be serializable by the configured serdes. + serdes: SerDes used to serialize and deserialize the polled state at + each checkpoint. Defaults to the SDK's default JSON serdes when None. + """ wait_strategy: Callable[[T, int], WaitForConditionDecision] initial_state: T