Skip to content

[Bug] Retried ActivityHandle.terminate() can fail after the activity is already terminated #2008

Open

Description

@RaphaelFakhri

What are you really trying to do?

Terminate a standalone activity with ActivityHandle.terminate() and rely on the client's built-in RPC retries.

Describe the bug

_ClientImpl.terminate_activity in temporalio/client/_impl.py builds TerminateActivityExecutionRequest without request_id and sends it with retry=True. The sibling calls (cancel_activity, pause_activity, unpause_activity, start_activity) all set request_id=str(uuid.uuid4()), and the Go SDK sets RequestId on this request too.

On the server, an empty request ID is replaced with a new random one for each attempt (validateAndNormalizeRequestID in chasm/lib/activity/validator.go). Activity.Terminate returns success for a repeat only when the request ID matches the stored one, and otherwise returns NotFound: no running activity execution: it closed with status TERMINATED. So if the first attempt terminates the activity but the response is lost (for example a transient UNAVAILABLE), the automatic retry fails and terminate() raises although the activity was terminated.

Minimal Reproduction

from unittest import mock

import temporalio.api.workflowservice.v1
from temporalio.client import Client


async def main() -> None:
    rpc = mock.AsyncMock(
        return_value=temporalio.api.workflowservice.v1.TerminateActivityExecutionResponse()
    )
    service_client = mock.MagicMock()
    service_client.config.identity = "test-identity"
    service_client.workflow_service.terminate_activity_execution = rpc
    client = Client(service_client)
    await client.get_activity_handle("my-activity").terminate(reason="done")
    print(repr(rpc.call_args.args[0].request_id))  # prints '' on main

Environment/Versions

  • SDK: main at 600cbec
  • Python 3.13, Linux x86_64

Additional context

Adding request_id=str(uuid.uuid4()) to the request makes retries of the same call idempotent, matching the other activity operator calls.

Activity

  1. self-assigned this
    on Oct 22, 2026
  2. musi22 commented on Oct 25, 2026

    @musi22

    I'd like to submit a PR for this. Adding
    equest_id=str(uuid.uuid4())\ to \TerminateActivityExecutionRequest\ in _ClientImpl.terminate_activity\ makes it consistent with \cancel_activity, \pause_activity, and \start_activity, ensuring retries are properly deduplicated on the server. I'll include a unit test verifying the request ID is populated on terminate calls.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions