fix: sam local invoke leaks Docker container and temp directory on OOM

[Adityaj0]

Which issue(s) does this change fix?

Fixes #9182

Why is this change necessary?

LambdaRuntime._on_invoke_done() calls _check_exit_state(container) before self._container_manager.stop(container) and self._clean_decompressed_paths():

def _on_invoke_done(self, container):
    if container:
        self._check_exit_state(container)
        self._container_manager.stop(container)
    self._clean_decompressed_paths()

_on_invoke_done is invoked unconditionally from invoke()'s finally block, but has no try/except of its own. When a function is OOM-killed, _check_exit_state() raises ContainerFailureError, and that exception propagates immediately out of _on_invoke_done, skipping both _container_manager.stop(container) and _clean_decompressed_paths().

Every OOM'd invocation of sam local invoke (or start-api/start-lambda) therefore leaves a stopped-but-not-removed Docker container and an unzipped-code temp directory on disk. This accumulates over repeated local-invoke testing, which is exactly the workflow most likely to trigger it (iterating on MemorySize tuning).

How does this change work?

Wrap the exit-state check in a try/finally so the container is always stopped and the temp directory is always cleaned up, regardless of whether _check_exit_state raises:

def _on_invoke_done(self, container):
    try:
        if container:
            self._check_exit_state(container)
    finally:
        if container:
            self._container_manager.stop(container)
        self._clean_decompressed_paths()

The ContainerFailureError (or any other exception from _check_exit_state) still propagates after cleanup runs — this only fixes cleanup, not the error-reporting behavior for OOM.

What tests ran and what were the results?

Added a regression test, test_on_invoke_done_stops_container_and_cleans_paths_even_when_check_exit_state_raises, in tests/unit/local/lambdafn/test_runtime.py. It fails against the pre-fix code (stop() is never called when _check_exit_state raises) and passes after the fix.

Full unit suite (tests/unit) passes: 9387 passed, 25 skipped.
mypy on the changed file: clean.

Checklist

• Add/update tests for this change
• make pr passes locally (unit tests + mypy on changed files; full local make pr target run via pytest tests/unit -q and scoped mypy)
• Write a clear PR title and description

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[aws-sam-tooling-bot]

Code Review Results

Reviewed: e1f4bf6..5950ac4
Files: 2
Comments: 1

[aws-sam-tooling-bot]

[RESOURCE_MANAGEMENT] The two cleanup steps run sequentially in the same finally block, so a failure in the first still skips the second — the same class of leak this PR fixes.

ContainerManager.stop() calls Container.stop() and Container.delete(), and both re-raise docker.errors.APIError unless the message matches the "removal of container ... is already in progress" special case (see samcli/local/docker/container.py). If that raises, self._clean_decompressed_paths() never runs, leaving the unzipped-code temp directory behind. It also replaces the in-flight ContainerFailureError, so an OOM'd invocation would surface an opaque Docker API error instead of "Container invocation failed due to maximum memory usage".

Nesting the cleanup makes both steps independent and preserves the original exception:

try:
   if container:
       self._check_exit_state(container)
finally:
   try:
       if container:
           self._container_manager.stop(container)
   finally:
       self._clean_decompressed_paths()