fix(runtime-sdk): retain the ASGI finalizer proxy until completion

[audreyfeldroy]

A streaming ASGI response can return HTTP 200 while its waitUntil task rejects with a destroyed borrowed-proxy error. The finalizer introduced in #229 currently passes a Python task directly to JavaScript:

wait_until(run_in_background(finalize_request()))

waitUntil retains the task after returning to Python. The implicit proxy is borrowed and expires at that return boundary, before JavaScript finishes assimilating the task. This can prevent request finalization, including background work and lifespan shutdown, from completing reliably after the response.

Create an explicitly owned proxy and destroy it when the finalizer task completes, following the WebSocket task-ownership pattern already used in #166. Existing task retention, exception logging, and the finalizer's finally: await shutdown() remain intact.

The new regression runs inside Workerd. It substitutes JavaScript Array.push for wait_until: both retain their argument and return no Promise. It retrieves and awaits that retained task after fetch() returns, rather than checking only the response body. It also verifies the complete streamed body and background work preceding lifespan shutdown. This catches the rejected task that an otherwise successful response can hide.

Related work: #229 introduced this HTTP finalizer; #166 provides the existing ownership pattern. The open worker-lifetime lifespan proposal #239 does not address this proxy lifetime.

Test Plan

• Upstream ASGI Workerd suite, Python 3.13: 22 passed, including snapshot creation and reload, with the fix.
• Same suite with the new test but unmodified main source: 21 passed, 1 failed. The new test fails with pyodide.ffi.JsException reporting the destroyed borrowed proxy when awaiting the retained finalizer.
• Ruff check and format check passed using the CI-pinned Ruff 0.9.1; git diff --check passed.
• The equivalent narrow patch was also verified in a real deployment: the same 12 controlled requests produced the proxy error in all 12 runtime events before the patch and none afterward. Details: https://github.com/writerstead/writerstead.com/pull/322#issuecomment-5592011784

Local harness details: macOS, Node 24.19.0, uv 0.12.3, Workerd 1.20260815.1. The host harness used that existing pinned Workerd binary instead of its unpinned npm i workerd step. Python dependencies were constrained to releases before September 2 for the local test setup.

The wider local compatibility run could not execute Python 3.12 (Pyodide interpreter query failed during setup with ModuleNotFoundError: No module named 'python') or Python 3.14 (the installed Workerd does not recognize python_workers_314). These are unverified configurations, not passing checks. The new regression is included in the existing ASGI suite for the normal CI matrix.

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[audreyfeldroy]

I have read the CLA Document and I hereby sign the CLA

[hoodmane]

I'm confused about this patch because wait_until() should already keep the PyProxy alive until the promise resolves. See here:
https://github.com/cloudflare/workerd/blob/main/src/pyodide/python-entrypoint-helper.ts#L58-L70

[hoodmane]

It definitely fails without this change though.

[hoodmane]

Oh right the test monkeypatches wait_until.

[hoodmane]

Closing as not valid in my opinion.

[hoodmane]

Could you post a reproducer for your problem? Perhaps there's a different problem here to solve. This link is broken though:
https://github.com/writerstead/writerstead.com/pull/322#issuecomment-5592011784

[audreyfeldroy]

Here’s the standalone reproducer you requested, attached below. Thanks for catching the problem with the earlier test.

This reproducer calls the real runtime APIs. With workers-runtime-sdk 1.8.4 and Workerd 1.20260815.1:

• workers.wait_until(task) logs a destroyed borrowed-proxy error, with no background completion marker within two seconds.
• self.ctx.waitUntil(task) completes successfully.
• workers.wait_until(create_proxy(task)) also completes successfully.

All three return HTTP 200, and each case runs in a fresh Worker instance. The ZIP includes pinned dependencies, instructions, a verification script, and captured results.

Could the module-level export be missing the proxy retention provided by patchWaitUntil for execution contexts?

workers-wait-until-reproducer.zip

[hoodmane]

Thanks for the reproducer. So:

• http://localhost:8796/global - Error: This borrowed proxy was automatically destroyed at the end of a function call.
• http://localhost:8796/context - works

That is, this fails:

from workers import wait_until
wait_until(task)

but this works:

self.ctx.waitUntil(task)

I will try to understand what we should do to fix this.

[hoodmane]

Okay I have a fix. Will open a PR later today. Thanks again @audreyfeldroy for the reproducer!

[hoodmane]

Created cloudflare/workerd#7293 with a fix.