Skip to content

fix: Keep AsyncPredictor serializers in sync with wrapped predictor - #6335

Open
lucasjia-aws wants to merge 1 commit into
aws:masterfrom
lucasjia-aws:fix/3100-async-predictor-serde
Open

lucasjia-aws wants to merge 1 commit into
aws:masterfrom
lucasjia-aws:fix/3100-async-predictor-serde

Conversation

@lucasjia-aws

Copy link
Copy Markdown
Collaborator

Issue

Fixes #3100 (V3 side). V2 counterpart: #6334

Description

sagemaker.serve.predictor_async.AsyncPredictor carries the same defect as the V2 class: the serializer and deserializer are copied from the wrapped predictor in __init__, the upload uses the outer copy, and the Accept header and S3 result decoding use the wrapped predictor's deserializer. Overriding AsyncPredictor.deserializer therefore has no effect on the response.

Note that in V3 the recommended async path is ModelBuilder.deploy(...), which returns a sagemaker.core.resources.Endpoint, invoked via Endpoint.invoke_async(content_type=..., accept=...). That path passes content types per call and is not affected by this issue. This change keeps the ported AsyncPredictor consistent with the V2 fix.

Changes

  • Replace the copied attributes in sagemaker-serve/src/sagemaker/serve/predictor_async.py with serializer / deserializer properties that read from and write to the wrapped predictor.
  • Remove the attribute copies from __init__.

Testing

  • Added TestAsyncPredictorSerializerOverrides in sagemaker-serve/tests/unit/test_predictor_async.py covering the deserializer override (Accept header and result decoding), the serializer override (S3 upload body and ContentType), and values set on the wrapped predictor.
  • The new tests fail without the fix and pass with it.
  • sagemaker-serve/tests/unit/test_predictor_async.py and tests/unit/async_inference/: 28 passed.
  • black, flake8, pydocstyle, and pylint are clean on the changed files.

Port the V2 fix for aws#3100 to sagemaker-serve. AsyncPredictor copied
the wrapped predictor's serializer and deserializer at construction,
so overriding AsyncPredictor.deserializer did not affect the Accept
header or result decoding, which read from the wrapped predictor.

Make serializer and deserializer properties that delegate to the
wrapped predictor.

Fixes aws#3100
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Claude finished @lucasjia-aws's task in 59s —— View job


Review: Keep AsyncPredictor serializers in sync with wrapped predictor

  • Read PR diff and changed files
  • Analyze correctness of the property-based fix
  • Review test coverage
  • Post review

I reviewed the diff against predictor_async.py and its callers. This is a clean, correct fix. Summary below.

Correctness ✅

The bug was that __init__ copied serializer/deserializer onto the AsyncPredictor, while the code paths that actually matter read from the wrapped predictor:

  • _upload_data_to_s3 (predictor_async.py:205-207) uses self.serializer for the S3 body + ContentType.
  • _create_request_args (:226) sets the Accept header from self.predictor.accept (derived from predictor.deserializer).
  • _handle_response (:272, :337, :341) decodes the S3 result via self.predictor.

So overriding AsyncPredictor.deserializer had no effect on the response, and the outer serializer copy could drift from the predictor. Replacing the copied attributes with properties that read/write self.predictor.serializer/.deserializer makes all three paths honor the configured value. This lines up the V3 class with the V2 fix (#6334).

Notes (non-blocking)

  • Intentional behavior change: the setter now mutates the wrapped predictor's serializer/deserializer, so an app sharing one predictor instance between an AsyncPredictor and other use would see that predictor's (de)serializer change too. This is the intended "keep in sync" semantics and matches V2, so no action needed — just worth being explicit about in case anyone relied on the old (buggy) independent-copy behavior.
  • The property assumes self.predictor always exposes serializer/deserializer. That held before (the old __init__ read the same attributes), so no new constraint is introduced.

Tests ✅

TestAsyncPredictorSerializerOverrides covers the three relevant behaviors well:

  • deserializer override → Accept header (application/json) and decoded result,
  • serializer override → S3 Body and ContentType,
  • reflecting values set directly on the wrapped predictor.

Using a _StubPredictor (rather than the real Predictor) is reasonable and keeps the test focused. As the description notes, these fail without the fix and pass with it.

Nothing else stands out — no security, performance, or backward-incompatibility concerns beyond the intentional sync semantics above. LGTM.

Note: the inline-comment tool wasn't available in this run, so this feedback is posted as a single review comment rather than line-anchored comments.
· fix/3100-async-predictor-serde

This branch was successfully deployed

1 active deployment
auto-approve — 538d95cd Deployed Sep 25, 2026 by lucasjia-aws via wait-for-approval #443
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Setting AsyncPredictor.deserializer doesn't work

1 participant