fix: Keep AsyncPredictor serializers in sync with wrapped predictor - #6334
Conversation
AsyncPredictor copied the wrapped Predictor's serializer and deserializer onto itself at construction time. Requests were then serialized with the outer copy, while the Accept header and response decoding used the inner predictor's deserializer. Overriding AsyncPredictor.deserializer after creation therefore had no effect. Make serializer and deserializer properties that read from and write to the wrapped Predictor, so a single source of truth drives the upload, the Accept header, and result decoding. Fixes aws#3100
mohamedzeidan2021
left a comment
There was a problem hiding this comment.
Approving. Reproduced the issue exactly: on master the serializer override works for the S3 upload but the deserializer is silently ignored — Accept: */* instead of application/json, and the result comes back as raw bytes. Clean with the fix. The property delegation is what the issue author recommended, and it's the root cause rather than a mask.
Checked every writer: model.py, multidatamodel.py and pipeline.py all set serializers on the inner predictor before wrapping, so existing behaviour is unchanged; jumpstart/factory/model.py hard-raises for non-base-Predictor, so that mutation path never sees an AsyncPredictor. No subclasses anywhere, deepcopy survives, and the documented workaround degrades to a harmless redundant double-write. 3 new tests fail on reverted source; 23 passed in the async suite, 240 across model/predictor/multidatamodel/pipeline.
Best integ evidence in this batch: tests/integ/test_async_inference.py::test_async_walkthrough PASSED against a live async endpoint in this run. The job reads red only because the OIDC session expired at 99% completion, with zero test failures reported.
Non-blocking follow-up: _create_request_args does ", ".join(self.predictor.accept), but Predictor.accept's setter is annotated val: str and stores a plain string — so predictor.accept = "text/csv" renders the header "t, e, x, t, /, c, s, v". Pre-existing and identical on master, so not a regression here, but it's the same "content-type lever silently doesn't work in async" family as #3100 and worth its own issue.
Note this overlaps files with #6337 — whichever lands second will need a rebase and re-run.
Issue
Fixes #3100. V3 counterpart: #6335
Description
AsyncPredictorexposesserializeranddeserializerattributes, but setting them after the predictor is created only partially works. The request payload is serialized withAsyncPredictor.serializer, while theAcceptheader (self.predictor.accept) and the decoding of the S3 output (self.predictor._handle_response) use the wrappedPredictor's deserializer. As a result,async_predictor.deserializer = JSONDeserializer()is silently ignored and the default deserializer andAcceptvalue are still used. The reverse also holds: a serializer set on the wrapped predictor is not used for the upload.Root cause
AsyncPredictor.__init__copiespredictor.serializerandpredictor.deserializeronto the async predictor, creating two independent copies. Different code paths then read from different copies.Changes
src/sagemaker/predictor_async.pywithserializer/deserializerproperties that read from and write to the wrappedPredictor.__init__.Testing
tests/unit/test_predictor_async.pycovering: a deserializer override drives both theAcceptheader and result decoding; a serializer override drives the S3 upload body andContentType; values set on the wrapped predictor are reflected by the async predictor.tests/unit/test_predictor_async.py,tests/unit/sagemaker/async_inference/,tests/unit/test_expected_bucket_owner.py,tests/unit/sagemaker/model/test_deploy.py: 103 passed.