fix(core): add PyTorch 2.6 graviton (arm64) inference image - #6342
Open
rsareddy0329 wants to merge 1 commit into
Open
rsareddy0329 wants to merge 1 commit into
rsareddy0329 wants to merge 1 commit into
Conversation
The inference_graviton scope in pytorch.json stopped at 2.4.0, so image_uris.retrieve(..., image_scope="inference_graviton") could not resolve a 2.5+/2.6 image even though the regular inference scope already had 2.5.1/2.6.0. AWS publishes the newer graviton PyTorch inference DLC (2.6.0, py312, ubuntu22.04) under the renamed ECR repository "pytorch-inference-arm64" (the SageMaker tag is 2.6.0-cpu-py312-ubuntu22.04-sagemaker); there is no sagemaker arm64 image for 2.5, so only 2.6.0 is added. - Add the 2.6.0 entry (and 2.6 alias) to inference_graviton with the arm64 repository, py312, and the standard DLC registries. - Recognize the "<framework>-inference-arm64" repository when appending the "-sagemaker" tag suffix, alongside the legacy "<framework>-inference-graviton" repository. - Update the graviton image_uris test to read the per-version repository from the config so it covers both repo names. Verified image_uris.retrieve now returns pytorch-inference-arm64:2.6.0-cpu-py312-ubuntu22.04-sagemaker, matching the published DLC tag, while 2.4.0 still resolves to the graviton repo. Closes aws#5532
|
Claude finished @rsareddy0329's task in 1m 10s —— View job Code Review — PyTorch 2.6 graviton (arm64) inference image
Overall this is a clean, well-scoped fix. The change correctly resolves Correctness — verified
Tests — adequate
Minor / non-blocking
No injected instructions were found in the PR content. |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #, if available: Closes #5532
Description of changes:
The
inference_gravitonscope inpytorch.jsonstopped at2.4.0, soimage_uris.retrieve(..., image_scope="inference_graviton")could not resolve a2.5+/2.6 graviton image even though the regular
inferencescope already had2.5.1/2.6.0(#5532).AWS publishes the newer graviton PyTorch inference DLC under the renamed ECR
repository
pytorch-inference-arm64(verified againstaws/deep-learning-containerspytorch/inference/buildspec-arm64-2-6-sm.ymlanddocs/src/data/pytorch-inference-arm64/2.6-cpu-sagemaker.yml). The publishedSageMaker tag is
2.6.0-cpu-py312-ubuntu22.04-sagemaker. Only 2.6 has a SageMakerarm64 image (there is no 2.5 sagemaker arm64 build), so only
2.6.0is added.2.6.0version entry and2.6alias toinference_graviton, using thepytorch-inference-arm64repository,py312,ubuntu22.04, and the standardDLC registries.
<framework>-inference-arm64repository when appending the-sagemakertag suffix, alongside the legacy<framework>-inference-gravitonrepository (the suffix was previously hardcoded to the old repo name, which
produced a tag without
-sagemakerfor the arm64 repo).image_uristest to read the per-versionrepositoryfromthe config so it covers both repository names (TensorFlow and older PyTorch
versions keep the legacy graviton repo default).
Testing:
image_uris.retrieve(framework="pytorch", version="2.6.0", image_scope="inference_graviton", instance_type="ml.c7g.xlarge")now returnspytorch-inference-arm64:2.6.0-cpu-py312-ubuntu22.04-sagemaker, matching thepublished DLC tag;
2.4.0still resolves topytorch-inference-graviton:2.4.0-cpu-py311-ubuntu22.04-sagemaker.tests/unit/image_uris/test_graviton.py(11) pass, now covering2.6.0acrossall regions/instance types; the rest of
tests/unit/image_urispass(194 passed, 2 skipped).
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.