Skip to content

Recognise the repeat_interleave idiom in RemovePermutesAroundElementwiseOps (#21790) - #21790

Open
mcremon-meta wants to merge 1 commit into
mainfrom
export-D114508262
Open

Recognise the repeat_interleave idiom in RemovePermutesAroundElementwiseOps (#21790)#21790
mcremon-meta wants to merge 1 commit into
mainfrom
export-D114508262

Conversation

@mcremon-meta

@mcremon-meta mcremon-meta commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary:

repeat_interleave(scale, dim) lowers to a three-node idiom::

unsqueeze(dim + 1) -> expand_copy(scale at dim + 1)
                   -> view_copy(merge dim, dim + 1)

which the region walker could not cross, so any permuted region containing one
was rejected. torchaudio's Stretch2d is built from this, so it blocks every
upsampling network -- on wavernn it strands the permutes ConvToChannelsLast
put around each conv in the upsample path.

Adding expand_copy to _permutable_ops on its own does not help: traversal
then dies one node later at the merging view_copy. That merge is not
unconditionally layout-invariant the way cat/slice are -- flattening two
dims only commutes with a permutation that leaves them adjacent and in order,
otherwise the flatten reorders elements.

Worse, handling the three nodes separately would force a choice of un-permuted
position for the intermediate unit dim (inserting a size-1 dim is ambiguous:
any position gives a consistent permutation). That arbitrary choice then
decides whether the later merge is still legal, coupling two rewrites that
look independent.

So the triple is matched as a unit (_interleave_triple). It is rank-preserving
overall, so the permutation flows through unchanged and the only rewrite needed
is remapping the interleaved dim, new_dim = P[old_dim], plus rebuilding the
three shape args in un-permuted space (update_interleave). No intermediate
layout is ever materialised, so the ambiguity does not arise.

Guards: head and expand must each have exactly one user (otherwise the in-place
rewrite would corrupt another consumer), every non-inserted dim of the expand
must pass through untouched, and the view's output shape must be exactly the
merge. Symbolic shapes are declined. The head may be either an explicit
unsqueeze_copy or a view_copy that inserts one unit dim, matching how the
pass already treats the two spellings interchangeably.

Differential Revision: D114508262

@pytorch-bot

pytorch-bot Bot commented Aug 12, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21790

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 2ec948f with merge base c6213ae (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 12, 2026
@meta-codesync

meta-codesync Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

@mcremon-meta has exported this pull request. If you are a Meta employee, you can view the originating Diff in D114508262.

@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@meta-codesync meta-codesync Bot changed the title Recognise the repeat_interleave idiom in RemovePermutesAroundElementwiseOps Recognise the repeat_interleave idiom in RemovePermutesAroundElementwiseOps (#21790) Aug 12, 2026
meta-codesync Bot pushed a commit that referenced this pull request Aug 12, 2026
…iseOps (#21790)

Summary:

`repeat_interleave(scale, dim)` lowers to a three-node idiom::

    unsqueeze(dim + 1) -> expand_copy(scale at dim + 1)
                       -> view_copy(merge dim, dim + 1)

which the region walker could not cross, so any permuted region containing one
was rejected. torchaudio's `Stretch2d` is built from this, so it blocks every
upsampling network -- on wavernn it strands the permutes `ConvToChannelsLast`
put around each conv in the upsample path.

Adding `expand_copy` to `_permutable_ops` on its own does not help: traversal
then dies one node later at the merging `view_copy`. That merge is *not*
unconditionally layout-invariant the way `cat`/`slice` are -- flattening two
dims only commutes with a permutation that leaves them adjacent and in order,
otherwise the flatten reorders elements.

Worse, handling the three nodes separately would force a choice of un-permuted
position for the intermediate unit dim (inserting a size-1 dim is ambiguous:
any position gives a consistent permutation). That arbitrary choice then
decides whether the later merge is still legal, coupling two rewrites that
look independent.

So the triple is matched as a unit (`_interleave_triple`). It is rank-preserving
overall, so the permutation flows through unchanged and the only rewrite needed
is remapping the interleaved dim, `new_dim = P[old_dim]`, plus rebuilding the
three shape args in un-permuted space (`update_interleave`). No intermediate
layout is ever materialised, so the ambiguity does not arise.

Guards: head and expand must each have exactly one user (otherwise the in-place
rewrite would corrupt another consumer), every non-inserted dim of the expand
must pass through untouched, and the view's output shape must be exactly the
merge. Symbolic shapes are declined. The head may be either an explicit
`unsqueeze_copy` or a `view_copy` that inserts one unit dim, matching how the
pass already treats the two spellings interchangeably.

Differential Revision: D114508262
@meta-codesync
meta-codesync Bot force-pushed the export-D114508262 branch from f7d45fa to 657d2fd Compare August 12, 2026 20:32
…iseOps (#21790)

Summary:

`repeat_interleave(scale, dim)` lowers to a three-node idiom::

    unsqueeze(dim + 1) -> expand_copy(scale at dim + 1)
                       -> view_copy(merge dim, dim + 1)

which the region walker could not cross, so any permuted region containing one
was rejected. torchaudio's `Stretch2d` is built from this, so it blocks every
upsampling network -- on wavernn it strands the permutes `ConvToChannelsLast`
put around each conv in the upsample path.

Adding `expand_copy` to `_permutable_ops` on its own does not help: traversal
then dies one node later at the merging `view_copy`. That merge is *not*
unconditionally layout-invariant the way `cat`/`slice` are -- flattening two
dims only commutes with a permutation that leaves them adjacent and in order,
otherwise the flatten reorders elements.

Worse, handling the three nodes separately would force a choice of un-permuted
position for the intermediate unit dim (inserting a size-1 dim is ambiguous:
any position gives a consistent permutation). That arbitrary choice then
decides whether the later merge is still legal, coupling two rewrites that
look independent.

So the triple is matched as a unit (`_interleave_triple`). It is rank-preserving
overall, so the permutation flows through unchanged and the only rewrite needed
is remapping the interleaved dim, `new_dim = P[old_dim]`, plus rebuilding the
three shape args in un-permuted space (`update_interleave`). No intermediate
layout is ever materialised, so the ambiguity does not arise.

Guards: head and expand must each have exactly one user (otherwise the in-place
rewrite would corrupt another consumer), every non-inserted dim of the expand
must pass through untouched, and the view's output shape must be exactly the
merge. Symbolic shapes are declined. The head may be either an explicit
`unsqueeze_copy` or a `view_copy` that inserts one unit dim, matching how the
pass already treats the two spellings interchangeably.

Differential Revision: D114508262
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant