[ppraneth]

No description provided.

[ValerianRey]

Nice catch!

I think the formulation is a bit wrong. It's not that a forward pass is not used to compute the output, it's that it's a forward pass that doesn't have an associated backward pass.

In your example, the output is losses, which does depend on target, so the forward pass of encoder used to compute target is actually used to compute the output. It's just that we won't ever backward through the target because of the detach.

Could you fix the formulation? Then I think we can merge this!

[ValerianRey]

Basically, for future reference, as discussed on discord privately, the reason of this bug is that during a forward pass, the remaining_counter of the module is always incremented, but sometimes (when the output of the module is detached, like in the example), the backward pass of the module is never called, so the remaining_counter never reaches zero, and the contribution from other backward passes of this same module is thus never added to the global gramian. Since the reasons making the remaining_counter not reach 0 could also be different, it's safer to just raise an error, which this PR adds.

cc @PierreQuinton