Repository navigation
Structure for backwarding gramians #100
Description
It appears to be important to be able to bakward Gramians of Jacobians through some Sequential. It is thus important that we think about how to do that.
I believe we could in principle have a backward_gramian method that takes as input the vector to backward (the output of the module) and the Sequential. It would then go through each layer, compute the Jacobian and accumulate the Gramian.
From this we could extract weights (with a weight extractor that maps a Gramian to weights) and backpropagates them using a normal call to autograd.backward.
This may not be the best way to proceed, so we need further discussions.
Activity
[-]Structure for backward gramians[/-][+]Structure for backwarding gramians[/+]ValerianRey commentedon Aug 26, 2024
I think this is an entirely new project, not sure we should discuss about this in a github issue.
PierreQuinton commentedon Aug 26, 2024
I don't agree, this could be as simple as a new function (
backward_gramian) and refactoring a bit the structure of the aggregators to exhibit Gramian based weight extractor.