Repository navigation
Retrieving loss info in MTL #323
Description
Great paper. I wonder if there is any way to retrieve the loss information in multi-task learning?
In particular, in the example of MTL, do you still have to add loss=loss1+loss2 into the script?
Activity
ValerianRey commentedon Apr 24, 2025
Hi! In multi-task learning, we typically consider each loss independently, as they each represent a different task. You may still want to track the average loss, or the sum of losses, by recording the value of
loss=loss1+loss2for instance. But note that with JD, thislossis not necessary anymore for training, as you use the individual losses instead.Also, note that
loss1andloss2may have very different scales (e.g.loss1could be typically around1e-5whileloss2could be around1e-2). In this example, summing them could thus give a value very close toloss2, which would give little information onloss1.So yes, you can do that, but be careful!