Tracking down warnings from logger

Hi, I’m getting this error periodically during training WARNING - root - NaN or Inf found in input tensor. But my loss isn’t diverging, so I don’t think there’s an issue in my modeling code. I see that there was a GitHub issue on this, but I’m using the latest code on Master and it’s still happening. Is there any way to track down where this message is coming from? I tried doing

import warnings

so that the code would stop when this warning happened, but it didn’t work. looks pretty relevant.

You could also modify the source (I’m guessing that’s in PyTorch?) to dump out more information when that warning is triggered.

I found the issue:
This fails if is 0. This is admittedly a goofy edge case, because if there’s no elements why is this parameter part of the model? Still, I’m not sure this should throw a warning.
Is the warning the desired behavior? If not, I can make a PR.

We’d welcome a PR which avoids this warning in this edge case. Thanks!