I’m trying to visualize (by DEMO) the attention weights of an existing model (copynent_seq2seq). Unfortunately, this information isn’t passed in the outputs dictionary, so it seems like I have to copy the model (ironic, I know…;)) and make some changes.
The relevant information is computed in _decoder_step() and take_search_step(), and should go all the way up to forward() which returns the outputs dictionary. So the path:
_decoder_step() -> take_search_step()-> _beam_search.search() ->_forward_beam_search() ->forward()
I thought to pass it as state[“attentive_weights”] state[“selective_weights”], but _beam_search.search() doesn’t return states. There is a “clean” (minimal changes) way to do so?
And more generally, I think it would be great to be able to get any internal data of any (existing) model, maybe by variable naming convention (?). It is so important for interpretability.