Predictor for LanguageModel

I would like to use a predictor to get the results_dict for a trained language model. In particular, I would like to retrieve the hidden state sequence of my language model.

Right now I’m using the built-in allennlp LanguageModel, and the basic Predictor. I’m attempting to produce results as follows:

predictor = Predictor(model, reader)
for inst in iter(dataset):
    results = predictor.predict_instance(inst)

However, this gets me:

  File "/home/willm/code/lm-distill/distill.py", line 12, in sample_trajectories
    results = predictor.predict_instance(inst)
  File "/home/willm/anaconda3/envs/bert-parsing/lib/python3.7/site-packages/allennlp/predictors/predictor.py", line 181, in predict_instance
    outputs = self._model.forward_on_instance(instance)
  File "/home/willm/anaconda3/envs/bert-parsing/lib/python3.7/site-packages/allennlp/models/model.py", line 124, in forward_on_instance
    return self.forward_on_instances([instance])[0]
  File "/home/willm/anaconda3/envs/bert-parsing/lib/python3.7/site-packages/allennlp/models/model.py", line 167, in forward_on_instances
    elif len(output) != batch_size:
TypeError: object of type 'NoneType' has no len()

Should I be using a different kind of Predictor with my language model?

Solved. The problem is that a unidirectional LanguageModel returns None for backward loss.

Perhaps the backward loss should just be absent from the return dict, so that this error doesn’t get thrown? My current work-around is to extend LanguageModel and override decode to remove elements of the return dict with value None.

Thanks for digging into this, @willm. Could you just make a small PR to modify the LanguageModel itself to not include this value? Optionally in the decode step as you mention. Thank you!

PR submitted: