I have a text of several sentences.
I want to use a pre-trained ELMO & BERT models, go over the sentences and:
- For each sentence , calculate a score of how “likely” (perplexity?) it is in the LM. So, for example, the score for
the desk is greenshould be relatively high and the score for
the paper lies on the floorshould be low
- Given a context, what is the probability of a word. For example, given the context
the desk iswhat is the probability for the words
high, etc’ …
Is this possible? How can I do it in Python?