Use pre ELMO and BERT to get probability of sequence of text

I have a text of several sentences.
I want to use a pre-trained ELMO & BERT models, go over the sentences and:

  1. For each sentence , calculate a score of how “likely” (perplexity?) it is in the LM. So, for example, the score for the desk is green should be relatively high and the score for the paper lies on the floor should be low
  2. Given a context, what is the probability of a word. For example, given the context the desk is what is the probability for the words green, high, etc’ …

Is this possible? How can I do it in Python?