Using beaker with external packages

I’d like to use beaker to experiment with a model which is not part of the allennlp package. How do you --include-package while using scripts/ai2-internal/run_with_beaker.py?

Hi @wammar. Unfortunately I don’t know think it would be very easy to use --include-package with run_with_beaker.py. I would typically advocate creating a Docker image for your external package that depends on an AllenNLP base image and running that with Beaker. You can look at the Dockerfile in the demo as an example (https://github.com/allenai/allennlp-demo).

A big benefit you’ll get with this approach is that you won’t have to rebuild the entire AllenNLP image which can take a long time.

1 Like

In general, it seems like we need a better way of running beaker experiments once we split out separate task-specific repositories. We should make sure that these repos set up experiments in a way that’s a good example for people that have this question in the future.

I don’t think it would be hard to create a run_with_beaker equivalent that uses the allennlp/allennlp base image and only expects you to supply supplemental modules

Honestly I think running Beaker jobs on dependent repositories is easier than on the main repository already. @joelg I don’t disagree that it would be easy to make a derivative script.