How to use poetry in Google Colab?
Question:
Context
I am currently working on a team project, where we need to train neural networks. Some members are working on their local computer, and some on Colab (for GPU usage). We need to have the same dependencies. I am already familiar in using poetry on a local computer, but not on Colab, and I was wondering how to use it in Colab.
So I did some tests, and I encountered some issues. Maybe I can find some answers here. Thank you in advance!
Issues
1st issue: poetry add <package> does not update pyproject.toml
I want to add a new package, suppose torch.
According to poetry’s documentation, to install a new package, we need to run poetry add <package>. Since I run this command for the first time, a virtual environment is created, as well as the poetry.lock. But no packages were installed. Moreover, the poetry.lock file is updated, but not the pyproject.toml. This happens only on Colab. I tried on my local computer, and the command indeed automatically update the pyproject.toml file as well.
2nd issue: poetry run pip install <package> does not update pyproject.toml
Instead, we can install the package with poetry run pip install <package>. I saw this command on the following GitHub gist. The packages are now installed in the virtual environment, but the pyproject.toml was not updated.
Here is a link to the Colab notebook I used for those tests.
Thank you again!
Answers:
Hello for everyone reading this post.
I settled on a solution. The main issue was that pyproject.toml was not updated automatically, so I just decided to modify it by hand.
Here is the steps for using poetry in Colab, whether you create your own poetry project, or cloning a repo on Github.
I don’t know if it is the best solution or not, but I hope it will help other people 🙂
Context
I am currently working on a team project, where we need to train neural networks. Some members are working on their local computer, and some on Colab (for GPU usage). We need to have the same dependencies. I am already familiar in using poetry on a local computer, but not on Colab, and I was wondering how to use it in Colab.
So I did some tests, and I encountered some issues. Maybe I can find some answers here. Thank you in advance!
Issues
1st issue: poetry add <package> does not update pyproject.toml
I want to add a new package, suppose torch.
According to poetry’s documentation, to install a new package, we need to run poetry add <package>. Since I run this command for the first time, a virtual environment is created, as well as the poetry.lock. But no packages were installed. Moreover, the poetry.lock file is updated, but not the pyproject.toml. This happens only on Colab. I tried on my local computer, and the command indeed automatically update the pyproject.toml file as well.
2nd issue: poetry run pip install <package> does not update pyproject.toml
Instead, we can install the package with poetry run pip install <package>. I saw this command on the following GitHub gist. The packages are now installed in the virtual environment, but the pyproject.toml was not updated.
Here is a link to the Colab notebook I used for those tests.
Thank you again!
Hello for everyone reading this post.
I settled on a solution. The main issue was that pyproject.toml was not updated automatically, so I just decided to modify it by hand.
Here is the steps for using poetry in Colab, whether you create your own poetry project, or cloning a repo on Github.
I don’t know if it is the best solution or not, but I hope it will help other people 🙂