Where in a virtualenv does the custom code go?
Question:
What sort of directory structure should one follow when using virtualenv? For instance, if I were building a WSGI application and created a virtualenv called foobar I would start with a directory structure like:
/foobar
/bin
{activate, activate.py, easy_install, python}
/include
{python2.6/...}
/lib
{python2.6/...}
Once this environment is created, where would one place their own:
- python files?
- static files (images/etc)?
- “custom” packages, such as those available online but not found in the cheese-shop?
in relation to the virtualenv directories?
(Assume I already know where the virtualenv directories themselves should go.)
Answers:
If you only have a few projects every so often, nothing stops you from creating a new virtualenv for each one, and putting your packages right inside:
/foobar
/bin
{activate, activate.py, easy_install, python}
/include
{python2.6/...}
/lib
{python2.6/...}
/mypackage1
__init__.py
/mypackage2
__init__.py
The advantage of this approach is that you can always be sure to find find the activate script that belongs to the project inside.
$ cd /foobar
$ source bin/activate
$ python
>>> import mypackage1
>>>
If you decide to be a bit more organized, you should consider putting all your virtualenvs into one folder, and name each of them after the project you are working on.
/virtualenvs
/foobar
/bin
{activate, activate.py, easy_install, python}
/include
{python2.6/...}
/lib
{python2.6/...}
/foobar
/mypackage1
__init__.py
/mypackage2
__init__.py
This way you can always start over with a new virtualenv when things go wrong, and your project files stay safe.
Another advantage is that several of your projects can use the same virtualenv, so you don’t have to do the same installation over and over if you have a lot of dependencies.
$ cd /foobar
$ source ../virtualenvs/foobar/bin/activate
$ python
>>> import mypackage2
>>>
For users that regularly have to set up and tear down virtualenvs it would make sense to look at virtualenvwrapper.
http://pypi.python.org/pypi/virtualenvwrapper
With virtualenvwrapper you can
* create and delete virtual environments
* organize virtual environments in a central place
* easily switch between environments
You no more have to worry about where your virtualenvs are when working on the projects “foo” and “bar”:
/foo
/mypackage1
__init__.py
/bar
/mypackage2
__init__.py
This is how you start working on project “foo”:
$ cd foo
$ workon
bar
foo
$ workon foo
(foo)$ python
>>> import mypackage1
>>>
Then switching to project “bar” is as simple as this:
$ cd ../bar
$ workon bar
(bar)$ python
>>> import mypackage2
>>>
Pretty neat, isn’t it?
If you give your project a setup.py, pip can import it from version control directly.
Do something like this:
$ virtualenv --no-site-packages myproject
$ . myproject/bin/activate
$ easy_install pip
$ pip install -e hg+http://bitbucket.org/owner/myproject#egg=proj
The -e will put the project in myproject/src, but link it to myproject/lib/pythonX.X/site-packages/, so any changes you make will get picked up immediately in modules that import it from your local site-packages. The #egg bit tells pip what name you want to give to the egg package it creates for you.
If you don’t use --no-site-packages, be careful to specify that you want pip to install into the virtualenv with the -E option
virtualenv provides a python interpreter instance, not an application instance. You wouldn’t normally create your application files within the directories containing a system’s default Python, likewise there’s no requirement to locate your application within a virtualenv directory.
For example, you might have a project where you have multiple applications using the same virtualenv. Or, you may be testing an application with a virtualenv that will later be deployed with a system Python. Or, you may be packaging up a standalone app where it might make sense to have the virtualenv directory located somewhere within the app directory itself.
So, in general, I don’t think there is one right answer to the question. And, a good thing about virtualenv is that it supports many different use cases: there doesn’t need to be one right way.
Because virtualenvs are not relocatable, in my opinion it is bad practice to place your project files inside a virtualenv directory. The virtualenv itself is a generated development/deployment artifact (sort of like a .pyc file), not part of the project; it should be easy to blow it away and recreate it anytime, or create a new one on a new deploy host, etc.
Many people in fact use virtualenvwrapper, which removes the actual virtualenvs from your awareness almost completely, placing them all side-by-side in $HOME/.virtualenvs by default.
What sort of directory structure should one follow when using virtualenv? For instance, if I were building a WSGI application and created a virtualenv called foobar I would start with a directory structure like:
/foobar
/bin
{activate, activate.py, easy_install, python}
/include
{python2.6/...}
/lib
{python2.6/...}
Once this environment is created, where would one place their own:
- python files?
- static files (images/etc)?
- “custom” packages, such as those available online but not found in the cheese-shop?
in relation to the virtualenv directories?
(Assume I already know where the virtualenv directories themselves should go.)
If you only have a few projects every so often, nothing stops you from creating a new virtualenv for each one, and putting your packages right inside:
/foobar
/bin
{activate, activate.py, easy_install, python}
/include
{python2.6/...}
/lib
{python2.6/...}
/mypackage1
__init__.py
/mypackage2
__init__.py
The advantage of this approach is that you can always be sure to find find the activate script that belongs to the project inside.
$ cd /foobar
$ source bin/activate
$ python
>>> import mypackage1
>>>
If you decide to be a bit more organized, you should consider putting all your virtualenvs into one folder, and name each of them after the project you are working on.
/virtualenvs
/foobar
/bin
{activate, activate.py, easy_install, python}
/include
{python2.6/...}
/lib
{python2.6/...}
/foobar
/mypackage1
__init__.py
/mypackage2
__init__.py
This way you can always start over with a new virtualenv when things go wrong, and your project files stay safe.
Another advantage is that several of your projects can use the same virtualenv, so you don’t have to do the same installation over and over if you have a lot of dependencies.
$ cd /foobar
$ source ../virtualenvs/foobar/bin/activate
$ python
>>> import mypackage2
>>>
For users that regularly have to set up and tear down virtualenvs it would make sense to look at virtualenvwrapper.
http://pypi.python.org/pypi/virtualenvwrapper
With virtualenvwrapper you can
* create and delete virtual environments
* organize virtual environments in a central place
* easily switch between environments
You no more have to worry about where your virtualenvs are when working on the projects “foo” and “bar”:
/foo
/mypackage1
__init__.py
/bar
/mypackage2
__init__.py
This is how you start working on project “foo”:
$ cd foo
$ workon
bar
foo
$ workon foo
(foo)$ python
>>> import mypackage1
>>>
Then switching to project “bar” is as simple as this:
$ cd ../bar
$ workon bar
(bar)$ python
>>> import mypackage2
>>>
Pretty neat, isn’t it?
If you give your project a setup.py, pip can import it from version control directly.
Do something like this:
$ virtualenv --no-site-packages myproject
$ . myproject/bin/activate
$ easy_install pip
$ pip install -e hg+http://bitbucket.org/owner/myproject#egg=proj
The -e will put the project in myproject/src, but link it to myproject/lib/pythonX.X/site-packages/, so any changes you make will get picked up immediately in modules that import it from your local site-packages. The #egg bit tells pip what name you want to give to the egg package it creates for you.
If you don’t use --no-site-packages, be careful to specify that you want pip to install into the virtualenv with the -E option
virtualenv provides a python interpreter instance, not an application instance. You wouldn’t normally create your application files within the directories containing a system’s default Python, likewise there’s no requirement to locate your application within a virtualenv directory.
For example, you might have a project where you have multiple applications using the same virtualenv. Or, you may be testing an application with a virtualenv that will later be deployed with a system Python. Or, you may be packaging up a standalone app where it might make sense to have the virtualenv directory located somewhere within the app directory itself.
So, in general, I don’t think there is one right answer to the question. And, a good thing about virtualenv is that it supports many different use cases: there doesn’t need to be one right way.
Because virtualenvs are not relocatable, in my opinion it is bad practice to place your project files inside a virtualenv directory. The virtualenv itself is a generated development/deployment artifact (sort of like a .pyc file), not part of the project; it should be easy to blow it away and recreate it anytime, or create a new one on a new deploy host, etc.
Many people in fact use virtualenvwrapper, which removes the actual virtualenvs from your awareness almost completely, placing them all side-by-side in $HOME/.virtualenvs by default.