How do I run uwsgi with virtualenv
Question:
I’m currently developing my first real python flask project and am about to set up the build server to deploy the “Latest Build” which is built on every check-in.
I have set up a startup script where I start the application using uwsgi and this part is working fine. I have recently also started using virtualenv and by doing so the packages installed are added to my project under projectnameflaskLibsite-packages.
I’m using nginx as the web server and the config looks like this:
location / { try_files $uri @graderbuild; }
location @graderbuild {
include uwsgi_params;
uwsgi_param UWSGI_CHDIR /usr/local/grader/build;
uwsgi_param UWSGI_PYHOME /usr/local/grader/build;
uwsgi_pass 127.0.0.1:3031;
}
I’m starting uwsgi using this:
exec /usr/local/bin/uwsgi --master --socket 127.0.0.1:3031
--wsgi-file restserver.py --callable app --processes 4 --die-on-term
--threads 2 >> /var/log/grader-build.log 2>&1
Now to where I know if I’m doing it right… currently I am deploying the entire folder to the build server. I don’t want to install global python modules just to get my build to work. Right or wrong?
The error I get currently is:
ImportError: No module named flask_wtf
If I’m right, how do I configure the setup to use the virtualenv site-packages? My preferred location would be in the startup script and not in the nginx config.
Answers:
Use -H to set virtualenv to python path.
uwsgi -H /path/to/your/virtualenv
http://uwsgi-docs.readthedocs.org/en/latest/Options.html#virtualenv
As user995394 pointed out, there is a way to tell uWSGI use existing virtual environment.
However, when I pass uWSGI option in form virtualenv = /full/path/to/my/virtualenv (it’s from INI config) it complains about ImportError: No module named site. The workaround I found is that you launch uWSGI from folder where your virtualenv is and pass just virtualenv = my_virtualenv_name (i.e. path is relative).
I use uWSGI 2.0.
To use the activated virtualenv you can use this config snippet in your uwsgi.ini:
; If VIRTUAL_ENV is set then use its value to specify the virtualenv directory
if-env = VIRTUAL_ENV
virtualenv = %(_)
endif =
I had this issue a few months back and have a full example of demo configs here including nginx, uwsgi starting automatically with upstart on linux.
Beau’s answer resolved this issue for me.
I never did find a good explanation for uwsgi’s ini file directives.
Until Beau’s answer I never saw an answer to what explicitly the virtualenv value should be set to – the root of the python tree in the venv, the app’s folder under site-packages or the root of the VENV TREE. What if you aren’t using a venv, what do you set home to, top of app tree, top of python bin folder, python lib folder or to dist-packages?
I have this app working on another system, so it really shouldn’t have been that difficult to run it under a docker container. Now that I have got it working I reviewed that working installation and now see it points to the top of the venv tree. I was using virtualenvwrapper there, so it’s a different path than when using only virtualenv.
It makes me wonder if it’s possible to run this app without the venv. Since this will run in a docker container there isn’t really a good reason to use venvs, but in looking at the python folder structure differences they are quite different between system python and venv python.
System’s python3 is split into separate folders and the files are not all under a single hierarchy as they are under a venv. If you install your packages with pip they will end up in /usr/local/lib/python3/dist-packages, and that location does NOT have site.py or encodings folders which is why so many have import errors.
After only a few trials I discovered that to run my app without a venv the uwsgi ini should NOT define either home OR virtualenv settings. If your system path includes both /usr/bin AND /usr/local/bin it should work and find everything, even tho pip installed packages go somewhere else with a different folder hierarchy.
Others’ answers didn’t help, and I added path to virtualenv to uwsgi.ini configuration file. The error disappeared.
pythonpath = /path-to-virtualenv/project/lib/python2.7/site-packages
I’m currently developing my first real python flask project and am about to set up the build server to deploy the “Latest Build” which is built on every check-in.
I have set up a startup script where I start the application using uwsgi and this part is working fine. I have recently also started using virtualenv and by doing so the packages installed are added to my project under projectnameflaskLibsite-packages.
I’m using nginx as the web server and the config looks like this:
location / { try_files $uri @graderbuild; }
location @graderbuild {
include uwsgi_params;
uwsgi_param UWSGI_CHDIR /usr/local/grader/build;
uwsgi_param UWSGI_PYHOME /usr/local/grader/build;
uwsgi_pass 127.0.0.1:3031;
}
I’m starting uwsgi using this:
exec /usr/local/bin/uwsgi --master --socket 127.0.0.1:3031
--wsgi-file restserver.py --callable app --processes 4 --die-on-term
--threads 2 >> /var/log/grader-build.log 2>&1
Now to where I know if I’m doing it right… currently I am deploying the entire folder to the build server. I don’t want to install global python modules just to get my build to work. Right or wrong?
The error I get currently is:
ImportError: No module named flask_wtf
If I’m right, how do I configure the setup to use the virtualenv site-packages? My preferred location would be in the startup script and not in the nginx config.
Use -H to set virtualenv to python path.
uwsgi -H /path/to/your/virtualenv
http://uwsgi-docs.readthedocs.org/en/latest/Options.html#virtualenv
As user995394 pointed out, there is a way to tell uWSGI use existing virtual environment.
However, when I pass uWSGI option in form virtualenv = /full/path/to/my/virtualenv (it’s from INI config) it complains about ImportError: No module named site. The workaround I found is that you launch uWSGI from folder where your virtualenv is and pass just virtualenv = my_virtualenv_name (i.e. path is relative).
I use uWSGI 2.0.
To use the activated virtualenv you can use this config snippet in your uwsgi.ini:
; If VIRTUAL_ENV is set then use its value to specify the virtualenv directory
if-env = VIRTUAL_ENV
virtualenv = %(_)
endif =
I had this issue a few months back and have a full example of demo configs here including nginx, uwsgi starting automatically with upstart on linux.
Beau’s answer resolved this issue for me.
I never did find a good explanation for uwsgi’s ini file directives.
Until Beau’s answer I never saw an answer to what explicitly the virtualenv value should be set to – the root of the python tree in the venv, the app’s folder under site-packages or the root of the VENV TREE. What if you aren’t using a venv, what do you set home to, top of app tree, top of python bin folder, python lib folder or to dist-packages?
I have this app working on another system, so it really shouldn’t have been that difficult to run it under a docker container. Now that I have got it working I reviewed that working installation and now see it points to the top of the venv tree. I was using virtualenvwrapper there, so it’s a different path than when using only virtualenv.
It makes me wonder if it’s possible to run this app without the venv. Since this will run in a docker container there isn’t really a good reason to use venvs, but in looking at the python folder structure differences they are quite different between system python and venv python.
System’s python3 is split into separate folders and the files are not all under a single hierarchy as they are under a venv. If you install your packages with pip they will end up in /usr/local/lib/python3/dist-packages, and that location does NOT have site.py or encodings folders which is why so many have import errors.
After only a few trials I discovered that to run my app without a venv the uwsgi ini should NOT define either home OR virtualenv settings. If your system path includes both /usr/bin AND /usr/local/bin it should work and find everything, even tho pip installed packages go somewhere else with a different folder hierarchy.
Others’ answers didn’t help, and I added path to virtualenv to uwsgi.ini configuration file. The error disappeared.
pythonpath = /path-to-virtualenv/project/lib/python2.7/site-packages