Calling IPython from a virtualenv
Question:
I understand that IPython is not virtualenv-aware and that the most logical solution to this is to install ipython in each virtualenv seperately using
pip install ipython
So far so good. One thing I noticed is that if the system-wide copy of IPython is called from within a virtualenv using $> ipython
before IPython is installed under this virtualenv, subsequent $> ipython
commands will continue to bring up the system-wide ipython copy.
On the other hand, if ipython is not called prior to installing it under a virtualenv $> ipython
will bring up the newly installed copy.
What is the explanation for this?
It also makes me wonder if this behavior means I should expect some trouble down the way?
Answers:
alias ipy="python -c 'import IPython; IPython.terminal.ipapp.launch_new_instance()'"
This is a great way of always being sure that the ipython instance always belongs to the virtualenv’s python version.
This works only on ipython >2.0.
You can force IPython to use a virtual environment if available by adding file below to ~/.ipython/profile_default/startups
:
import os
import sys
if 'VIRTUAL_ENV' in os.environ:
py_version = sys.version_info[:2] # formatted as X.Y
py_infix = os.path.join('lib', ('python%d.%d' % py_version))
virtual_site = os.path.join(os.environ.get('VIRTUAL_ENV'), py_infix, 'site-packages')
dist_site = os.path.join('/usr', py_infix, 'dist-packages')
# OPTIONAL: exclude debian-based system distributions sites
sys.path = filter(lambda p: not p.startswith(dist_site), sys.path)
# add virtualenv site
sys.path.insert(0, virtual_site)
I recommend naming it 00-virtualenv.py
so changes will be made as early as possible.
Note: Make sure ipython is installed in the new virtual environment to get this to work.
As others mentioned, recent versions of ipython are virtualenv aware, so you can use your virtualenv bin activate script to run ipython using your virtualenv, e.g.
$ source venv/bin/activate
(venv) $ ipython
WARNING: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
If you’re trying to open a notebook, even ipython 5 won’t help – ipython will disregard the virtualenv (at least on my machine/setup).
You’ll need to use rgtk’s script, but please make sure to modify the optional filter part and the sys.path.insert as below:
import os
import sys
if 'VIRTUAL_ENV' in os.environ:
py_version = sys.version_info[:2] # formatted as X.Y
py_infix = os.path.join('lib', ('python%d.%d' % py_version))
virtual_site = os.path.join(os.environ.get('VIRTUAL_ENV'), py_infix, 'site-packages')
dist_site = os.path.join('/usr', py_infix, 'dist-packages')
# OPTIONAL: exclude debian-based system distributions sites
# ADD1: sys.path must be a list
sys.path = list(filter(lambda p: not p.startswith(dist_site), sys.path))
# add virtualenv site
# ADD2: insert(0 is wrong and breaks conformance of sys.path
sys.path.insert(1, virtual_site)
- ADD1: in the original script we get back a filter object, we would break sys.path and insert below would fail
- ADD2: see this question and python documentation
(Debian/Ubuntu) assuming some version (x) of Python3 is installed, then:
$ sudo apt-get install -y ipython
$ virtualenv --python=python3.x .venv
$ source .venv/bin/activate
$ pip3 install ipython
$ ipython3
will launch ipython running your version of Python3.
The answer given by @SiddharthaRT is good! Following this approach, it is simpler for me just:
python -m IPython
This will use the module IPython through the python bin, ensuring that it refers to the bin from the virtual env.
-
Activate your virtual environment by using source ~/.virtualenvs/my_venv/bin/activate
or by running workon my_venv
(Depending on how you’ve installed the my_venv virtual environment)
-
Install ipython
pip install ipython
- Now run ipython from my_venv.
If it still loads the system’s ipython, then run hash -r
in your shell
This will reset the cache of your shell and ensure you don’t run another ipython that was already called in this shell session.
I’ll chime in years later in hopes someone finds this useful.
This solution solves a few problems:
- You don’t need iPython installed in the current virtualenv, only for the global Python that matches your virtualenv’s Python version (
3.6 != 3.7
).
- Works for users of
pyenv
where your global Python version might be 3.7
and your local virtualenv Python is 3.6
therefore using the global ipython
will fail.
- Works outside of virtual environments (though not particularly useful as it always targets
python
).
Throw this in your ~/.bashrc
or ~/.zshrc
or what have you:
# This is a roundabout way to start ipython from inside a virtualenv without it being installed
# in that virtualenv. The only caveot is that the "global" python must have ipython installed.
# What this function does that's different than simply calling the global ipython is it ensures to
# call the ipython that is installed for the same major.minor python version as in the virtualenv.
# This is most useful if you use pyenv for example as global python3 could be 3.7 and local
# virtualenv python3 is 3.6.
function ipy {
local PY_BIN
local IPYTHON
local PYV
# This quick way will work if ipython is in the virtualenv
PY_BIN="$(python -c 'import sys; print(sys.executable)')"
IPYTHON="$(dirname "$PY_BIN")/ipython"
if [[ -x "$IPYTHON" ]]; then
"$IPYTHON"
else
# Ask the current python what version it is
PYV="$(python -c 'import sys; print(".".join(str(i) for i in sys.version_info[:2]))')"
echo "Looking for iPython for Python $PYV"
# In a new shell (where pyenv should load if equipped) try to find that version
PY_BIN="$($SHELL -i -c "python$PYV -c 'import sys; print(sys.executable)'")"
"$(dirname "$PY_BIN")/ipython"
fi
}
Then source
or open a new terminal and run ipy
.
I understand that IPython is not virtualenv-aware and that the most logical solution to this is to install ipython in each virtualenv seperately using
pip install ipython
So far so good. One thing I noticed is that if the system-wide copy of IPython is called from within a virtualenv using $> ipython
before IPython is installed under this virtualenv, subsequent $> ipython
commands will continue to bring up the system-wide ipython copy.
On the other hand, if ipython is not called prior to installing it under a virtualenv $> ipython
will bring up the newly installed copy.
What is the explanation for this?
It also makes me wonder if this behavior means I should expect some trouble down the way?
alias ipy="python -c 'import IPython; IPython.terminal.ipapp.launch_new_instance()'"
This is a great way of always being sure that the ipython instance always belongs to the virtualenv’s python version.
This works only on ipython >2.0.
You can force IPython to use a virtual environment if available by adding file below to ~/.ipython/profile_default/startups
:
import os
import sys
if 'VIRTUAL_ENV' in os.environ:
py_version = sys.version_info[:2] # formatted as X.Y
py_infix = os.path.join('lib', ('python%d.%d' % py_version))
virtual_site = os.path.join(os.environ.get('VIRTUAL_ENV'), py_infix, 'site-packages')
dist_site = os.path.join('/usr', py_infix, 'dist-packages')
# OPTIONAL: exclude debian-based system distributions sites
sys.path = filter(lambda p: not p.startswith(dist_site), sys.path)
# add virtualenv site
sys.path.insert(0, virtual_site)
I recommend naming it 00-virtualenv.py
so changes will be made as early as possible.
Note: Make sure ipython is installed in the new virtual environment to get this to work.
As others mentioned, recent versions of ipython are virtualenv aware, so you can use your virtualenv bin activate script to run ipython using your virtualenv, e.g.
$ source venv/bin/activate
(venv) $ ipython
WARNING: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
If you’re trying to open a notebook, even ipython 5 won’t help – ipython will disregard the virtualenv (at least on my machine/setup).
You’ll need to use rgtk’s script, but please make sure to modify the optional filter part and the sys.path.insert as below:
import os
import sys
if 'VIRTUAL_ENV' in os.environ:
py_version = sys.version_info[:2] # formatted as X.Y
py_infix = os.path.join('lib', ('python%d.%d' % py_version))
virtual_site = os.path.join(os.environ.get('VIRTUAL_ENV'), py_infix, 'site-packages')
dist_site = os.path.join('/usr', py_infix, 'dist-packages')
# OPTIONAL: exclude debian-based system distributions sites
# ADD1: sys.path must be a list
sys.path = list(filter(lambda p: not p.startswith(dist_site), sys.path))
# add virtualenv site
# ADD2: insert(0 is wrong and breaks conformance of sys.path
sys.path.insert(1, virtual_site)
- ADD1: in the original script we get back a filter object, we would break sys.path and insert below would fail
- ADD2: see this question and python documentation
(Debian/Ubuntu) assuming some version (x) of Python3 is installed, then:
$ sudo apt-get install -y ipython
$ virtualenv --python=python3.x .venv
$ source .venv/bin/activate
$ pip3 install ipython
$ ipython3
will launch ipython running your version of Python3.
The answer given by @SiddharthaRT is good! Following this approach, it is simpler for me just:
python -m IPython
This will use the module IPython through the python bin, ensuring that it refers to the bin from the virtual env.
-
Activate your virtual environment by using
source ~/.virtualenvs/my_venv/bin/activate
or by runningworkon my_venv
(Depending on how you’ve installed the my_venv virtual environment) -
Install ipython
pip install ipython
- Now run ipython from my_venv.
If it still loads the system’s ipython, then run hash -r
in your shell
This will reset the cache of your shell and ensure you don’t run another ipython that was already called in this shell session.
I’ll chime in years later in hopes someone finds this useful.
This solution solves a few problems:
- You don’t need iPython installed in the current virtualenv, only for the global Python that matches your virtualenv’s Python version (
3.6 != 3.7
). - Works for users of
pyenv
where your global Python version might be3.7
and your local virtualenv Python is3.6
therefore using the globalipython
will fail. - Works outside of virtual environments (though not particularly useful as it always targets
python
).
Throw this in your ~/.bashrc
or ~/.zshrc
or what have you:
# This is a roundabout way to start ipython from inside a virtualenv without it being installed
# in that virtualenv. The only caveot is that the "global" python must have ipython installed.
# What this function does that's different than simply calling the global ipython is it ensures to
# call the ipython that is installed for the same major.minor python version as in the virtualenv.
# This is most useful if you use pyenv for example as global python3 could be 3.7 and local
# virtualenv python3 is 3.6.
function ipy {
local PY_BIN
local IPYTHON
local PYV
# This quick way will work if ipython is in the virtualenv
PY_BIN="$(python -c 'import sys; print(sys.executable)')"
IPYTHON="$(dirname "$PY_BIN")/ipython"
if [[ -x "$IPYTHON" ]]; then
"$IPYTHON"
else
# Ask the current python what version it is
PYV="$(python -c 'import sys; print(".".join(str(i) for i in sys.version_info[:2]))')"
echo "Looking for iPython for Python $PYV"
# In a new shell (where pyenv should load if equipped) try to find that version
PY_BIN="$($SHELL -i -c "python$PYV -c 'import sys; print(sys.executable)'")"
"$(dirname "$PY_BIN")/ipython"
fi
}
Then source
or open a new terminal and run ipy
.