Correct way to use venv and pip in pypy
Question:
I’ve been using cpython forever, but I’m new to pypy.
In cpython, this is how I use virtual environments and pip.
python3 -m venv venv
source venv/bin/activate
python3 -m pip install <package>
I recently started using pypy for a project, and noticed that the following works.
pypy3 -m venv venv
source venv/bin/activate
pypy3 -m pip install <package>
Questions:
- Are there any differences between cpython venv/pip and pypy venv/pip?
- Can I create a venv using cpython, and use it with pypy, or vice-versa?
- Similarly, can I install packages using cpython’s pip, and use them from pypy interpreter, or vice-versa?
- Is what I’m doing "correct", or are there any downsides/issues I’ll face in future if I go down this road.
Reasons why I prefer the python3 -m ... invocations:
- venv is present in std. lib, so I don’t have to globally install virtualenv.
- Less ambiguous than using
pip and pip3.
References:
- What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, pipenv, etc?
- Should I use pip or pip3?
EDIT:
Tried to share venv’s between cpython and venv doesn’t work (seems obvious in hindsight). It’s still possible to create two separate venv’s like python3 -m venv cpython_venv and pypy3 -m venv pypy_venv and switch between them as needed. python will be bound to cpython or pypy based on which virtual env is active, and pypi packages need to be installed in every venv where it’s needed.
Answers:
Are there any differences between cpython venv/pip and pypy venv/pip?
Yes, PyPy make some changes in the venv Python code, so they may have some differences. Example for 3.7:
- CPython stdilb: https://github.com/python/cpython/blob/v3.7.13/Lib/venv/__init__.py
- PyPy stdlib: https://github.com/mozillazg/pypy/blob/release-pypy3.7-v7.3.9/lib-python/3/venv/__init__.py
Can I create a venv using cpython, and use it with pypy, or vice-versa?
I wouldn’t recommend that, since they presumably have good reasons for patching the stdlib venv code.
Similarly, can I install packages using cpython’s pip, and use them from pypy interpreter, or vice-versa?
I wouldn’t recommend that, for several reasons.
In the case of binary distributions with compatibility tags, the installer may select a wheel file which is specific to the Python interpreter that pip was running in. This package could be totally broken for a different Python runtime. Use python3 -m pip debug --verbose or pypy3 -m pip debug --verbose to list the supported compatibility tags of each runtime.
Even for pure-python packages with no compiled extensions you’re not safe – it’s also the job of the installer to generate bytecode (.pyc files) at installation time. If you install with a different interpreter, you’ll get incompatible bytecode.
Python packages can and do specify conditional dependencies using environment markers in the packaging metadata. It’s possible for dependency trees to be different between CPython and PyPy based on the platform_python_implementation environment marker.
Is what I’m doing "correct", or are there any downsides/issues I’ll face in future if I go down this road.
Your usage shown in the question is correct.
I’ve been using cpython forever, but I’m new to pypy.
In cpython, this is how I use virtual environments and pip.
python3 -m venv venv
source venv/bin/activate
python3 -m pip install <package>
I recently started using pypy for a project, and noticed that the following works.
pypy3 -m venv venv
source venv/bin/activate
pypy3 -m pip install <package>
Questions:
- Are there any differences between cpython venv/pip and pypy venv/pip?
- Can I create a venv using cpython, and use it with pypy, or vice-versa?
- Similarly, can I install packages using cpython’s pip, and use them from pypy interpreter, or vice-versa?
- Is what I’m doing "correct", or are there any downsides/issues I’ll face in future if I go down this road.
Reasons why I prefer the python3 -m ... invocations:
- venv is present in std. lib, so I don’t have to globally install virtualenv.
- Less ambiguous than using
pipandpip3.
References:
- What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, pipenv, etc?
- Should I use pip or pip3?
EDIT:
Tried to share venv’s between cpython and venv doesn’t work (seems obvious in hindsight). It’s still possible to create two separate venv’s like python3 -m venv cpython_venv and pypy3 -m venv pypy_venv and switch between them as needed. python will be bound to cpython or pypy based on which virtual env is active, and pypi packages need to be installed in every venv where it’s needed.
Are there any differences between cpython venv/pip and pypy venv/pip?
Yes, PyPy make some changes in the venv Python code, so they may have some differences. Example for 3.7:
- CPython stdilb: https://github.com/python/cpython/blob/v3.7.13/Lib/venv/__init__.py
- PyPy stdlib: https://github.com/mozillazg/pypy/blob/release-pypy3.7-v7.3.9/lib-python/3/venv/__init__.py
Can I create a venv using cpython, and use it with pypy, or vice-versa?
I wouldn’t recommend that, since they presumably have good reasons for patching the stdlib venv code.
Similarly, can I install packages using cpython’s pip, and use them from pypy interpreter, or vice-versa?
I wouldn’t recommend that, for several reasons.
In the case of binary distributions with compatibility tags, the installer may select a wheel file which is specific to the Python interpreter that pip was running in. This package could be totally broken for a different Python runtime. Use python3 -m pip debug --verbose or pypy3 -m pip debug --verbose to list the supported compatibility tags of each runtime.
Even for pure-python packages with no compiled extensions you’re not safe – it’s also the job of the installer to generate bytecode (.pyc files) at installation time. If you install with a different interpreter, you’ll get incompatible bytecode.
Python packages can and do specify conditional dependencies using environment markers in the packaging metadata. It’s possible for dependency trees to be different between CPython and PyPy based on the platform_python_implementation environment marker.
Is what I’m doing "correct", or are there any downsides/issues I’ll face in future if I go down this road.
Your usage shown in the question is correct.