Poetry add dependency that uses cython
Question:
I have a project which needs to depend on the latest commit of pysam, because I’m working in python 3.11.
This means building the package from source, so I do the following:
poetry add git+https://github.com/pysam-developers/pysam
However, I get an error which I think boils down to poetry not including cython in the build environment:
Unable to determine package info for path: /Users/agreen/Library/Caches/pypoetry/virtualenvs/rnacentral-pipeline-GU-1IkEM-py3.11/src/pysam
Fallback egg_info generation failed.
Command ['/var/folders/sg/3858brmd79z4rz781g0q__940000gp/T/tmpw8auvhsm/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1, and output:
# pysam: no cython available - using pre-compiled C
Traceback (most recent call last):
File "/Users/agreen/Library/Caches/pypoetry/virtualenvs/rnacentral-pipeline-GU-1IkEM-py3.11/src/pysam/setup.py", line 345, in <module>
raise ValueError(
ValueError: no cython installed, but can not find pysam/libchtslib.c.Make sure that cython is installed when building from the repository
Cython is definitely installed, its in the pyproject.toml, and I can call it from the poetry shell, or import it in a python started in the poetry virtualenv. However, If I use the python from the command poetry is running, then indeed cython is not available.
I think I’m missing some configuration of the build, or some extra option to poetry add. The documentation isn’t particularly clear about this use of cython – as far as I can tell it’s all about using cython in the package I’m writing, which is not quite what I want.
Answers:
Cython is a build dependency of pysam, but apparently pysam does not have a pyproject.toml and thus does not declare its build dependencies (Cython and maybe others). So this is a dead end.
If I were you I would build a wheel of pysam myself and tell Poetry to use this wheel until pysam releases wheels for Python 3.11 on PyPI themselves. Or I would use Python 3.10.
It seems like it is being worked on: https://github.com/pysam-developers/pysam/pull/1168
I have a project which needs to depend on the latest commit of pysam, because I’m working in python 3.11.
This means building the package from source, so I do the following:
poetry add git+https://github.com/pysam-developers/pysam
However, I get an error which I think boils down to poetry not including cython in the build environment:
Unable to determine package info for path: /Users/agreen/Library/Caches/pypoetry/virtualenvs/rnacentral-pipeline-GU-1IkEM-py3.11/src/pysam
Fallback egg_info generation failed.
Command ['/var/folders/sg/3858brmd79z4rz781g0q__940000gp/T/tmpw8auvhsm/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1, and output:
# pysam: no cython available - using pre-compiled C
Traceback (most recent call last):
File "/Users/agreen/Library/Caches/pypoetry/virtualenvs/rnacentral-pipeline-GU-1IkEM-py3.11/src/pysam/setup.py", line 345, in <module>
raise ValueError(
ValueError: no cython installed, but can not find pysam/libchtslib.c.Make sure that cython is installed when building from the repository
Cython is definitely installed, its in the pyproject.toml, and I can call it from the poetry shell, or import it in a python started in the poetry virtualenv. However, If I use the python from the command poetry is running, then indeed cython is not available.
I think I’m missing some configuration of the build, or some extra option to poetry add. The documentation isn’t particularly clear about this use of cython – as far as I can tell it’s all about using cython in the package I’m writing, which is not quite what I want.
Cython is a build dependency of pysam, but apparently pysam does not have a pyproject.toml and thus does not declare its build dependencies (Cython and maybe others). So this is a dead end.
If I were you I would build a wheel of pysam myself and tell Poetry to use this wheel until pysam releases wheels for Python 3.11 on PyPI themselves. Or I would use Python 3.10.
It seems like it is being worked on: https://github.com/pysam-developers/pysam/pull/1168