How to use python 3 as a build script in non-python travis configuration?

Question

I’m trying to use Travis CI on a C library that uses custom python3-based build scripts.

When the repository gets built, it fails on ./configure because configure uses Python 3, which isn’t installed by default.

If I were building a Python project, I would use this in my .travis.yml:

python:
  - "3.4"

However, this doesn’t seem to affect my repository because it is in C. (I tried running which python3 and python --version, which reported that python 3 didn’t exist and python 2.7 was in use instead.)

The build script that I tried:

language: c

python:
  - "3.4"

How can I have Python 3 available in Travis CI when my project is not a Python project?

Asked By: Cel Skeggs

||

Source

Answer 1

You should be able to just install the needed python3 packages by adding a before_install: section of your .travis.yml:

before_install:
- sudo apt-get update
- sudo apt-get install python3

Answered By: Eric Appelt

Answer 2

If you want to use the container based infrastructure you can make use of the apt addon:

addons:
  apt:
    sources:
      - deadsnakes # source required so it finds the package definition below
    packages:
      - python3.5

Packages that can be used are listed here

Update

In order to use dependencies with c-extensions like psycopg2 or pyYAML it is also necessary to add python3.4-dev

Answered By: mfussenegger

Answer 3

The answer of mfussenegger is the recommended way to install stuff nowadays, however deadsnakes packages don’t provide links for python3 it seems, only for python3.*. You would use it if you require a recent python 3 release. Current travis builds on Ubuntu LTS 12.04, which comes with python 3.2, which is enough for most scripting purposes.

python 3.* is a problem when firing up a script with a shebang, where we can’t include arbitrary name checks. That’s why you probably still want the python3 package, as in Eric Appelts answer:

addons:
  apt:
    packages:
      - python3

Answered By: Sebastian Graf

Answer 4

Travis uses pyenv to manage its Python, and has both Python 2.7.16 and 3.6.3 installed by default at the time of writing. You can use pyenv global 3.6 to use Python 3.6.3; you don’t need root for this.

This is the easiest and fastest way to get a Python 3 and useful in cases where you don’t require a specific Python 3 version (e.g. for a build script or the like). If you want specific Python versions and/or a build matrix then installing it as a package or addon as mentioned in some of the other answers is probably faster and easier.

Answered By: Martin Tournoij

Answer 5

This is how it works for me (on bionic):

dist: bionic
addons:
  apt:
    sources:
      - deadsnakes
    packages:
      - python3.7
before_install:
  - sudo apt-get update
  - sudo apt purge python2.7-minimal
  - sudo apt-get install python3 python3-pip python3-setuptools
  - pip3 install --upgrade pip

See it yourself: .travis.yml.

Answered By: yegor256

Answer 6

Unfortunately, my C++ CMake project failed to correctly find the python libs for linking when installing python3.7 and python3.7-dev from deadsnakes.

What worked was to use pyenv, but on the Ubuntu Xenial dist provided by Travis, the shipped version of pyenv was too old and didn’t have Python 3.7 available. What I had to do is to update pyenv first:

  install:
    - PYENV_VERSION=1.2.20
    - PYTHON_VERSION=3.7.8
    - pushd $(pyenv root)
    - git fetch
    - git checkout v$PYENV_VERSION
    - pyenv install -v $PYTHON_VERSION
    - popd
  before_script:
    - pyenv global $PYTHON_VERSION
    - PYTHON_PREFIX=$(python-config --prefix)
    - PYTHON_XY=python${PYTHON_VERSION%.*} # X.Y.Z -> pythonX.Y
    - CMAKE_EXTRA_ARGS+=" -DPYTHON_EXECUTABLE=$PYTHON_PREFIX/bin/$PYTHON_XY"
    - CMAKE_EXTRA_ARGS+=" -DPYTHON_LIBRARY=$PYTHON_PREFIX/lib/lib${PYTHON_XY}m.so"
    - CMAKE_EXTRA_ARGS+=" -DPYTHON_INCLUDE_DIR=$PYTHON_PREFIX/include/${PYTHON_XY}m"
  script:
    - cmake .. -DCMAKE_BUILD_TYPE=Release ${CMAKE_EXTRA_ARGS}
    - make

Answered By: Boris Dalstein

Answer 7

Overview

A cleaner, more modular approach is to use a .python-version configuration file [1] rather than having to directly specify the Python version in the Travis configuration file via the pyenv global 3.6 command.
e.g. I successfully installed and used yamllint with Python 3.6.9 in my Node.js project.

Solution Details

In the root directory of your project, create a .python-version and specify the python 3.6 alias as it’s contents.
- When using the bionic distribution, Travis will resolve 3.6 to it’s default 3.6 Python installation
- e.g. 3.6.9 [2]
Optionally, do the following in your Travis configuration file:
- (a) Instruct Travis to cache any Python packages between builds via the cache syntax
- (b) Upgrade the default version of pip that comes with Python 3.6.9.
- (c) Install yamllint as a global package
  - Note: It must be globally installed when the language is not python [3].

.python-version

3.6

.travis.yml

dist: bionic
language: node_js
cache:
  pip: true
before_install:
  # Upgrade pip
  - python -m pip install --quiet --upgrade pip
  # Install yamllint
  - pip install --quiet --user yamllint

Resources

Answered By: Mike Pawlowski