Using Yocto with a distribution using python3 by defaults
Question:
More and more Linux distributions use python 3.x as default python, but Yocto still uses python 2.7. How to use Yocto with one of those distributions?
Answers:
Linux distributions slowly transfer to Python3 on an application by application basis, by adapting the shebang line to use Python 3.
CentOS 7, Ubuntu 14.4 LTS, Debian Jessy all default to Python2.7 if you type python on the commandline.
If Yocto is installed using a package manager, it will be adapted to whatever works on the Linux distribution it works either with a generic sheband (loading python) or with an explicit one (loading python2 or python2.7.
If you install Yocto yourself, and it might not work because the system you are on defaults to a python from the 3 series, you can adapt the shebang line from:
#!/usr/bin/env python
to
#!/usr/bin/env python2
I assume that python2.7 will be available for a few years to come and installable on demands, even if python3 becomes the default on any of those distributions (just like python3 was available when not installed by default).
What you should consider when installing Yocto from source is run it in a virtualenv so that you setup a clean environment, that might be somewhat more work, depending on the dependencies, but ensures a clean working environment for your application, that cannot be broken by any system update of any packages. And if you do that your setup can even use a python2.7.X version different than that supplied by the Linux distribution.
Yocto always runs in a virtualenv. But I’ve found a way to trick it to use python 2 , setting the default python and including it in PATH env variable:
$ source oe-init-build-env build
$ mkdir build/python-bin
$ ln -s /usr/bin/python2 build/python-bin/python
$ ln -s /usr/bin/python2-config build/python-bin/python-config
$ export PATH=$(pwd)/build/python-bin:${PATH}
Thanks all for your help !
The canonical solution here is to use virtualenv to create an environment where “python” is python 2.
You can fix it by overwriting the hosttools symlink yocto creates..
I managed to start the yocto build with the fix from Shan-x but it didn’t build through.
Yocto sources a different env for all recipes.. Some of the recipes, especially from meta-openembedded require hosttools. These hosttools are for example python (which is then expected to be python2).
This hosttools are then symlinked in build/tmp/hosttools and this gets added to $PATH.
python -> /usr/bin/python
to change this to default to python2 just change the symlink to point to /usr/bin/python2
The entire setup:
$ mkdir build/python-bin
$ ln -s /usr/bin/python2 build/python-bin/python
$ ln -s /usr/bin/python2-config build/python-bin/python-config
$ mkdir -p build/tmp/hosttools
$ ln -sf /usr/bin/python2 build/tmp/hosttools/python
to automatically change to python2 add the export $PATH to
sources/poky/oe-init-build-env , just before the other stuff gets sourced:
diff --git a/oe-init-build-env b/oe-init-build-env
index e813230a98..c981358577 100755
--- a/oe-init-build-env
+++ b/oe-init-build-env
@@ -47,6 +47,8 @@ if [ -z "$OEROOT" ]; then
fi
unset THIS_SCRIPT
+export PATH=$(pwd)/build/python-bin:${PATH}
+
export OEROOT
. $OEROOT/scripts/oe-buildenv-internal &&
TEMPLATECONF="$TEMPLATECONF" $OEROOT/scripts/oe-setup-builddir || {
and then source the env:
$ source oe-init-build-env build
I just completed a fresh qemux86-64 (default) build of kirstone on Ubuntu 22.04 by just creating a symlink to python2 and setting up PATH to point to that. For example, from the root of the yocto repo:
ln -s /usr/bin/python2 ./python
export PATH=$PWD:$PATH
Crude but simple and effective.
More and more Linux distributions use python 3.x as default python, but Yocto still uses python 2.7. How to use Yocto with one of those distributions?
Linux distributions slowly transfer to Python3 on an application by application basis, by adapting the shebang line to use Python 3.
CentOS 7, Ubuntu 14.4 LTS, Debian Jessy all default to Python2.7 if you type python on the commandline.
If Yocto is installed using a package manager, it will be adapted to whatever works on the Linux distribution it works either with a generic sheband (loading python) or with an explicit one (loading python2 or python2.7.
If you install Yocto yourself, and it might not work because the system you are on defaults to a python from the 3 series, you can adapt the shebang line from:
#!/usr/bin/env python
to
#!/usr/bin/env python2
I assume that python2.7 will be available for a few years to come and installable on demands, even if python3 becomes the default on any of those distributions (just like python3 was available when not installed by default).
What you should consider when installing Yocto from source is run it in a virtualenv so that you setup a clean environment, that might be somewhat more work, depending on the dependencies, but ensures a clean working environment for your application, that cannot be broken by any system update of any packages. And if you do that your setup can even use a python2.7.X version different than that supplied by the Linux distribution.
Yocto always runs in a virtualenv. But I’ve found a way to trick it to use python 2 , setting the default python and including it in PATH env variable:
$ source oe-init-build-env build
$ mkdir build/python-bin
$ ln -s /usr/bin/python2 build/python-bin/python
$ ln -s /usr/bin/python2-config build/python-bin/python-config
$ export PATH=$(pwd)/build/python-bin:${PATH}
Thanks all for your help !
The canonical solution here is to use virtualenv to create an environment where “python” is python 2.
You can fix it by overwriting the hosttools symlink yocto creates..
I managed to start the yocto build with the fix from Shan-x but it didn’t build through.
Yocto sources a different env for all recipes.. Some of the recipes, especially from meta-openembedded require hosttools. These hosttools are for example python (which is then expected to be python2).
This hosttools are then symlinked in build/tmp/hosttools and this gets added to $PATH.
python -> /usr/bin/python
to change this to default to python2 just change the symlink to point to /usr/bin/python2
The entire setup:
$ mkdir build/python-bin
$ ln -s /usr/bin/python2 build/python-bin/python
$ ln -s /usr/bin/python2-config build/python-bin/python-config
$ mkdir -p build/tmp/hosttools
$ ln -sf /usr/bin/python2 build/tmp/hosttools/python
to automatically change to python2 add the export $PATH to
sources/poky/oe-init-build-env , just before the other stuff gets sourced:
diff --git a/oe-init-build-env b/oe-init-build-env
index e813230a98..c981358577 100755
--- a/oe-init-build-env
+++ b/oe-init-build-env
@@ -47,6 +47,8 @@ if [ -z "$OEROOT" ]; then
fi
unset THIS_SCRIPT
+export PATH=$(pwd)/build/python-bin:${PATH}
+
export OEROOT
. $OEROOT/scripts/oe-buildenv-internal &&
TEMPLATECONF="$TEMPLATECONF" $OEROOT/scripts/oe-setup-builddir || {
and then source the env:
$ source oe-init-build-env build
I just completed a fresh qemux86-64 (default) build of kirstone on Ubuntu 22.04 by just creating a symlink to python2 and setting up PATH to point to that. For example, from the root of the yocto repo:
ln -s /usr/bin/python2 ./python
export PATH=$PWD:$PATH
Crude but simple and effective.