Seeing escape characters when pressing the arrow keys in python shell
Question:
In shells like the interactive python shell, you can usually use the arrow keys to move around in the current line or get previous commands (with arrow-up) etc.
But after I ssh into another machine and start python there, I get sessions like:
>>> import os
>>> ^[[A
where the last character comes from arrow-up. Or, using arrow-left:
>>> impor^[[D
How can I fix this?
In the regular bash, arrow keys work fine. The weird behavior is just in the interactive python (or perl etc.) shell.
Answers:
Have you tried using a different SSH client? Some SSH clients have special, built-in keymappings for different remote processes. I ran into this one a lot with emacs.
What client are you using? I’d recommend trying Putty and SecureCRT to compare their behavior.
How’s your env variable $TERM set [a] when things work fine and [b] when they don’t? Env settings are often the key to such problems.
Did you call ssh with the -t parameter to tell ssh to allocate a virtual terminal for you?
From the man page:
-t
Force pseudo-tty allocation. This can be used to execute arbitrary screen-based programs on a remote machine, which can be very useful, e.g. when implementing menu services. Multiple -t options force tty allocation, even if ssh has no local tty.
Additionally you may also have to set the TERM environment variable on the server correctly as suggested in another post.
Looks like readline is not enabled. Check if PYTHONSTARTUP variable is defined, for me it points to /etc/pythonstart and that file is executed by the python process before going interactive, which setups readline/history handling.
Thanks to @chown here is the docs on this: http://docs.python.org/2/tutorial/interactive.html
- install readline-devel package.
- recompile python with readline module
- Bingo!
Try getting a key code library running on the server. If that does not work try to download a library with read-key ability.
Here are the steps which worked for me in ubuntu 12.04 for python 3.3.
1) open teminal and write sudo apt-get install libreadline-dev
2) download the source file of python 3.3.2 from http://www.python.org/ftp/python/3.3.2/Python-3.3.2.tar.xz
3) extract it and navigate to the Python-3.3.2/ directory in a shell
4) execute the following command:
./configure
make
make test
sudo make install
I’ve solved this issue by installing readline package:
pip install readline
On CentOS, I fix this by
yum install readline-devel
and then recompile python 3.4.
On OpenSUSE, I fix this by
pip3 install readline
following Valerio Crini’s answer.
Perhaps “pip3 install readline” is a general solution. Haven’t tried on my CentOS.
I was trying build Python 2.7 on Ubuntu 14.0. You will need libreadline-dev. However, if you get it from apt-get, the current version is 6.3, which is incompatible with Python 2.7 (not sure about Python 3). For example, the data type “Function” and “CPPFunction”, which were defined in previous versions of readline has been removed in 6.3, as reported here:
https://github.com/yyuu/pyenv/issues/126
That is to say you need to get the source code of an earlier version of readline. I installed libreadline 5.2 from apt-get for the library, and get the source code of 5.2 for the header files. Put them in /usr/include.
Finally the issue has been resolved.
I fixed this by doing the following:
- yum install readline-devel
-
pip install readline
-
I encountered another error here:
gcc: readline/libreadline.a: No such file or directory
gcc: readline/libhistory.a: No such file or directory
I fixed this by installing patch:
yum install patch
After that I managed to run pip install readline successfully which solved the escape characters in my python shell.
FYI, I’m using RedHat
On OS X, Xcode updates sometimes break readline. Solution:
brew uninstall readline
brew upgrade python3
brew install readline
pip3 install readline
If the problem still persists, try to remove readline using pip and install it using easy_install:
pip3 uninstall readline
easy_install readline
On OS X, I have different problem.
When I using system python shell, the keys is no problem, but problem in virtualenv. I’d try to reinstall/upgrade virtualenv/readline and nothing fixed.
While I try to import readline in problem python shell, get this error message:
ImportError: dlopen(/Users/raptor/.virtualenvs/bottle/lib/python2.7/lib-dynload/readline.so, 2): Library not loaded: /usr/local/opt/readline/lib/libreadline.6.dylib
Referenced from: /Users/raptor/.virtualenvs/bottle/lib/python2.7/lib-dynload/readline.so
Reason: image not found
Cause there is /usr/local/opt/readline/lib/libreadline.7.dylib but not libreadline.6.dylib, so I make a symbol link:
ln -s libreadline.7.dylib libreadline.6.dylib
Problem has been solved!
If you use Anaconda Python, you can fix this by running:
conda install readline
Worked for me!
On OS X, using python 3.5 and virtualenv
$ pip install gnureadline
In the interpreter do:
import gnureadline
Now arrow keys should work properly.
Additional information…
Note that as of Oct 1, 2015 – readline has been DEPRECATED (source https://github.com/ludwigschwardt/python-readline)
Use gnureadline instead (see: https://github.com/ludwigschwardt/python-gnureadline)
If I install readline instead of gnureadline using python 3.5, I receive errors after attempt to import in the interpreter:
>>> import readline
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: dlopen(/Users/pi/tmp/python-readline-test/.venv/lib/python3.5/readline.so, 2): Library not loaded: /usr/local/opt/readline/lib/libreadline.6.dylib
Referenced from: /Users/pi/tmp/python-readline-test/.venv/lib/python3.5/readline.so
Reason: image not found
For those using conda, installing the readline package from conda-forge channel will fix the problem:
conda install -c conda-forge readline=6.2
Was impacted after upgrading Mac to High Sierra, this successfully resolved it for me:
brew unlink python
xcode-select --install
brew install python
I had problems with shell history(tab/arrows commands) of Python 3.6.x on Ubuntu 16.04 LTS.
Python 3.6.x was installed from source.
What solved for me was install the module “gnureadline” as said by user12345, using this command line:
sudo pip3.6 install gnureadline
🙂
readline module has been deprecated which will cause invalid pointer error in latest python versions when executing quit() or exit() in python shell.
pip install gnureadline instead
I have run into this issue recently and after reading a lot about pip install readline (does not work for mac osx) and pip install gnureadline and not being satisfied, this is now my setup which enables using arrow keys in any python console:
- install gnureadline using
pip install gnureadline
now you can either do import gnureadline and arrow keys should work as expected. To make them work automatically follow the following steps:
- create (or append to) file
~/.startup.py: import gnureadline
- append to file
~/.bash_profile: export PYTHONSTARTUP=~/.startup.py
One thing that does not work, but did in my previous setup is: automatic import of gnureadline on pdb.set_trace(). If anyone has a good solution to this problem I would be grateful for a comment.
On Mac OS X Mojave 10.14.6 with various historical installs via brew I solved this with:
brew reinstall python2
There is likely no magic bullet given everyone has a different install scenario. I tried the above as well so it may have been a combination of a few of the answers. Brew defaults to python3 so if you installed the python2 package it also needs to be reinstalled.
On MacOsx, I fixed this by reinstalling readline
brew reinstall readline
None of these answers worked for me on two different version of Ubuntu. What worked for me, but isn’t a true fix, is wrapping my python code in a call to rlwrap (available in the ubuntu repositories):
rlwrap python mycode.py
In Unbuntu or Mint,
if you are using pyenv,
sudo apt install libreadline-dev
pyenv uninstall 3.8.8
pyenv install 3.8.8
Once installing libreadline-dev, you don’t need to install pip install gnureadline on every python version.
you can switch from ‘sh" to "bash" by
$ /sh/bash
In shells like the interactive python shell, you can usually use the arrow keys to move around in the current line or get previous commands (with arrow-up) etc.
But after I ssh into another machine and start python there, I get sessions like:
>>> import os
>>> ^[[A
where the last character comes from arrow-up. Or, using arrow-left:
>>> impor^[[D
How can I fix this?
In the regular bash, arrow keys work fine. The weird behavior is just in the interactive python (or perl etc.) shell.
Have you tried using a different SSH client? Some SSH clients have special, built-in keymappings for different remote processes. I ran into this one a lot with emacs.
What client are you using? I’d recommend trying Putty and SecureCRT to compare their behavior.
How’s your env variable $TERM set [a] when things work fine and [b] when they don’t? Env settings are often the key to such problems.
Did you call ssh with the -t parameter to tell ssh to allocate a virtual terminal for you?
From the man page:
-t
Force pseudo-tty allocation. This can be used to execute arbitrary screen-based programs on a remote machine, which can be very useful, e.g. when implementing menu services. Multiple -t options force tty allocation, even if ssh has no local tty.
Additionally you may also have to set the TERM environment variable on the server correctly as suggested in another post.
Looks like readline is not enabled. Check if PYTHONSTARTUP variable is defined, for me it points to /etc/pythonstart and that file is executed by the python process before going interactive, which setups readline/history handling.
Thanks to @chown here is the docs on this: http://docs.python.org/2/tutorial/interactive.html
- install readline-devel package.
- recompile python with readline module
- Bingo!
Try getting a key code library running on the server. If that does not work try to download a library with read-key ability.
Here are the steps which worked for me in ubuntu 12.04 for python 3.3.
1) open teminal and write sudo apt-get install libreadline-dev
2) download the source file of python 3.3.2 from http://www.python.org/ftp/python/3.3.2/Python-3.3.2.tar.xz
3) extract it and navigate to the Python-3.3.2/ directory in a shell
4) execute the following command:
./configure
make
make test
sudo make install
I’ve solved this issue by installing readline package:
pip install readline
On CentOS, I fix this by
yum install readline-devel
and then recompile python 3.4.
On OpenSUSE, I fix this by
pip3 install readline
following Valerio Crini’s answer.
Perhaps “pip3 install readline” is a general solution. Haven’t tried on my CentOS.
I was trying build Python 2.7 on Ubuntu 14.0. You will need libreadline-dev. However, if you get it from apt-get, the current version is 6.3, which is incompatible with Python 2.7 (not sure about Python 3). For example, the data type “Function” and “CPPFunction”, which were defined in previous versions of readline has been removed in 6.3, as reported here:
https://github.com/yyuu/pyenv/issues/126
That is to say you need to get the source code of an earlier version of readline. I installed libreadline 5.2 from apt-get for the library, and get the source code of 5.2 for the header files. Put them in /usr/include.
Finally the issue has been resolved.
I fixed this by doing the following:
- yum install readline-devel
-
pip install readline
-
I encountered another error here:
gcc: readline/libreadline.a: No such file or directorygcc: readline/libhistory.a: No such file or directoryI fixed this by installing
patch:yum install patch
-
After that I managed to run pip install readline successfully which solved the escape characters in my python shell.
FYI, I’m using RedHat
On OS X, Xcode updates sometimes break readline. Solution:
brew uninstall readline
brew upgrade python3
brew install readline
pip3 install readline
If the problem still persists, try to remove readline using pip and install it using easy_install:
pip3 uninstall readline
easy_install readline
On OS X, I have different problem.
When I using system python shell, the keys is no problem, but problem in virtualenv. I’d try to reinstall/upgrade virtualenv/readline and nothing fixed.
While I try to import readline in problem python shell, get this error message:
ImportError: dlopen(/Users/raptor/.virtualenvs/bottle/lib/python2.7/lib-dynload/readline.so, 2): Library not loaded: /usr/local/opt/readline/lib/libreadline.6.dylib
Referenced from: /Users/raptor/.virtualenvs/bottle/lib/python2.7/lib-dynload/readline.so
Reason: image not found
Cause there is /usr/local/opt/readline/lib/libreadline.7.dylib but not libreadline.6.dylib, so I make a symbol link:
ln -s libreadline.7.dylib libreadline.6.dylib
Problem has been solved!
If you use Anaconda Python, you can fix this by running:
conda install readline
Worked for me!
On OS X, using python 3.5 and virtualenv
$ pip install gnureadline
In the interpreter do:
import gnureadline
Now arrow keys should work properly.
Additional information…
Note that as of Oct 1, 2015 – readline has been DEPRECATED (source https://github.com/ludwigschwardt/python-readline)
Use gnureadline instead (see: https://github.com/ludwigschwardt/python-gnureadline)
If I install readline instead of gnureadline using python 3.5, I receive errors after attempt to import in the interpreter:
>>> import readline
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: dlopen(/Users/pi/tmp/python-readline-test/.venv/lib/python3.5/readline.so, 2): Library not loaded: /usr/local/opt/readline/lib/libreadline.6.dylib
Referenced from: /Users/pi/tmp/python-readline-test/.venv/lib/python3.5/readline.so
Reason: image not found
For those using conda, installing the readline package from conda-forge channel will fix the problem:
conda install -c conda-forge readline=6.2
Was impacted after upgrading Mac to High Sierra, this successfully resolved it for me:
brew unlink python
xcode-select --install
brew install python
I had problems with shell history(tab/arrows commands) of Python 3.6.x on Ubuntu 16.04 LTS.
Python 3.6.x was installed from source.
What solved for me was install the module “gnureadline” as said by user12345, using this command line:
sudo pip3.6 install gnureadline
🙂
readline module has been deprecated which will cause invalid pointer error in latest python versions when executing quit() or exit() in python shell.
pip install gnureadline instead
I have run into this issue recently and after reading a lot about pip install readline (does not work for mac osx) and pip install gnureadline and not being satisfied, this is now my setup which enables using arrow keys in any python console:
- install gnureadline using
pip install gnureadline
now you can either do import gnureadline and arrow keys should work as expected. To make them work automatically follow the following steps:
- create (or append to) file
~/.startup.py:import gnureadline - append to file
~/.bash_profile:export PYTHONSTARTUP=~/.startup.py
One thing that does not work, but did in my previous setup is: automatic import of gnureadline on pdb.set_trace(). If anyone has a good solution to this problem I would be grateful for a comment.
On Mac OS X Mojave 10.14.6 with various historical installs via brew I solved this with:
brew reinstall python2
There is likely no magic bullet given everyone has a different install scenario. I tried the above as well so it may have been a combination of a few of the answers. Brew defaults to python3 so if you installed the python2 package it also needs to be reinstalled.
On MacOsx, I fixed this by reinstalling readline
brew reinstall readline
None of these answers worked for me on two different version of Ubuntu. What worked for me, but isn’t a true fix, is wrapping my python code in a call to rlwrap (available in the ubuntu repositories):
rlwrap python mycode.py
In Unbuntu or Mint,
if you are using pyenv,
sudo apt install libreadline-dev
pyenv uninstall 3.8.8
pyenv install 3.8.8
Once installing libreadline-dev, you don’t need to install pip install gnureadline on every python version.
you can switch from ‘sh" to "bash" by
$ /sh/bash