How to Copy from IPython session without terminal prompts
Question:
Frequently, my workflow involves data cleaning/munging in an IPython shell. This has become particularly wonderful since IPython version 5.0 with all the great upgrades to the terminal interface. So, let’s say I make an attempt at sprucing up some piece of unstructured data:
In [11]: for i, (num, header, txt) in enumerate(data):
...: header = [e.strip() for e in header.strip().split('n')]
...: header[4] = header[4].strip(',').split(',')
...: data[i] = (num, header, txt)
...:
Fantastic, it works! But now, I would really like to add this to a script in my editor. If I copy and paste from my terminal, I capture all the junk on the left. I can clean this up more-or-less easily in an editor, but it would be great if I could copy the code directly to my clipboard from the terminal without touching the mouse and without grabbing the extra stuff either. Is there such a functionality in IPython?
Answers:
I don’t think terminal applications really get access to the copy/paste buffer. You’re going to have to use the mouse. How do do it depends on what terminal you’re using. Most modern terminals have some sort of “rectangular select” or “block select” mode.
With Windows, rectangular select is the default for cmd.exe and Powershell. If you’re using Cygwin’s mintty, hold Alt and then select the region with the mouse. The same goes for PuTTY.
On Linux (which I don’t have in front of me – take these with a grain of salt), xterm doesn’t support it, Gnome Terminal uses Ctrl as the modifier, and KDE’s Konsole uses Ctrl+Alt.
For OS X Terminal, the Internet tells me that you use ⌘ while clicking.
Other terminals (and GNU Screen) likely have the feature, it’s just a matter of figuring out how to activate it.
You can use the %history magic to extract the interesting parts from your session. They will be shown in terminal without any of the junk.
Example
In [1]: import numpy as np
In [2]: a = np.random(10)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-2-83ce219ad17b> in <module>()
----> 1 a = np.random(10)
TypeError: 'module' object is not callable
In [3]: a = np.random.random(10)
In [4]: for i in a:
...: print(i)
...:
0.688626523886
[...]
0.341394850998
If I want to save a part of the session above I can use:
In [5]: %history 1 3-4
import numpy as np
a = np.random.random(10)
for i in a:
print(i)
In the example above I used %history 1 3-4 to assemble all the commands I want to keep and omit the ones I do not need (Line 2, the one with the error). Now you have version of your session that can be nicely copied.
Writing a file
You can also directly write this to file using the -f FILENAME as parameter.
In [8]: %history 1 3-4 -f /tmp/foo.py
Be careful though, this will overwrite existing files.
More Details can be found in the documentation of the %history magic.
In the shell you can first convert the IPython file to a regular Python file (.py) and then do the clean up:
http://ipython.org/ipython-doc/3/notebook/nbconvert.html (see –to script format)
You can also download the file in the notebook editor as Python file and perform the cleanup after this step.
So, I have finally found a great solution that is essentially exactly what I wanted: Use Vi mode in IPython. On version 5, this requires:
$ ipython --TerminalInteractiveShell.editing_mode=vi
Now I can use handy vi-like visual mode and yank whatever I need!
Which leads to the following new alias in my .bash_profile/.bash_rc:
alias vpython='ipython --TerminalInteractiveShell.editing_mode=vi'
The save magic command [documentation] saves the input lines you want to a file; the -a option is for “append” mode so that the lines are added at the end of the file instead of overwriting the file. I use it all the time.
With your example:
%save -a myfile.py 11
# the '%' is not necessary
save -a myfile.py 11
Then you can keep coding in IPython.
When there is another command you want to write to the same file, you can just type save then use the up arrow to bring back the last use of “save” (so that the -a option and the filename are already there) and just edit the line number.
Note that you can give several lines to save and also line ranges:
save -a myfile.py 15 18 19-25
This is a bit of a hack, but you can use these keystrokes:
ggIprint("""<Esc>GA""")
This will make the contents of the entire cell into a multiline string, then print it. As long as the number of lines isn’t massive, this is usually convenient enough.
It may also be possible to convert it into a keybinding/macro, though I haven’t investigated this.
Frequently, my workflow involves data cleaning/munging in an IPython shell. This has become particularly wonderful since IPython version 5.0 with all the great upgrades to the terminal interface. So, let’s say I make an attempt at sprucing up some piece of unstructured data:
In [11]: for i, (num, header, txt) in enumerate(data):
...: header = [e.strip() for e in header.strip().split('n')]
...: header[4] = header[4].strip(',').split(',')
...: data[i] = (num, header, txt)
...:
Fantastic, it works! But now, I would really like to add this to a script in my editor. If I copy and paste from my terminal, I capture all the junk on the left. I can clean this up more-or-less easily in an editor, but it would be great if I could copy the code directly to my clipboard from the terminal without touching the mouse and without grabbing the extra stuff either. Is there such a functionality in IPython?
I don’t think terminal applications really get access to the copy/paste buffer. You’re going to have to use the mouse. How do do it depends on what terminal you’re using. Most modern terminals have some sort of “rectangular select” or “block select” mode.
With Windows, rectangular select is the default for cmd.exe and Powershell. If you’re using Cygwin’s mintty, hold Alt and then select the region with the mouse. The same goes for PuTTY.
On Linux (which I don’t have in front of me – take these with a grain of salt), xterm doesn’t support it, Gnome Terminal uses Ctrl as the modifier, and KDE’s Konsole uses Ctrl+Alt.
For OS X Terminal, the Internet tells me that you use ⌘ while clicking.
Other terminals (and GNU Screen) likely have the feature, it’s just a matter of figuring out how to activate it.
You can use the %history magic to extract the interesting parts from your session. They will be shown in terminal without any of the junk.
Example
In [1]: import numpy as np
In [2]: a = np.random(10)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-2-83ce219ad17b> in <module>()
----> 1 a = np.random(10)
TypeError: 'module' object is not callable
In [3]: a = np.random.random(10)
In [4]: for i in a:
...: print(i)
...:
0.688626523886
[...]
0.341394850998
If I want to save a part of the session above I can use:
In [5]: %history 1 3-4
import numpy as np
a = np.random.random(10)
for i in a:
print(i)
In the example above I used %history 1 3-4 to assemble all the commands I want to keep and omit the ones I do not need (Line 2, the one with the error). Now you have version of your session that can be nicely copied.
Writing a file
You can also directly write this to file using the -f FILENAME as parameter.
In [8]: %history 1 3-4 -f /tmp/foo.py
Be careful though, this will overwrite existing files.
More Details can be found in the documentation of the %history magic.
In the shell you can first convert the IPython file to a regular Python file (.py) and then do the clean up:
http://ipython.org/ipython-doc/3/notebook/nbconvert.html (see –to script format)
You can also download the file in the notebook editor as Python file and perform the cleanup after this step.
So, I have finally found a great solution that is essentially exactly what I wanted: Use Vi mode in IPython. On version 5, this requires:
$ ipython --TerminalInteractiveShell.editing_mode=vi
Now I can use handy vi-like visual mode and yank whatever I need!
Which leads to the following new alias in my .bash_profile/.bash_rc:
alias vpython='ipython --TerminalInteractiveShell.editing_mode=vi'
The save magic command [documentation] saves the input lines you want to a file; the -a option is for “append” mode so that the lines are added at the end of the file instead of overwriting the file. I use it all the time.
With your example:
%save -a myfile.py 11
# the '%' is not necessary
save -a myfile.py 11
Then you can keep coding in IPython.
When there is another command you want to write to the same file, you can just type save then use the up arrow to bring back the last use of “save” (so that the -a option and the filename are already there) and just edit the line number.
Note that you can give several lines to save and also line ranges:
save -a myfile.py 15 18 19-25
This is a bit of a hack, but you can use these keystrokes:
ggIprint("""<Esc>GA""")
This will make the contents of the entire cell into a multiline string, then print it. As long as the number of lines isn’t massive, this is usually convenient enough.
It may also be possible to convert it into a keybinding/macro, though I haven’t investigated this.