Interactive Python: cannot get `%lprun` to work, although line_profiler is imported properly
Question:
Problem
Most iPython “magic functions” work fine for me right off the bat: %hist, %time, %prun, etc. However, I noticed that %lprun could not be found with iPython as I’d installed it originally.
Attempt to Resolve
I then discovered that I should install the line_profiler module. I have installed this module, but still cannot seem to get the magic function to work correctly. If I attempt to call %lprun, iPython still cannot find the function. If I call it with the full name ( line_profiler.magic_lprun ), the function can be found, but I cannot get it to work at all. Below is an example of what I’d done (which is taken step by step from “Python for Data Analysis” book):
Success Using %prun
[In:]
def add_and_sum(x, y):
added = x + y
summed = added.sum(axis=1)
return summed
x = randn(3000, 3000)
y = randn(3000, 3000)
add_and_sum(x, y)
With this I get a nice answer, as expected:
[Out:]
array([-23.6223074 , -10.08590736, -31.2957222 , ..., -14.17271747,
63.84057725, -50.28469621])
And I can do the profiling magic function %prun:
[In:]
%prun add_and_sum(x, y)
[Out:]
6 function calls in 0.042 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.020 0.020 0.029 0.029 <ipython-input-27-19f64f63ba0a>:1(add_and_sum)
1 0.013 0.013 0.042 0.042 <string>:1(<module>)
1 0.009 0.009 0.009 0.009 {method 'reduce' of 'numpy.ufunc' objects}
1 0.000 0.000 0.009 0.009 _methods.py:16(_sum)
1 0.000 0.000 0.009 0.009 {method 'sum' of 'numpy.ndarray' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
Fail Using %lprun
But when I try %lprun, I cannot get anything:
[In:]
%lprun -f add_and_sum add_and_sum(x, y)
[Out:]
ERROR: Line magic function `%lprun` not found.
And if I try to call the function with its standard name, it also does not work:
[In:]
line_profiler.magic_lprun -f add_and_sum.test test.add_and_sum(x, y)
[Out:]
line_profiler.magic_lprun -f add_and_sum.test test.add_and_sum(x, y)
^
SyntaxError: invalid syntax
But the library has been imported properly, or at least this is what it says:
[In:]
line_profiler
[Out:]
<module 'line_profiler' from '/Users/<edit>/anaconda/lib/python2.7/site-packages/line_profiler-1.0b3-py2.7-macosx-10.5-x86_64.egg/line_profiler.pyc'>
[In:]
line_profiler.magic_lprun
[Out:]
<function line_profiler.magic_lprun>
It seems as if there is something extra that I am supposed to configure so that these new magic functions that I add can be identified as such. I could not find anything via a web search.
I am running Spyder as an IDE (still using iPython as the console), but I have also tried it directly with iPython and with iPython notebook. I have had no luck in any format.
Answers:
To make %lprun work, you need to load the extension into your session, using this command:
In [1]: %load_ext line_profiler
Check out this notebook to see some examples that use the magic.
Besides, if you are working with Spyder, there is also a third-party line_profiler plugin, which you can find here.
You have two ways to make work %lprun, one solution is temporal, i.e., it lasts until you finish your ipython session, and the other one is permanent.
Temporal: (as Carlos Cordoba’s answer)
After starting ipython run the following:
In [1]: %load_ext line_profiler
Permanent:
Add the following lines to ~/.ipython/profile_default/ipython_config.py:
c.TerminalIPythonApp.extensions = [
'line_profiler',
]
If you don’t have the file ~/.ipython/profile_default/ipython_config.py, you can create it by (see this for more info):
ipython profile create
Problem
Most iPython “magic functions” work fine for me right off the bat: %hist, %time, %prun, etc. However, I noticed that %lprun could not be found with iPython as I’d installed it originally.
Attempt to Resolve
I then discovered that I should install the line_profiler module. I have installed this module, but still cannot seem to get the magic function to work correctly. If I attempt to call %lprun, iPython still cannot find the function. If I call it with the full name ( line_profiler.magic_lprun ), the function can be found, but I cannot get it to work at all. Below is an example of what I’d done (which is taken step by step from “Python for Data Analysis” book):
Success Using %prun
[In:]
def add_and_sum(x, y):
added = x + y
summed = added.sum(axis=1)
return summed
x = randn(3000, 3000)
y = randn(3000, 3000)
add_and_sum(x, y)
With this I get a nice answer, as expected:
[Out:]
array([-23.6223074 , -10.08590736, -31.2957222 , ..., -14.17271747,
63.84057725, -50.28469621])
And I can do the profiling magic function %prun:
[In:]
%prun add_and_sum(x, y)
[Out:]
6 function calls in 0.042 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.020 0.020 0.029 0.029 <ipython-input-27-19f64f63ba0a>:1(add_and_sum)
1 0.013 0.013 0.042 0.042 <string>:1(<module>)
1 0.009 0.009 0.009 0.009 {method 'reduce' of 'numpy.ufunc' objects}
1 0.000 0.000 0.009 0.009 _methods.py:16(_sum)
1 0.000 0.000 0.009 0.009 {method 'sum' of 'numpy.ndarray' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
Fail Using %lprun
But when I try %lprun, I cannot get anything:
[In:]
%lprun -f add_and_sum add_and_sum(x, y)
[Out:]
ERROR: Line magic function `%lprun` not found.
And if I try to call the function with its standard name, it also does not work:
[In:]
line_profiler.magic_lprun -f add_and_sum.test test.add_and_sum(x, y)
[Out:]
line_profiler.magic_lprun -f add_and_sum.test test.add_and_sum(x, y)
^
SyntaxError: invalid syntax
But the library has been imported properly, or at least this is what it says:
[In:]
line_profiler
[Out:]
<module 'line_profiler' from '/Users/<edit>/anaconda/lib/python2.7/site-packages/line_profiler-1.0b3-py2.7-macosx-10.5-x86_64.egg/line_profiler.pyc'>
[In:]
line_profiler.magic_lprun
[Out:]
<function line_profiler.magic_lprun>
It seems as if there is something extra that I am supposed to configure so that these new magic functions that I add can be identified as such. I could not find anything via a web search.
I am running Spyder as an IDE (still using iPython as the console), but I have also tried it directly with iPython and with iPython notebook. I have had no luck in any format.
To make %lprun work, you need to load the extension into your session, using this command:
In [1]: %load_ext line_profiler
Check out this notebook to see some examples that use the magic.
Besides, if you are working with Spyder, there is also a third-party line_profiler plugin, which you can find here.
You have two ways to make work %lprun, one solution is temporal, i.e., it lasts until you finish your ipython session, and the other one is permanent.
Temporal: (as Carlos Cordoba’s answer)
After starting ipython run the following:
In [1]: %load_ext line_profiler
Permanent:
Add the following lines to ~/.ipython/profile_default/ipython_config.py:
c.TerminalIPythonApp.extensions = [
'line_profiler',
]
If you don’t have the file ~/.ipython/profile_default/ipython_config.py, you can create it by (see this for more info):
ipython profile create