How to run an IPython magic from a script (or timing a Python script)
Question:
The IPython %timeit magic command does its job well for measuring time required to run some Python code. Now, I want to use something analogous in the Python script. I know about the timeit module, however, it has several disadvantages, for example, how to select the number of runs adaptively? i.e., the default code
import timeit
t=timeit.Timer("code(f)", "from __main__ import code,f")
t.timeit()
runs the code million times. The %timeit IPyhton magic command does it automatically. I suggest that I could use something like the MATLAB code
http://www.mathworks.com/matlabcentral/fileexchange/18798
that does all the job automatically (and also tells if the overhead of the function is large).
How can I call %timeit magic from a Python script (or maybe there is a better timing solution) ?
Answers:
Both IPython and the timeit module, when called with python -m timeit, execute the same loop with a growing value of number until the timing result surpasses a certain threshold that guarantees the time measurement is mostly free of operating system interferences.
You can compare the IPython implementation of the %timeit magic with the Python timeit standard module to see that they are doing mostly the same.
So, answering your question, you should probably replicate the same loop until you find the correct value for the number parameter.
It depends a bit on which version of IPython you have. If you have 1.x:
from IPython import get_ipython
ipython = get_ipython()
If you have an older version:
import IPython.core.ipapi
ipython = IPython.core.ipapi.get()
or
import IPython.ipapi
ipython = IPython.ipapi.get()
Once that’s done, run a magic command like this:
ipython.magic("timeit abs(-42)")
Note that the script must be run via ipython.
The following works if one runs the Python script interactively in IPython. E.g., test.py:
def f():
# Algorithm 1
pass
def g():
# Algorithm 2
pass
# which one is faster?
mgc = get_ipython().magic
mgc(u'%timeit f()')
mgc(u'%timeit g()')
Then run it interactively in IPython
%run -i test.py
to spit out the timings. The -i switch is necessary so that the variables are in scope. I have not figured out how to do this without running an IPython instance, i.e., by simply importing timeit from IPython and using it as a function. However, this solution works for my purposes, which is to automate some timing runs.
One way to run ipython magic function probably is using embed ipython instance.
For example: (most of the codes are borrowed from ipython website)
from IPython.terminal.embed import InteractiveShellEmbed
ipshell = InteractiveShellEmbed()
ipshell.dummy_mode = True
print('nTrying to call IPython which is now "dummy":')
ipshell.magic("%timeit abs(-42)");
ipshell()
print('Nothing happened...')
This can work by using python interpreter
PS: using dummy_mode will prevent from invoking interactive shell.
According to the documentation of the timeit.py module, when timeit is run in command-line mode,
If -n is not given, a suitable number of loops is calculated by trying successive powers of 10 until the total time is at least 0.2 seconds.
This is what IPython does. That is why the number of loops is always a power of 10. You could do something similar in your own code by embedding the call to t.timeit() inside a loop that makes sure you don’t wait too long:
import timeit
t = timeit.Timer("code(f)", "from __main__ import code, f")
max_time = 0.2
N = 0
curr_time = t.timeit(1)
while curr_time < max_time:
N += 1
curr_time = t.timeit(10**N)
mean_time = curr_time / float(10**N)
This would ensure that the profiling time is at least 0.2 seconds, but not significantly more — unless calling the function once takes a long time.
The IPython %timeit magic command does its job well for measuring time required to run some Python code. Now, I want to use something analogous in the Python script. I know about the timeit module, however, it has several disadvantages, for example, how to select the number of runs adaptively? i.e., the default code
import timeit
t=timeit.Timer("code(f)", "from __main__ import code,f")
t.timeit()
runs the code million times. The %timeit IPyhton magic command does it automatically. I suggest that I could use something like the MATLAB code
http://www.mathworks.com/matlabcentral/fileexchange/18798
that does all the job automatically (and also tells if the overhead of the function is large).
How can I call %timeit magic from a Python script (or maybe there is a better timing solution) ?
Both IPython and the timeit module, when called with python -m timeit, execute the same loop with a growing value of number until the timing result surpasses a certain threshold that guarantees the time measurement is mostly free of operating system interferences.
You can compare the IPython implementation of the %timeit magic with the Python timeit standard module to see that they are doing mostly the same.
So, answering your question, you should probably replicate the same loop until you find the correct value for the number parameter.
It depends a bit on which version of IPython you have. If you have 1.x:
from IPython import get_ipython
ipython = get_ipython()
If you have an older version:
import IPython.core.ipapi
ipython = IPython.core.ipapi.get()
or
import IPython.ipapi
ipython = IPython.ipapi.get()
Once that’s done, run a magic command like this:
ipython.magic("timeit abs(-42)")
Note that the script must be run via ipython.
The following works if one runs the Python script interactively in IPython. E.g., test.py:
def f():
# Algorithm 1
pass
def g():
# Algorithm 2
pass
# which one is faster?
mgc = get_ipython().magic
mgc(u'%timeit f()')
mgc(u'%timeit g()')
Then run it interactively in IPython
%run -i test.py
to spit out the timings. The -i switch is necessary so that the variables are in scope. I have not figured out how to do this without running an IPython instance, i.e., by simply importing timeit from IPython and using it as a function. However, this solution works for my purposes, which is to automate some timing runs.
One way to run ipython magic function probably is using embed ipython instance.
For example: (most of the codes are borrowed from ipython website)
from IPython.terminal.embed import InteractiveShellEmbed
ipshell = InteractiveShellEmbed()
ipshell.dummy_mode = True
print('nTrying to call IPython which is now "dummy":')
ipshell.magic("%timeit abs(-42)");
ipshell()
print('Nothing happened...')
This can work by using python interpreter
PS: using dummy_mode will prevent from invoking interactive shell.
According to the documentation of the timeit.py module, when timeit is run in command-line mode,
If -n is not given, a suitable number of loops is calculated by trying successive powers of 10 until the total time is at least 0.2 seconds.
This is what IPython does. That is why the number of loops is always a power of 10. You could do something similar in your own code by embedding the call to t.timeit() inside a loop that makes sure you don’t wait too long:
import timeit
t = timeit.Timer("code(f)", "from __main__ import code, f")
max_time = 0.2
N = 0
curr_time = t.timeit(1)
while curr_time < max_time:
N += 1
curr_time = t.timeit(10**N)
mean_time = curr_time / float(10**N)
This would ensure that the profiling time is at least 0.2 seconds, but not significantly more — unless calling the function once takes a long time.