Get time of execution of a block of code in Python 2.7
Question:
I would like to measure the time elapsed to evaluate a block of code in a Python program,
possibly separating between user cpu time, system cpu time and elapsed time.
I know the timeit module, but I have many self-written functions and it is not very easy
to pass them in the setup process.
I would rather have something that could be used like:
#up to here I have done something....
start_counting() #or whatever command used to mark that I want to measure
#the time elapsed in the next rows
# code I want to evaluate
user,system,elapsed = stop_counting() #or whatever command says:
#stop the timer and return the times
The user and system CPU times are not essential (though I would like to measure them),
but for the elapsed time I would like to be able to do something like this,
rather than using complicated commands or modules.
Answers:
To get the elapsed time in seconds, you can use timeit.default_timer():
import timeit
start_time = timeit.default_timer()
# code you want to evaluate
elapsed = timeit.default_timer() - start_time
timeit.default_timer() is used instead of time.time() or time.clock() because it will choose the timing function that has the higher resolution for any platform.
I always use a decorator to do some extra work for a existing function, including to get the execution time. It is pythonic and simple.
import time
def time_usage(func):
def wrapper(*args, **kwargs):
beg_ts = time.time()
retval = func(*args, **kwargs)
end_ts = time.time()
print("elapsed time: %f" % (end_ts - beg_ts))
return retval
return wrapper
@time_usage
def test():
for i in xrange(0, 10000):
pass
if __name__ == "__main__":
test()
You can achieve this through the Context Manager, for example:
from contextlib import contextmanager
import time
import logging
@contextmanager
def _log_time_usage(prefix=""):
'''log the time usage in a code block
prefix: the prefix text to show
'''
start = time.time()
try:
yield
finally:
end = time.time()
elapsed_seconds = float("%.2f" % (end - start))
logging.debug('%s: elapsed seconds: %s', prefix, elapsed_seconds)
use example:
with _log_time_usage("sleep 1: "):
time.sleep(1)
I found myself solving this problem again and again, so I finally created a library for it. Install with pip install timer_cm. Then:
from time import sleep
from timer_cm import Timer
with Timer('Long task') as timer:
with timer.child('First step'):
sleep(1)
for _ in range(5):
with timer.child('Baby steps'):
sleep(.5)
Output:
Long task: 3.520s
Baby steps: 2.518s (71%)
First step: 1.001s (28%)
There is one more option which i loves a lot now for simplicity – ipython. In ipython you got a lot of useful stuff plus:
%time <expression> – to get straight cpu and wall time on expression
%timeit <expression> – to get cpu and wall time in a loop of expression
Python 3 – Simple solution using standard library
Option 1: Triple quote the code
import inspect
import timeit
code_block = inspect.cleandoc("""
base = 123456789
exponent = 100
return base ** exponent
""")
print(f'Code block: {timeit.timeit(code_block, number=1, globals=globals())} elapsed seconds')
inspect.cleandoc handles the removal of extra tabs and whitespace so that blocks of code can be copied and pasted without getting indentation errors.
Option 2: Place code block in a function
import timeit
def my_function():
base = 123456789
exponent = 100
return base ** exponent
if __name__ == '__main__':
print(f'With lambda wrapper: {timeit.timeit(lambda: my_function(), number=1)} elapsed seconds')
Note that a function call will add additional execution time versus timing the function body directly.
I would like to measure the time elapsed to evaluate a block of code in a Python program,
possibly separating between user cpu time, system cpu time and elapsed time.
I know the timeit module, but I have many self-written functions and it is not very easy
to pass them in the setup process.
I would rather have something that could be used like:
#up to here I have done something....
start_counting() #or whatever command used to mark that I want to measure
#the time elapsed in the next rows
# code I want to evaluate
user,system,elapsed = stop_counting() #or whatever command says:
#stop the timer and return the times
The user and system CPU times are not essential (though I would like to measure them),
but for the elapsed time I would like to be able to do something like this,
rather than using complicated commands or modules.
To get the elapsed time in seconds, you can use timeit.default_timer():
import timeit
start_time = timeit.default_timer()
# code you want to evaluate
elapsed = timeit.default_timer() - start_time
timeit.default_timer() is used instead of time.time() or time.clock() because it will choose the timing function that has the higher resolution for any platform.
I always use a decorator to do some extra work for a existing function, including to get the execution time. It is pythonic and simple.
import time
def time_usage(func):
def wrapper(*args, **kwargs):
beg_ts = time.time()
retval = func(*args, **kwargs)
end_ts = time.time()
print("elapsed time: %f" % (end_ts - beg_ts))
return retval
return wrapper
@time_usage
def test():
for i in xrange(0, 10000):
pass
if __name__ == "__main__":
test()
You can achieve this through the Context Manager, for example:
from contextlib import contextmanager
import time
import logging
@contextmanager
def _log_time_usage(prefix=""):
'''log the time usage in a code block
prefix: the prefix text to show
'''
start = time.time()
try:
yield
finally:
end = time.time()
elapsed_seconds = float("%.2f" % (end - start))
logging.debug('%s: elapsed seconds: %s', prefix, elapsed_seconds)
use example:
with _log_time_usage("sleep 1: "):
time.sleep(1)
I found myself solving this problem again and again, so I finally created a library for it. Install with pip install timer_cm. Then:
from time import sleep
from timer_cm import Timer
with Timer('Long task') as timer:
with timer.child('First step'):
sleep(1)
for _ in range(5):
with timer.child('Baby steps'):
sleep(.5)
Output:
Long task: 3.520s
Baby steps: 2.518s (71%)
First step: 1.001s (28%)
There is one more option which i loves a lot now for simplicity – ipython. In ipython you got a lot of useful stuff plus:
%time <expression> – to get straight cpu and wall time on expression
%timeit <expression> – to get cpu and wall time in a loop of expression
Python 3 – Simple solution using standard library
Option 1: Triple quote the code
import inspect
import timeit
code_block = inspect.cleandoc("""
base = 123456789
exponent = 100
return base ** exponent
""")
print(f'Code block: {timeit.timeit(code_block, number=1, globals=globals())} elapsed seconds')
inspect.cleandoc handles the removal of extra tabs and whitespace so that blocks of code can be copied and pasted without getting indentation errors.
Option 2: Place code block in a function
import timeit
def my_function():
base = 123456789
exponent = 100
return base ** exponent
if __name__ == '__main__':
print(f'With lambda wrapper: {timeit.timeit(lambda: my_function(), number=1)} elapsed seconds')
Note that a function call will add additional execution time versus timing the function body directly.