How to measure time taken between lines of code in python?
Question:
So in Java, we can do How to measure time taken by a function to execute
But how is it done in python? To measure the time start and end time between lines of code?
Something that does this:
import some_time_library
starttime = some_time_library.some_module()
code_tobe_measured()
endtime = some_time_library.some_module()
time_taken = endtime - starttime
Answers:
If you want to measure CPU time, can use time.process_time()
for Python 3.3 and above:
import time
start = time.process_time()
# your code here
print(time.process_time() - start)
First call turns the timer on, and second call tells you how many seconds have elapsed.
There is also a function time.clock()
, but it is deprecated since Python 3.3 and will be removed in Python 3.8.
There are better profiling tools like timeit
and profile
, however time.process_time() will measure the CPU time and this is what you’re are asking about.
If you want to measure wall clock time instead, use time.time()
.
With a help of a small convenience class, you can measure time spent in indented lines like this:
with CodeTimer():
line_to_measure()
another_line()
# etc...
Which will show the following after the indented line(s) finishes executing:
Code block took: x.xxx ms
UPDATE: You can now get the class with pip install linetimer
and then from linetimer import CodeTimer
. See this GitHub project.
The code for above class:
import timeit
class CodeTimer:
def __init__(self, name=None):
self.name = " '" + name + "'" if name else ''
def __enter__(self):
self.start = timeit.default_timer()
def __exit__(self, exc_type, exc_value, traceback):
self.took = (timeit.default_timer() - self.start) * 1000.0
print('Code block' + self.name + ' took: ' + str(self.took) + ' ms')
You could then name the code blocks you want to measure:
with CodeTimer('loop 1'):
for i in range(100000):
pass
with CodeTimer('loop 2'):
for i in range(100000):
pass
Code block 'loop 1' took: 4.991 ms
Code block 'loop 2' took: 3.666 ms
And nest them:
with CodeTimer('Outer'):
for i in range(100000):
pass
with CodeTimer('Inner'):
for i in range(100000):
pass
for i in range(100000):
pass
Code block 'Inner' took: 2.382 ms
Code block 'Outer' took: 10.466 ms
Regarding timeit.default_timer()
, it uses the best timer based on OS and Python version, see this answer.
You can also use time
library:
import time
start = time.time()
# your code
# end
print(f'Time: {time.time() - start}')
I always prefer to check time in hours, minutes and seconds (%H:%M:%S) format:
from datetime import datetime
start = datetime.now()
# your code
end = datetime.now()
time_taken = end - start
print('Time: ',time_taken)
output:
Time: 0:00:00.000019
I was looking for a way how to output a formatted time with minimal code, so here is my solution. Many people use Pandas anyway, so in some cases this can save from additional library imports.
import pandas as pd
start = pd.Timestamp.now()
# code
print(pd.Timestamp.now()-start)
Output:
0 days 00:05:32.541600
I would recommend using this if time precision is not the most important, otherwise use time
library:
%timeit pd.Timestamp.now()
outputs 3.29 µs ± 214 ns per loop
%timeit time.time()
outputs 154 ns ± 13.3 ns per loop
You can try this as well:
from time import perf_counter
t0 = perf_counter()
...
t1 = perf_counter()
time_taken = t1 - t0
Putting the code in a function, then using a decorator for timing is another option. (Source) The advantage of this method is that you define timer once and use it with a simple additional line for every function.
First, define timer
decorator:
import functools
import time
def timer(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
start_time = time.perf_counter()
value = func(*args, **kwargs)
end_time = time.perf_counter()
run_time = end_time - start_time
print("Finished {} in {} secs".format(repr(func.__name__), round(run_time, 3)))
return value
return wrapper
Then, use the decorator while defining the function:
@timer
def doubled_and_add(num):
res = sum([i*2 for i in range(num)])
print("Result : {}".format(res))
Let’s try:
doubled_and_add(100000)
doubled_and_add(1000000)
Output:
Result : 9999900000
Finished 'doubled_and_add' in 0.0119 secs
Result : 999999000000
Finished 'doubled_and_add' in 0.0897 secs
Note: I’m not sure why to use time.perf_counter
instead of time.time
. Comments are welcome.
Let me add a little more to https://stackoverflow.com/a/63665115/7412781 solution.
- Removed dependency on
functools
.
- Used process time taken
time.process_time()
instead of absolute counter of time.perf_counter()
because the process can be context switched out via kernel.
- Used the raw function pointer print to get the correct class name as well.
This is the decorator code.
import time
def decorator_time_taken(fnc):
def inner(*args):
start = time.process_time()
ret = fnc(*args)
end = time.process_time()
print("{} took {} seconds".format(fnc, round((end - start), 6)))
return ret
return inner
This is the usage sample code. It’s checking if 193939 is prime or not.
class PrimeBrute:
@decorator_time_taken
def isPrime(self, a):
for i in range(a-2):
if a % (i+2) == 0: return False
return True
inst = PrimeBrute()
print(inst.isPrime(193939))
This is the output.
<function PrimeBrute.isPrime at 0x7fc0c6919ae8> took 0.015789 seconds
True
Use timeit
module to benchmark your performance:
def test():
print("test")
emptyFunction()
for i in [x for x in range(10000)]:
i**i
def emptyFunction():
pass
if __name__ == "__main__":
import timeit
print(timeit.timeit("test()", number = 5, globals = globals()))
#print(timeit.timeit("test()", setup = "from __main__ import test",
# number = 5))
the first parameter defines the piece of code which we want to execute test
in this case & number
defines how many times you want to repeat the execution.
Output:
test
test
test
test
test
36.81822113099952
Using the module time
, we can calculate unix time at the start of the function and at the end of a function. Here is how the code might look like:
from time import time as unix
This code imports time.time
which allows us to calculate unix time.
from time import sleep
This is not mandatory, but I am also importing time.sleep
for one of the demonstrations.
START_TIME = unix()
This is what calculates unix time and puts it in a variable. Remember, the function unix is not an actual function. I imported time.time
as unix, so if you did not put as unix
in the first import, you will need to use time.time()
.
After this, we put whichever function or code we want.
At the end of the code snippet we put
TOTAL_TIME = unix()-START_TIME
This line of code does two things: It calculates unix time at the end of the function, and using the variable START_TIME
from before, we calculate the amount of time it took to execute the code snippet.
We can then use this variable wherever we want, including for a print()
function.
print("The snippet took {} seconds to execute".format(TOTAL_TIME))
Here I wrote a quick demonstration code that has two experiments as a demonstration. (Fully commented)
from time import time as unix # Import the module to measure unix time
from time import sleep
# Here are a few examples:
# 1. Counting to 100 000
START_TIME = unix()
for i in range(0, 100001):
print("Number: {}r".format(i), end="")
TOTAL_TIME = unix() - START_TIME
print("nFinal time (Expirement 1): {} sn".format(TOTAL_TIME))
# 2. Precision of sleep
for i in range(10):
START_TIME = unix()
sleep(0.1)
TOTAL_TIME = unix() - START_TIME
print("Sleep(0.1): Index: {}, Time: {} s".format(i,TOTAL_TIME))
Here was my output:
Number: 100000
Final time (Expirement 1): 16.666812419891357 s
Sleep(0.1): Index: 0, Time: 0.10014867782592773 s
Sleep(0.1): Index: 1, Time: 0.10016226768493652 s
Sleep(0.1): Index: 2, Time: 0.10202860832214355 s
Sleep(0.1): Index: 3, Time: 0.10015869140625 s
Sleep(0.1): Index: 4, Time: 0.10014724731445312 s
Sleep(0.1): Index: 5, Time: 0.10013675689697266 s
Sleep(0.1): Index: 6, Time: 0.10014677047729492 s
Sleep(0.1): Index: 7, Time: 0.1001439094543457 s
Sleep(0.1): Index: 8, Time: 0.10044598579406738 s
Sleep(0.1): Index: 9, Time: 0.10014700889587402 s
>
import datetime
#this code before computation
%%timeit
~code~
So in Java, we can do How to measure time taken by a function to execute
But how is it done in python? To measure the time start and end time between lines of code?
Something that does this:
import some_time_library
starttime = some_time_library.some_module()
code_tobe_measured()
endtime = some_time_library.some_module()
time_taken = endtime - starttime
If you want to measure CPU time, can use time.process_time()
for Python 3.3 and above:
import time
start = time.process_time()
# your code here
print(time.process_time() - start)
First call turns the timer on, and second call tells you how many seconds have elapsed.
There is also a function time.clock()
, but it is deprecated since Python 3.3 and will be removed in Python 3.8.
There are better profiling tools like timeit
and profile
, however time.process_time() will measure the CPU time and this is what you’re are asking about.
If you want to measure wall clock time instead, use time.time()
.
With a help of a small convenience class, you can measure time spent in indented lines like this:
with CodeTimer():
line_to_measure()
another_line()
# etc...
Which will show the following after the indented line(s) finishes executing:
Code block took: x.xxx ms
UPDATE: You can now get the class with pip install linetimer
and then from linetimer import CodeTimer
. See this GitHub project.
The code for above class:
import timeit
class CodeTimer:
def __init__(self, name=None):
self.name = " '" + name + "'" if name else ''
def __enter__(self):
self.start = timeit.default_timer()
def __exit__(self, exc_type, exc_value, traceback):
self.took = (timeit.default_timer() - self.start) * 1000.0
print('Code block' + self.name + ' took: ' + str(self.took) + ' ms')
You could then name the code blocks you want to measure:
with CodeTimer('loop 1'):
for i in range(100000):
pass
with CodeTimer('loop 2'):
for i in range(100000):
pass
Code block 'loop 1' took: 4.991 ms
Code block 'loop 2' took: 3.666 ms
And nest them:
with CodeTimer('Outer'):
for i in range(100000):
pass
with CodeTimer('Inner'):
for i in range(100000):
pass
for i in range(100000):
pass
Code block 'Inner' took: 2.382 ms
Code block 'Outer' took: 10.466 ms
Regarding timeit.default_timer()
, it uses the best timer based on OS and Python version, see this answer.
You can also use time
library:
import time
start = time.time()
# your code
# end
print(f'Time: {time.time() - start}')
I always prefer to check time in hours, minutes and seconds (%H:%M:%S) format:
from datetime import datetime
start = datetime.now()
# your code
end = datetime.now()
time_taken = end - start
print('Time: ',time_taken)
output:
Time: 0:00:00.000019
I was looking for a way how to output a formatted time with minimal code, so here is my solution. Many people use Pandas anyway, so in some cases this can save from additional library imports.
import pandas as pd
start = pd.Timestamp.now()
# code
print(pd.Timestamp.now()-start)
Output:
0 days 00:05:32.541600
I would recommend using this if time precision is not the most important, otherwise use time
library:
%timeit pd.Timestamp.now()
outputs 3.29 µs ± 214 ns per loop
%timeit time.time()
outputs 154 ns ± 13.3 ns per loop
You can try this as well:
from time import perf_counter
t0 = perf_counter()
...
t1 = perf_counter()
time_taken = t1 - t0
Putting the code in a function, then using a decorator for timing is another option. (Source) The advantage of this method is that you define timer once and use it with a simple additional line for every function.
First, define timer
decorator:
import functools
import time
def timer(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
start_time = time.perf_counter()
value = func(*args, **kwargs)
end_time = time.perf_counter()
run_time = end_time - start_time
print("Finished {} in {} secs".format(repr(func.__name__), round(run_time, 3)))
return value
return wrapper
Then, use the decorator while defining the function:
@timer
def doubled_and_add(num):
res = sum([i*2 for i in range(num)])
print("Result : {}".format(res))
Let’s try:
doubled_and_add(100000)
doubled_and_add(1000000)
Output:
Result : 9999900000
Finished 'doubled_and_add' in 0.0119 secs
Result : 999999000000
Finished 'doubled_and_add' in 0.0897 secs
Note: I’m not sure why to use time.perf_counter
instead of time.time
. Comments are welcome.
Let me add a little more to https://stackoverflow.com/a/63665115/7412781 solution.
- Removed dependency on
functools
. - Used process time taken
time.process_time()
instead of absolute counter oftime.perf_counter()
because the process can be context switched out via kernel. - Used the raw function pointer print to get the correct class name as well.
This is the decorator code.
import time
def decorator_time_taken(fnc):
def inner(*args):
start = time.process_time()
ret = fnc(*args)
end = time.process_time()
print("{} took {} seconds".format(fnc, round((end - start), 6)))
return ret
return inner
This is the usage sample code. It’s checking if 193939 is prime or not.
class PrimeBrute:
@decorator_time_taken
def isPrime(self, a):
for i in range(a-2):
if a % (i+2) == 0: return False
return True
inst = PrimeBrute()
print(inst.isPrime(193939))
This is the output.
<function PrimeBrute.isPrime at 0x7fc0c6919ae8> took 0.015789 seconds
True
Use timeit
module to benchmark your performance:
def test():
print("test")
emptyFunction()
for i in [x for x in range(10000)]:
i**i
def emptyFunction():
pass
if __name__ == "__main__":
import timeit
print(timeit.timeit("test()", number = 5, globals = globals()))
#print(timeit.timeit("test()", setup = "from __main__ import test",
# number = 5))
the first parameter defines the piece of code which we want to execute test
in this case & number
defines how many times you want to repeat the execution.
Output:
test
test
test
test
test
36.81822113099952
Using the module time
, we can calculate unix time at the start of the function and at the end of a function. Here is how the code might look like:
from time import time as unix
This code imports time.time
which allows us to calculate unix time.
from time import sleep
This is not mandatory, but I am also importing time.sleep
for one of the demonstrations.
START_TIME = unix()
This is what calculates unix time and puts it in a variable. Remember, the function unix is not an actual function. I imported time.time
as unix, so if you did not put as unix
in the first import, you will need to use time.time()
.
After this, we put whichever function or code we want.
At the end of the code snippet we put
TOTAL_TIME = unix()-START_TIME
This line of code does two things: It calculates unix time at the end of the function, and using the variable START_TIME
from before, we calculate the amount of time it took to execute the code snippet.
We can then use this variable wherever we want, including for a print()
function.
print("The snippet took {} seconds to execute".format(TOTAL_TIME))
Here I wrote a quick demonstration code that has two experiments as a demonstration. (Fully commented)
from time import time as unix # Import the module to measure unix time
from time import sleep
# Here are a few examples:
# 1. Counting to 100 000
START_TIME = unix()
for i in range(0, 100001):
print("Number: {}r".format(i), end="")
TOTAL_TIME = unix() - START_TIME
print("nFinal time (Expirement 1): {} sn".format(TOTAL_TIME))
# 2. Precision of sleep
for i in range(10):
START_TIME = unix()
sleep(0.1)
TOTAL_TIME = unix() - START_TIME
print("Sleep(0.1): Index: {}, Time: {} s".format(i,TOTAL_TIME))
Here was my output:
Number: 100000
Final time (Expirement 1): 16.666812419891357 s
Sleep(0.1): Index: 0, Time: 0.10014867782592773 s
Sleep(0.1): Index: 1, Time: 0.10016226768493652 s
Sleep(0.1): Index: 2, Time: 0.10202860832214355 s
Sleep(0.1): Index: 3, Time: 0.10015869140625 s
Sleep(0.1): Index: 4, Time: 0.10014724731445312 s
Sleep(0.1): Index: 5, Time: 0.10013675689697266 s
Sleep(0.1): Index: 6, Time: 0.10014677047729492 s
Sleep(0.1): Index: 7, Time: 0.1001439094543457 s
Sleep(0.1): Index: 8, Time: 0.10044598579406738 s
Sleep(0.1): Index: 9, Time: 0.10014700889587402 s
>
import datetime
#this code before computation
%%timeit
~code~