Python context manager that measures time
Question:
I am struggling to make a piece of code that allows to measure time spent within a “with” statement and assigns the time measured (a float) to the variable provided in the “with” statement.
import time
class catchtime:
def __enter__(self):
self.t = time.clock()
return 1
def __exit__(self, type, value, traceback):
return time.clock() - self.t
with catchtime() as t:
pass
This code leaves t=1
and not the difference between clock() calls. How to approach this problem? I need a way to assign a new value from within the exit method.
PEP 343 describes in more detail how contect manager works but I do not understand most of it.
Answers:
Solved (almost). Resulting variable is coercible and convertible to a float (but not a float itself).
class catchtime:
def __enter__(self):
self.t = time.clock()
return self
def __exit__(self, type, value, traceback):
self.e = time.clock()
def __float__(self):
return float(self.e - self.t)
def __coerce__(self, other):
return (float(self), other)
def __str__(self):
return str(float(self))
def __repr__(self):
return str(float(self))
with catchtime() as t:
pass
print t
print repr(t)
print float(t)
print 0+t
print 1*t
1.10000000001e-05
1.10000000001e-05
1.10000000001e-05
1.10000000001e-05
1.10000000001e-05
You can’t get that to assign your timing to t
. As described in the PEP, the variable you specify in the as
clause (if any) gets assigned the result of calling __enter__
, not __exit__
. In other words, t
is only assigned at the start of the with
block, not at the end.
What you could do is change your __exit__
so that instead of returning the value, it does self.t = time.clock() - self.t
. Then, after the with
block finishes, the t
attribute of the context manager will hold the elapsed time.
To make that work, you also want to return self
instead of 1
from __enter__
. Not sure what you were trying to achieve by using 1
.
So it looks like this:
class catchtime(object):
def __enter__(self):
self.t = time.clock()
return self
def __exit__(self, type, value, traceback):
self.t = time.clock() - self.t
with catchtime() as t:
time.sleep(1)
print(t.t)
And a value pretty close to 1 is printed.
Here is an example of using contextmanager
from time import perf_counter
from contextlib import contextmanager
@contextmanager
def catchtime() -> float:
start = perf_counter()
yield lambda: perf_counter() - start
with catchtime() as t:
import time
time.sleep(1)
print(f"Execution time: {t():.4f} secs")
Output:
Execution time: 1.0014 secs
The top-rated answer can give the incorrect time
As noted by @Mercury, the top answer by @Vlad Bezden, while slick, is technically incorrect since the value yielded by t()
is also potentially affected by code executed outside of the with
statement. For example, if you run time.sleep(5)
after the with
statement but before the print
statement, then calling t()
in the print statement will give you ~6 sec, not ~1 sec.
In some cases, this can be avoided by inserting the print command inside the context manager as below:
from time import perf_counter
from contextlib import contextmanager
@contextmanager
def catchtime() -> float:
start = perf_counter()
yield lambda: perf_counter() - start
print(f'Time: {perf_counter() - start:.3f} seconds')
However, even with this modification, notice how running sleep(5) later on causes the incorrect time to be printed:
from time import sleep
with catchtime() as t:
sleep(1)
# >>> "Time: 1.000 seconds"
sleep(5)
print(f'Time: {t():.3f} seconds')
# >>> "Time: 6.000 seconds"
Solution #1: A fix for the above approach
This solution cumulatively nets the difference between two timer objects, t1
and t2
. The catchtime function can be distilled down into 3 lines of code. Note that perf_counter
has been renamed to press_button
to help with visualization.
- Initialize
t1
and t2
simultaneously. Later t2 will be reassigned to preserve the tick count after the yield statement. For this step, imagine simultaneously pressing the on/off button for timer1 and timer2.
- Measure the difference between timer2 and timer1. Initially, this will be 0 on pass #1, but in subsequent runs, it will be the absolute difference between the tick count in t1 and t2.
- This final step is equivalent to pressing the on/off button for timer2 again. This puts the two timers out of sync, making the difference measurement in the step before meaningful from pass #2 onwards.
from time import perf_counter as press_button
from time import sleep
@contextmanager
def catchtime() -> float:
t1 = t2 = press_button()
yield lambda: t2 - t1
t2 = press_button()
with catchtime() as t:
sleep(1)
sleep(5)
print(f'Time: {t():.3f} seconds')
# >>> Time: 1.000 seconds
Solution #2: An alternative, more flexible approach
This code is similar to the excellent answer given by @BrenBarn, except that it:
- Automatically prints the executed time as a formatted string (remove this by deleting the final
print(self.readout)
)
- Saves the formatted string for later use (
self.readout
)
- Saves the
float
result for later use (self.time
)
from time import perf_counter
class catchtime:
def __enter__(self):
self.time = perf_counter()
return self
def __exit__(self, type, value, traceback):
self.time = perf_counter() - self.time
self.readout = f'Time: {self.time:.3f} seconds'
print(self.readout)
Notice how the intermediate sleep(5)
commands no longer affect the printed time.
from time import sleep
with catchtime() as t:
sleep(1)
# >>> "Time: 1.000 seconds"
sleep(5)
print(t.time)
# >>> 1.000283900000009
sleep(5)
print(t.readout)
# >>> "Time: 1.000 seconds"
The issue in top rated answer could be also fixed as below:
@contextmanager
def catchtime() -> float:
start = perf_counter()
end = start
yield lambda: end - start
end = perf_counter()
You could do it in this way below:
import time
class Exectime:
def __enter__(self):
self.time = time.time()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.time = time.time() - self.time
with Exectime() as ext:
<your code here in with statement>
print('execution time is:' +str(ext.time))
It will calculate time spent to process codes within ‘with’ statement.
I like this approach, which is simple to use and allows a contextual message:
from time import perf_counter
from contextlib import ContextDecorator
class cmtimer(ContextDecorator):
def __init__(self, msg):
self.msg = msg
def __enter__(self):
self.time = perf_counter()
return self
def __exit__(self, type, value, traceback):
elapsed = perf_counter() - self.time
print(f'{self.msg} took {elapsed:.3f} seconds')
Use it this way:
with cmtimer('Loading JSON'):
with open('data.json') as f:
results = json.load(f)
Output:
Loading JSON took 1.577 seconds
With this implementtion you can get time during the process and any time after
from contextlib import contextmanager
from time import perf_counter
@contextmanager
def catchtime(task_name='It', verbose=True):
class timer:
def __init__(self):
self._t1 = None
self._t2 = None
def start(self):
self._t1 = perf_counter()
self._t2 = None
def stop(self):
self._t2 = perf_counter()
@property
def time(self):
return (self._t2 or perf_counter()) - self._t1
t = timer()
t.start()
try:
yield t
finally:
t.stop()
if verbose:
print(f'{task_name} took {t.time :.3f} seconds')
Usage examples:
from time import sleep
############################
# 1. will print result
with catchtime('First task'):
sleep(1)
############################
# 2. will print result (without task name) and save result to t object
with catchtime() as t:
sleep(1)
t.time # operation time is saved here
############################
# 3. will not print anyhting but will save result to t object
with catchtime() as t:
sleep(1)
t.time # operation time is saved here
I am struggling to make a piece of code that allows to measure time spent within a “with” statement and assigns the time measured (a float) to the variable provided in the “with” statement.
import time
class catchtime:
def __enter__(self):
self.t = time.clock()
return 1
def __exit__(self, type, value, traceback):
return time.clock() - self.t
with catchtime() as t:
pass
This code leaves t=1
and not the difference between clock() calls. How to approach this problem? I need a way to assign a new value from within the exit method.
PEP 343 describes in more detail how contect manager works but I do not understand most of it.
Solved (almost). Resulting variable is coercible and convertible to a float (but not a float itself).
class catchtime:
def __enter__(self):
self.t = time.clock()
return self
def __exit__(self, type, value, traceback):
self.e = time.clock()
def __float__(self):
return float(self.e - self.t)
def __coerce__(self, other):
return (float(self), other)
def __str__(self):
return str(float(self))
def __repr__(self):
return str(float(self))
with catchtime() as t:
pass
print t
print repr(t)
print float(t)
print 0+t
print 1*t
1.10000000001e-05
1.10000000001e-05
1.10000000001e-05
1.10000000001e-05
1.10000000001e-05
You can’t get that to assign your timing to t
. As described in the PEP, the variable you specify in the as
clause (if any) gets assigned the result of calling __enter__
, not __exit__
. In other words, t
is only assigned at the start of the with
block, not at the end.
What you could do is change your __exit__
so that instead of returning the value, it does self.t = time.clock() - self.t
. Then, after the with
block finishes, the t
attribute of the context manager will hold the elapsed time.
To make that work, you also want to return self
instead of 1
from __enter__
. Not sure what you were trying to achieve by using 1
.
So it looks like this:
class catchtime(object):
def __enter__(self):
self.t = time.clock()
return self
def __exit__(self, type, value, traceback):
self.t = time.clock() - self.t
with catchtime() as t:
time.sleep(1)
print(t.t)
And a value pretty close to 1 is printed.
Here is an example of using contextmanager
from time import perf_counter
from contextlib import contextmanager
@contextmanager
def catchtime() -> float:
start = perf_counter()
yield lambda: perf_counter() - start
with catchtime() as t:
import time
time.sleep(1)
print(f"Execution time: {t():.4f} secs")
Output:
Execution time: 1.0014 secs
The top-rated answer can give the incorrect time
As noted by @Mercury, the top answer by @Vlad Bezden, while slick, is technically incorrect since the value yielded by t()
is also potentially affected by code executed outside of the with
statement. For example, if you run time.sleep(5)
after the with
statement but before the print
statement, then calling t()
in the print statement will give you ~6 sec, not ~1 sec.
In some cases, this can be avoided by inserting the print command inside the context manager as below:
from time import perf_counter
from contextlib import contextmanager
@contextmanager
def catchtime() -> float:
start = perf_counter()
yield lambda: perf_counter() - start
print(f'Time: {perf_counter() - start:.3f} seconds')
However, even with this modification, notice how running sleep(5) later on causes the incorrect time to be printed:
from time import sleep
with catchtime() as t:
sleep(1)
# >>> "Time: 1.000 seconds"
sleep(5)
print(f'Time: {t():.3f} seconds')
# >>> "Time: 6.000 seconds"
Solution #1: A fix for the above approach
This solution cumulatively nets the difference between two timer objects, t1
and t2
. The catchtime function can be distilled down into 3 lines of code. Note that perf_counter
has been renamed to press_button
to help with visualization.
- Initialize
t1
andt2
simultaneously. Later t2 will be reassigned to preserve the tick count after the yield statement. For this step, imagine simultaneously pressing the on/off button for timer1 and timer2. - Measure the difference between timer2 and timer1. Initially, this will be 0 on pass #1, but in subsequent runs, it will be the absolute difference between the tick count in t1 and t2.
- This final step is equivalent to pressing the on/off button for timer2 again. This puts the two timers out of sync, making the difference measurement in the step before meaningful from pass #2 onwards.
from time import perf_counter as press_button
from time import sleep
@contextmanager
def catchtime() -> float:
t1 = t2 = press_button()
yield lambda: t2 - t1
t2 = press_button()
with catchtime() as t:
sleep(1)
sleep(5)
print(f'Time: {t():.3f} seconds')
# >>> Time: 1.000 seconds
Solution #2: An alternative, more flexible approach
This code is similar to the excellent answer given by @BrenBarn, except that it:
- Automatically prints the executed time as a formatted string (remove this by deleting the final
print(self.readout)
) - Saves the formatted string for later use (
self.readout
) - Saves the
float
result for later use (self.time
)
from time import perf_counter
class catchtime:
def __enter__(self):
self.time = perf_counter()
return self
def __exit__(self, type, value, traceback):
self.time = perf_counter() - self.time
self.readout = f'Time: {self.time:.3f} seconds'
print(self.readout)
Notice how the intermediate sleep(5)
commands no longer affect the printed time.
from time import sleep
with catchtime() as t:
sleep(1)
# >>> "Time: 1.000 seconds"
sleep(5)
print(t.time)
# >>> 1.000283900000009
sleep(5)
print(t.readout)
# >>> "Time: 1.000 seconds"
The issue in top rated answer could be also fixed as below:
@contextmanager
def catchtime() -> float:
start = perf_counter()
end = start
yield lambda: end - start
end = perf_counter()
You could do it in this way below:
import time
class Exectime:
def __enter__(self):
self.time = time.time()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.time = time.time() - self.time
with Exectime() as ext:
<your code here in with statement>
print('execution time is:' +str(ext.time))
It will calculate time spent to process codes within ‘with’ statement.
I like this approach, which is simple to use and allows a contextual message:
from time import perf_counter
from contextlib import ContextDecorator
class cmtimer(ContextDecorator):
def __init__(self, msg):
self.msg = msg
def __enter__(self):
self.time = perf_counter()
return self
def __exit__(self, type, value, traceback):
elapsed = perf_counter() - self.time
print(f'{self.msg} took {elapsed:.3f} seconds')
Use it this way:
with cmtimer('Loading JSON'):
with open('data.json') as f:
results = json.load(f)
Output:
Loading JSON took 1.577 seconds
With this implementtion you can get time during the process and any time after
from contextlib import contextmanager
from time import perf_counter
@contextmanager
def catchtime(task_name='It', verbose=True):
class timer:
def __init__(self):
self._t1 = None
self._t2 = None
def start(self):
self._t1 = perf_counter()
self._t2 = None
def stop(self):
self._t2 = perf_counter()
@property
def time(self):
return (self._t2 or perf_counter()) - self._t1
t = timer()
t.start()
try:
yield t
finally:
t.stop()
if verbose:
print(f'{task_name} took {t.time :.3f} seconds')
Usage examples:
from time import sleep
############################
# 1. will print result
with catchtime('First task'):
sleep(1)
############################
# 2. will print result (without task name) and save result to t object
with catchtime() as t:
sleep(1)
t.time # operation time is saved here
############################
# 3. will not print anyhting but will save result to t object
with catchtime() as t:
sleep(1)
t.time # operation time is saved here