How do I get time of a Python program's execution?
Question:
I have a command line program in Python that takes a while to finish. I want to know the exact time it takes to finish running.
I’ve looked at the timeit module, but it seems it’s only for small snippets of code. I want to time the whole program.
Answers:
In Linux or Unix:
$ time python yourprogram.py
In Windows, see this StackOverflow question: How do I measure execution time of a command on the Windows command line?
For more verbose output,
$ time -v python yourprogram.py
Command being timed: "python3 yourprogram.py"
User time (seconds): 0.08
System time (seconds): 0.02
Percent of CPU this job got: 98%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.10
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 9480
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 1114
Voluntary context switches: 0
Involuntary context switches: 22
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
The simplest way in Python:
import time
start_time = time.time()
main()
print("--- %s seconds ---" % (time.time() - start_time))
This assumes that your program takes at least a tenth of second to run.
Prints:
--- 0.764891862869 seconds ---
import time
start_time = time.clock()
main()
print(time.clock() - start_time, "seconds")
time.clock() returns the processor time, which allows us to calculate only the time used by this process (on Unix anyway). The documentation says "in any case, this is the function to use for benchmarking Python or timing algorithms"
I put this timing.py module into my own site-packages directory, and just insert import timing at the top of my module:
import atexit
from time import clock
def secondsToStr(t):
return "%d:%02d:%02d.%03d" %
reduce(lambda ll,b : divmod(ll[0],b) + ll[1:],
[(t*1000,),1000,60,60])
line = "="*40
def log(s, elapsed=None):
print line
print secondsToStr(clock()), '-', s
if elapsed:
print "Elapsed time:", elapsed
print line
print
def endlog():
end = clock()
elapsed = end-start
log("End Program", secondsToStr(elapsed))
def now():
return secondsToStr(clock())
start = clock()
atexit.register(endlog)
log("Start Program")
I can also call timing.log from within my program if there are significant stages within the program I want to show. But just including import timing will print the start and end times, and overall elapsed time. (Forgive my obscure secondsToStr function, it just formats a floating point number of seconds to hh:mm:ss.sss form.)
Note: A Python 3 version of the above code can be found here or here.
I really like Paul McGuire’s answer, but I use Python 3. So for those who are interested: here’s a modification of his answer that works with Python 3 on *nix (I imagine, under Windows, that clock() should be used instead of time()):
#python3
import atexit
from time import time, strftime, localtime
from datetime import timedelta
def secondsToStr(elapsed=None):
if elapsed is None:
return strftime("%Y-%m-%d %H:%M:%S", localtime())
else:
return str(timedelta(seconds=elapsed))
def log(s, elapsed=None):
line = "="*40
print(line)
print(secondsToStr(), '-', s)
if elapsed:
print("Elapsed time:", elapsed)
print(line)
print()
def endlog():
end = time()
elapsed = end-start
log("End Program", secondsToStr(elapsed))
start = time()
atexit.register(endlog)
log("Start Program")
If you find this useful, you should still up-vote his answer instead of this one, as he did most of the work ;).
You can use the Python profiler cProfile to measure CPU time and additionally how much time is spent inside each function and how many times each function is called. This is very useful if you want to improve performance of your script without knowing where to start. This answer to another Stack Overflow question is pretty good. It’s always good to have a look in the documentation too.
Here’s an example how to profile a script using cProfile from a command line:
$ python -m cProfile euler048.py
1007 function calls in 0.061 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.061 0.061 <string>:1(<module>)
1000 0.051 0.000 0.051 0.000 euler048.py:2(<lambda>)
1 0.005 0.005 0.061 0.061 euler048.py:2(<module>)
1 0.000 0.000 0.061 0.061 {execfile}
1 0.002 0.002 0.053 0.053 {map}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler objects}
1 0.000 0.000 0.000 0.000 {range}
1 0.003 0.003 0.003 0.003 {sum}
I like the output the datetime module provides, where time delta objects show days, hours, minutes, etc. as necessary in a human-readable way.
For example:
from datetime import datetime
start_time = datetime.now()
# do your work here
end_time = datetime.now()
print('Duration: {}'.format(end_time - start_time))
Sample output e.g.
Duration: 0:00:08.309267
or
Duration: 1 day, 1:51:24.269711
As J.F. Sebastian mentioned, this approach might encounter some tricky cases with local time, so it’s safer to use:
import time
from datetime import timedelta
start_time = time.monotonic()
end_time = time.monotonic()
print(timedelta(seconds=end_time - start_time))
There is a timeit module which can be used to time the execution times of Python code.
It has detailed documentation and examples in Python documentation, 26.6. timeit — Measure execution time of small code snippets.
I liked Paul McGuire’s answer too and came up with a context manager form which suited my needs more.
import datetime as dt
import timeit
class TimingManager(object):
"""Context Manager used with the statement 'with' to time some execution.
Example:
with TimingManager() as t:
# Code to time
"""
clock = timeit.default_timer
def __enter__(self):
"""
"""
self.start = self.clock()
self.log('n=> Start Timing: {}')
return self
def __exit__(self, exc_type, exc_val, exc_tb):
"""
"""
self.endlog()
return False
def log(self, s, elapsed=None):
"""Log current time and elapsed time if present.
:param s: Text to display, use '{}' to format the text with
the current time.
:param elapsed: Elapsed time to display. Dafault: None, no display.
"""
print s.format(self._secondsToStr(self.clock()))
if(elapsed is not None):
print 'Elapsed time: {}n'.format(elapsed)
def endlog(self):
"""Log time for the end of execution with elapsed time.
"""
self.log('=> End Timing: {}', self.now())
def now(self):
"""Return current elapsed time as hh:mm:ss string.
:return: String.
"""
return str(dt.timedelta(seconds = self.clock() - self.start))
def _secondsToStr(self, sec):
"""Convert timestamp to h:mm:ss string.
:param sec: Timestamp.
"""
return str(dt.datetime.fromtimestamp(sec))
I’ve looked at the timeit module, but it seems it’s only for small snippets of code. I want to time the whole program.
$ python -mtimeit -n1 -r1 -t -s "from your_module import main" "main()"
It runs your_module.main() function one time and print the elapsed time using time.time() function as a timer.
To emulate /usr/bin/time in Python see Python subprocess with /usr/bin/time: how to capture timing info but ignore all other output?.
To measure CPU time (e.g., don’t include time during time.sleep()) for each function, you could use profile module (cProfile on Python 2):
$ python3 -mprofile your_module.py
You could pass -p to timeit command above if you want to use the same timer as profile module uses.
This is Paul McGuire’s answer that works for me. Just in case someone was having trouble running that one.
import atexit
from time import clock
def reduce(function, iterable, initializer=None):
it = iter(iterable)
if initializer is None:
value = next(it)
else:
value = initializer
for element in it:
value = function(value, element)
return value
def secondsToStr(t):
return "%d:%02d:%02d.%03d" %
reduce(lambda ll,b : divmod(ll[0],b) + ll[1:],
[(t*1000,),1000,60,60])
line = "="*40
def log(s, elapsed=None):
print (line)
print (secondsToStr(clock()), '-', s)
if elapsed:
print ("Elapsed time:", elapsed)
print (line)
def endlog():
end = clock()
elapsed = end-start
log("End Program", secondsToStr(elapsed))
def now():
return secondsToStr(clock())
def main():
start = clock()
atexit.register(endlog)
log("Start Program")
Call timing.main() from your program after importing the file.
In IPython, “timeit” any script:
def foo():
%run bar.py
timeit foo()
from time import time
start_time = time()
...
end_time = time()
time_taken = end_time - start_time # time_taken is in seconds
hours, rest = divmod(time_taken,3600)
minutes, seconds = divmod(rest, 60)
time.clock()
Deprecated since version 3.3: The behavior of this function depends
on the platform: use perf_counter() or process_time() instead,
depending on your requirements, to have a well-defined behavior.
time.perf_counter()
Return the value (in fractional seconds) of a performance counter,
i.e. a clock with the highest available resolution to measure a short
duration. It does include time elapsed during sleep and is
system-wide.
time.process_time()
Return the value (in fractional seconds) of the sum of the system and
user CPU time of the current process. It does not include time elapsed
during sleep.
start = time.process_time()
... do something
elapsed = (time.process_time() - start)
The following snippet prints elapsed time in a nice human readable <HH:MM:SS> format.
import time
from datetime import timedelta
start_time = time.time()
#
# Perform lots of computations.
#
elapsed_time_secs = time.time() - start_time
msg = "Execution took: %s secs (Wall clock time)" % timedelta(seconds=round(elapsed_time_secs))
print(msg)
To use metakermit’s updated answer for Python 2.7, you will require the monotonic package.
The code would then be as follows:
from datetime import timedelta
from monotonic import monotonic
start_time = monotonic()
end_time = monotonic()
print(timedelta(seconds=end_time - start_time))
The time of a Python program’s execution measure could be inconsistent depending on:
- Same program can be evaluated using different algorithms
- Running time varies between algorithms
- Running time varies between implementations
- Running time varies between computers
- Running time is not predictable based on small inputs
This is because the most effective way is using the “Order of Growth” and learn the Big “O” notation to do it properly.
Anyway, you can try to evaluate the performance of any Python program in specific machine counting steps per second using this simple algorithm:
adapt this to the program you want to evaluate
import time
now = time.time()
future = now + 10
step = 4 # Why 4 steps? Because until here already four operations executed
while time.time() < future:
step += 3 # Why 3 again? Because a while loop executes one comparison and one plus equal statement
step += 4 # Why 3 more? Because one comparison starting while when time is over plus the final assignment of step + 1 and print statement
print(str(int(step / 10)) + " steps per second")
Just use the timeit module. It works with both Python 2 and Python 3.
import timeit
start = timeit.default_timer()
# All the program statements
stop = timeit.default_timer()
execution_time = stop - start
print("Program Executed in "+str(execution_time)) # It returns time in seconds
It returns in seconds and you can have your execution time. It is simple, but you should write these in thew main function which starts program execution. If you want to get the execution time even when you get an error then take your parameter “Start” to it and calculate there like:
def sample_function(start,**kwargs):
try:
# Your statements
except:
# except statements run when your statements raise an exception
stop = timeit.default_timer()
execution_time = stop - start
print("Program executed in " + str(execution_time))
Timeit is a class in Python used to calculate the execution time of small blocks of code.
Default_timer is a method in this class which is used to measure the wall clock timing, not CPU execution time. Thus other process execution might interfere with this. Thus it is useful for small blocks of code.
A sample of the code is as follows:
from timeit import default_timer as timer
start= timer()
# Some logic
end = timer()
print("Time taken:", end-start)
Use line_profiler.
line_profiler will profile the time individual lines of code take to execute. The profiler is implemented in C via Cython in order to reduce the overhead of profiling.
from line_profiler import LineProfiler
import random
def do_stuff(numbers):
s = sum(numbers)
l = [numbers[i]/43 for i in range(len(numbers))]
m = ['hello'+str(numbers[i]) for i in range(len(numbers))]
numbers = [random.randint(1,100) for i in range(1000)]
lp = LineProfiler()
lp_wrapper = lp(do_stuff)
lp_wrapper(numbers)
lp.print_stats()
The results will be:
Timer unit: 1e-06 s
Total time: 0.000649 s
File: <ipython-input-2-2e060b054fea>
Function: do_stuff at line 4
Line # Hits Time Per Hit % Time Line Contents
==============================================================
4 def do_stuff(numbers):
5 1 10 10.0 1.5 s = sum(numbers)
6 1 186 186.0 28.7 l = [numbers[i]/43 for i in range(len(numbers))]
7 1 453 453.0 69.8 m = ['hello'+str(numbers[i]) for i in range(len(numbers))]
For the data folks using Jupyter Notebook
In a cell, you can use Jupyter’s %%time magic command to measure the execution time:
%%time
[ x**2 for x in range(10000)]
Output
CPU times: user 4.54 ms, sys: 0 ns, total: 4.54 ms
Wall time: 4.12 ms
This will only capture the execution time of a particular cell. If you’d like to capture the execution time of the whole notebook (i.e. program), you can create a new notebook in the same directory and in the new notebook execute all cells:
Suppose the notebook above is called example_notebook.ipynb. In a new notebook within the same directory:
# Convert your notebook to a .py script:
!jupyter nbconvert --to script example_notebook.ipynb
# Run the example_notebook with -t flag for time
%run -t example_notebook
Output
IPython CPU timings (estimated):
User : 0.00 s.
System : 0.00 s.
Wall time: 0.00 s.
I used a very simple function to time a part of code execution:
import time
def timing():
start_time = time.time()
return lambda x: print("[{:.2f}s] {}".format(time.time() - start_time, x))
And to use it, just call it before the code to measure to retrieve function timing, and then call the function after the code with comments. The time will appear in front of the comments. For example:
t = timing()
train = pd.read_csv('train.csv',
dtype={
'id': str,
'vendor_id': str,
'pickup_datetime': str,
'dropoff_datetime': str,
'passenger_count': int,
'pickup_longitude': np.float64,
'pickup_latitude': np.float64,
'dropoff_longitude': np.float64,
'dropoff_latitude': np.float64,
'store_and_fwd_flag': str,
'trip_duration': int,
},
parse_dates = ['pickup_datetime', 'dropoff_datetime'],
)
t("Loaded {} rows data from 'train'".format(len(train)))
Then the output will look like this:
[9.35s] Loaded 1458644 rows data from 'train'
If you want to measure time in microseconds, then you can use the following version, based completely on the answers of Paul McGuire and Nicojo – it’s Python 3 code. I’ve also added some colour to it:
import atexit
from time import time
from datetime import timedelta, datetime
def seconds_to_str(elapsed=None):
if elapsed is None:
return datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f")
else:
return str(timedelta(seconds=elapsed))
def log(txt, elapsed=None):
colour_cyan = ' 33[36m'
colour_reset = ' 33[0;0;39m'
colour_red = ' 33[31m'
print('n ' + colour_cyan + ' [TIMING]> [' + seconds_to_str() + '] ----> ' + txt + 'n' + colour_reset)
if elapsed:
print("n " + colour_red + " [TIMING]> Elapsed time ==> " + elapsed + "n" + colour_reset)
def end_log():
end = time()
elapsed = end-start
log("End Program", seconds_to_str(elapsed))
start = time()
atexit.register(end_log)
log("Start Program")
log() => function that prints out the timing information.
txt ==> first argument to log, and its string to mark timing.
atexit ==> Python module to register functions that you can call when the program exits.
You do this simply in Python. There is no need to make it complicated.
import time
start = time.localtime()
end = time.localtime()
"""Total execution time in minutes$ """
print(end.tm_min - start.tm_min)
"""Total execution time in seconds$ """
print(end.tm_sec - start.tm_sec)
This is the simplest way to get the elapsed time for the program:
Write the following code at the end of your program.
import time
print(time.clock())
I was having the same problem in many places, so I created a convenience package horology. You can install it with pip install horology and then do it in the elegant way:
from horology import Timing
with Timing(name='Important calculations: '):
prepare()
do_your_stuff()
finish_sth()
will output:
Important calculations: 12.43 ms
Or even simpler (if you have one function):
from horology import timed
@timed
def main():
...
will output:
main: 7.12 h
It takes care of units and rounding. It works with python 3.6 or newer.
Later answer, but I use the built-in timeit:
import timeit
code_to_test = """
a = range(100000)
b = []
for i in a:
b.append(i*2)
"""
elapsed_time = timeit.timeit(code_to_test, number=500)
print(elapsed_time)
# 10.159821493085474
- Wrap all your code, including any imports you may have, inside
code_to_test.
number argument specifies the amount of times the code should repeat.- Demo
Similar to the response from @rogeriopvl I added a slight modification to convert to hour minute seconds using the same library for long running jobs.
import time
start_time = time.time()
main()
seconds = time.time() - start_time
print('Time Taken:', time.strftime("%H:%M:%S",time.gmtime(seconds)))
Sample Output
Time Taken: 00:00:08
First, install humanfriendly package by opening Command Prompt (CMD) as administrator and type there –
pip install humanfriendly
Code:
from humanfriendly import format_timespan
import time
begin_time = time.time()
# Put your code here
end_time = time.time() - begin_time
print("Total execution time: ", format_timespan(end_time))
Output:
I tried and found time difference using the following scripts.
import time
start_time = time.perf_counter()
[main code here]
print (time.perf_counter() - start_time, "seconds")
For functions, I suggest using this simple decorator I created.
def timeit(method):
def timed(*args, **kwargs):
ts = time.time()
result = method(*args, **kwargs)
te = time.time()
if 'log_time' in kwargs:
name = kwargs.get('log_name', method.__name__.upper())
kwargs['log_time'][name] = int((te - ts) * 1000)
else:
print('%r %2.22f ms' % (method.__name__, (te - ts) * 1000))
return result
return timed
@timeit
def foo():
do_some_work()
# foo()
# 'foo' 0.000953 ms
Following this answer created a simple but convenient instrument.
import time
from datetime import timedelta
def start_time_measure(message=None):
if message:
print(message)
return time.monotonic()
def end_time_measure(start_time, print_prefix=None):
end_time = time.monotonic()
if print_prefix:
print(print_prefix + str(timedelta(seconds=end_time - start_time)))
return end_time
Usage:
total_start_time = start_time_measure()
start_time = start_time_measure('Doing something...')
# Do something
end_time_measure(start_time, 'Done in: ')
start_time = start_time_measure('Doing something else...')
# Do something else
end_time_measure(start_time, 'Done in: ')
end_time_measure(total_start_time, 'Total time: ')
The output:
Doing something...
Done in: 0:00:01.218000
Doing something else...
Done in: 0:00:01.313000
Total time: 0:00:02.672000
time.clock has been deprecated in Python 3.3 and will be removed from Python 3.8: use time.perf_counter or time.process_time instead
import time
start_time = time.perf_counter ()
for x in range(1, 100):
print(x)
end_time = time.perf_counter ()
print(end_time - start_time, "seconds")
I define the following Python decorator:
def profile(fct):
def wrapper(*args, **kw):
start_time = time.time()
ret = fct(*args, **kw)
print("{} {} {} return {} in {} seconds".format(args[0].__class__.__name__,
args[0].__class__.__module__,
fct.__name__,
ret,
time.time() - start_time))
return ret
return wrapper
and use it on functions or class/methods:
@profile
def main()
...
I use tic and toc from ttictoc.
pip install ttictoc
Then you can use in your script:
from ttictoc import tic,toc
tic()
# foo()
print(toc())
By default, Linux or Unix system (tested on macOS) comes with the time command on the terminal that you can use to run the Python script and get the real, user, sys time information for the execution of the running script.
However, the default output isn’t very clear (at least for me), and the default time command doesn’t even take any options as arguments to format the output. That’s because there are two versions of time – one is built into bash that provides just the minimal version and another one is located on /usr/bin/time.
The /usr/bin/time command accepts additional arguments like -al, -h, -p, and -o. My favorite is -p which shows the output in a new line like the following:
real 2.18
user 17.92
sys 2.71
time.clock has been deprecated in Python 3.3 and was removed from Python 3.8
You should use time.perf_counter or time.process_time instead.
time.perf_counter:
time.perf_counter returns the value (in fractional seconds) of a performance counter, i.e. a clock with the highest available resolution to measure a short duration. It does include time elapsed during sleep and is system-wide. The reference point of the returned value is undefined, so that only the difference between the results of two calls is valid.
time.process_time: time.process_time returns the value (in fractional seconds) of the sum of the system and user CPU time of the current process. It does not include time elapsed during sleep. It is process-wide by definition. The reference point of the returned value is undefined, so that only the difference between the results of two calls is valid.
You can see this documentation regarding the usage. Also, see the following example:
import time
s_time = time.perf_counter()
// call your function here
e_time = time.perf_counter()
print(e_time - s_time, "seconds")
I have a command line program in Python that takes a while to finish. I want to know the exact time it takes to finish running.
I’ve looked at the timeit module, but it seems it’s only for small snippets of code. I want to time the whole program.
In Linux or Unix:
$ time python yourprogram.py
In Windows, see this StackOverflow question: How do I measure execution time of a command on the Windows command line?
For more verbose output,
$ time -v python yourprogram.py
Command being timed: "python3 yourprogram.py"
User time (seconds): 0.08
System time (seconds): 0.02
Percent of CPU this job got: 98%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.10
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 9480
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 1114
Voluntary context switches: 0
Involuntary context switches: 22
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
The simplest way in Python:
import time
start_time = time.time()
main()
print("--- %s seconds ---" % (time.time() - start_time))
This assumes that your program takes at least a tenth of second to run.
Prints:
--- 0.764891862869 seconds ---
import time
start_time = time.clock()
main()
print(time.clock() - start_time, "seconds")
time.clock() returns the processor time, which allows us to calculate only the time used by this process (on Unix anyway). The documentation says "in any case, this is the function to use for benchmarking Python or timing algorithms"
I put this timing.py module into my own site-packages directory, and just insert import timing at the top of my module:
import atexit
from time import clock
def secondsToStr(t):
return "%d:%02d:%02d.%03d" %
reduce(lambda ll,b : divmod(ll[0],b) + ll[1:],
[(t*1000,),1000,60,60])
line = "="*40
def log(s, elapsed=None):
print line
print secondsToStr(clock()), '-', s
if elapsed:
print "Elapsed time:", elapsed
print line
print
def endlog():
end = clock()
elapsed = end-start
log("End Program", secondsToStr(elapsed))
def now():
return secondsToStr(clock())
start = clock()
atexit.register(endlog)
log("Start Program")
I can also call timing.log from within my program if there are significant stages within the program I want to show. But just including import timing will print the start and end times, and overall elapsed time. (Forgive my obscure secondsToStr function, it just formats a floating point number of seconds to hh:mm:ss.sss form.)
Note: A Python 3 version of the above code can be found here or here.
I really like Paul McGuire’s answer, but I use Python 3. So for those who are interested: here’s a modification of his answer that works with Python 3 on *nix (I imagine, under Windows, that clock() should be used instead of time()):
#python3
import atexit
from time import time, strftime, localtime
from datetime import timedelta
def secondsToStr(elapsed=None):
if elapsed is None:
return strftime("%Y-%m-%d %H:%M:%S", localtime())
else:
return str(timedelta(seconds=elapsed))
def log(s, elapsed=None):
line = "="*40
print(line)
print(secondsToStr(), '-', s)
if elapsed:
print("Elapsed time:", elapsed)
print(line)
print()
def endlog():
end = time()
elapsed = end-start
log("End Program", secondsToStr(elapsed))
start = time()
atexit.register(endlog)
log("Start Program")
If you find this useful, you should still up-vote his answer instead of this one, as he did most of the work ;).
You can use the Python profiler cProfile to measure CPU time and additionally how much time is spent inside each function and how many times each function is called. This is very useful if you want to improve performance of your script without knowing where to start. This answer to another Stack Overflow question is pretty good. It’s always good to have a look in the documentation too.
Here’s an example how to profile a script using cProfile from a command line:
$ python -m cProfile euler048.py
1007 function calls in 0.061 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.061 0.061 <string>:1(<module>)
1000 0.051 0.000 0.051 0.000 euler048.py:2(<lambda>)
1 0.005 0.005 0.061 0.061 euler048.py:2(<module>)
1 0.000 0.000 0.061 0.061 {execfile}
1 0.002 0.002 0.053 0.053 {map}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler objects}
1 0.000 0.000 0.000 0.000 {range}
1 0.003 0.003 0.003 0.003 {sum}
I like the output the datetime module provides, where time delta objects show days, hours, minutes, etc. as necessary in a human-readable way.
For example:
from datetime import datetime
start_time = datetime.now()
# do your work here
end_time = datetime.now()
print('Duration: {}'.format(end_time - start_time))
Sample output e.g.
Duration: 0:00:08.309267
or
Duration: 1 day, 1:51:24.269711
As J.F. Sebastian mentioned, this approach might encounter some tricky cases with local time, so it’s safer to use:
import time
from datetime import timedelta
start_time = time.monotonic()
end_time = time.monotonic()
print(timedelta(seconds=end_time - start_time))
There is a timeit module which can be used to time the execution times of Python code.
It has detailed documentation and examples in Python documentation, 26.6. timeit — Measure execution time of small code snippets.
I liked Paul McGuire’s answer too and came up with a context manager form which suited my needs more.
import datetime as dt
import timeit
class TimingManager(object):
"""Context Manager used with the statement 'with' to time some execution.
Example:
with TimingManager() as t:
# Code to time
"""
clock = timeit.default_timer
def __enter__(self):
"""
"""
self.start = self.clock()
self.log('n=> Start Timing: {}')
return self
def __exit__(self, exc_type, exc_val, exc_tb):
"""
"""
self.endlog()
return False
def log(self, s, elapsed=None):
"""Log current time and elapsed time if present.
:param s: Text to display, use '{}' to format the text with
the current time.
:param elapsed: Elapsed time to display. Dafault: None, no display.
"""
print s.format(self._secondsToStr(self.clock()))
if(elapsed is not None):
print 'Elapsed time: {}n'.format(elapsed)
def endlog(self):
"""Log time for the end of execution with elapsed time.
"""
self.log('=> End Timing: {}', self.now())
def now(self):
"""Return current elapsed time as hh:mm:ss string.
:return: String.
"""
return str(dt.timedelta(seconds = self.clock() - self.start))
def _secondsToStr(self, sec):
"""Convert timestamp to h:mm:ss string.
:param sec: Timestamp.
"""
return str(dt.datetime.fromtimestamp(sec))
I’ve looked at the timeit module, but it seems it’s only for small snippets of code. I want to time the whole program.
$ python -mtimeit -n1 -r1 -t -s "from your_module import main" "main()"
It runs your_module.main() function one time and print the elapsed time using time.time() function as a timer.
To emulate /usr/bin/time in Python see Python subprocess with /usr/bin/time: how to capture timing info but ignore all other output?.
To measure CPU time (e.g., don’t include time during time.sleep()) for each function, you could use profile module (cProfile on Python 2):
$ python3 -mprofile your_module.py
You could pass -p to timeit command above if you want to use the same timer as profile module uses.
This is Paul McGuire’s answer that works for me. Just in case someone was having trouble running that one.
import atexit
from time import clock
def reduce(function, iterable, initializer=None):
it = iter(iterable)
if initializer is None:
value = next(it)
else:
value = initializer
for element in it:
value = function(value, element)
return value
def secondsToStr(t):
return "%d:%02d:%02d.%03d" %
reduce(lambda ll,b : divmod(ll[0],b) + ll[1:],
[(t*1000,),1000,60,60])
line = "="*40
def log(s, elapsed=None):
print (line)
print (secondsToStr(clock()), '-', s)
if elapsed:
print ("Elapsed time:", elapsed)
print (line)
def endlog():
end = clock()
elapsed = end-start
log("End Program", secondsToStr(elapsed))
def now():
return secondsToStr(clock())
def main():
start = clock()
atexit.register(endlog)
log("Start Program")
Call timing.main() from your program after importing the file.
In IPython, “timeit” any script:
def foo():
%run bar.py
timeit foo()
from time import time
start_time = time()
...
end_time = time()
time_taken = end_time - start_time # time_taken is in seconds
hours, rest = divmod(time_taken,3600)
minutes, seconds = divmod(rest, 60)
time.clock()
Deprecated since version 3.3: The behavior of this function depends
on the platform: use perf_counter() or process_time() instead,
depending on your requirements, to have a well-defined behavior.
time.perf_counter()
Return the value (in fractional seconds) of a performance counter,
i.e. a clock with the highest available resolution to measure a short
duration. It does include time elapsed during sleep and is
system-wide.
time.process_time()
Return the value (in fractional seconds) of the sum of the system and
user CPU time of the current process. It does not include time elapsed
during sleep.
start = time.process_time()
... do something
elapsed = (time.process_time() - start)
The following snippet prints elapsed time in a nice human readable <HH:MM:SS> format.
import time
from datetime import timedelta
start_time = time.time()
#
# Perform lots of computations.
#
elapsed_time_secs = time.time() - start_time
msg = "Execution took: %s secs (Wall clock time)" % timedelta(seconds=round(elapsed_time_secs))
print(msg)
To use metakermit’s updated answer for Python 2.7, you will require the monotonic package.
The code would then be as follows:
from datetime import timedelta
from monotonic import monotonic
start_time = monotonic()
end_time = monotonic()
print(timedelta(seconds=end_time - start_time))
The time of a Python program’s execution measure could be inconsistent depending on:
- Same program can be evaluated using different algorithms
- Running time varies between algorithms
- Running time varies between implementations
- Running time varies between computers
- Running time is not predictable based on small inputs
This is because the most effective way is using the “Order of Growth” and learn the Big “O” notation to do it properly.
Anyway, you can try to evaluate the performance of any Python program in specific machine counting steps per second using this simple algorithm:
adapt this to the program you want to evaluate
import time
now = time.time()
future = now + 10
step = 4 # Why 4 steps? Because until here already four operations executed
while time.time() < future:
step += 3 # Why 3 again? Because a while loop executes one comparison and one plus equal statement
step += 4 # Why 3 more? Because one comparison starting while when time is over plus the final assignment of step + 1 and print statement
print(str(int(step / 10)) + " steps per second")
Just use the timeit module. It works with both Python 2 and Python 3.
import timeit
start = timeit.default_timer()
# All the program statements
stop = timeit.default_timer()
execution_time = stop - start
print("Program Executed in "+str(execution_time)) # It returns time in seconds
It returns in seconds and you can have your execution time. It is simple, but you should write these in thew main function which starts program execution. If you want to get the execution time even when you get an error then take your parameter “Start” to it and calculate there like:
def sample_function(start,**kwargs):
try:
# Your statements
except:
# except statements run when your statements raise an exception
stop = timeit.default_timer()
execution_time = stop - start
print("Program executed in " + str(execution_time))
Timeit is a class in Python used to calculate the execution time of small blocks of code.
Default_timer is a method in this class which is used to measure the wall clock timing, not CPU execution time. Thus other process execution might interfere with this. Thus it is useful for small blocks of code.
A sample of the code is as follows:
from timeit import default_timer as timer
start= timer()
# Some logic
end = timer()
print("Time taken:", end-start)
Use line_profiler.
line_profiler will profile the time individual lines of code take to execute. The profiler is implemented in C via Cython in order to reduce the overhead of profiling.
from line_profiler import LineProfiler
import random
def do_stuff(numbers):
s = sum(numbers)
l = [numbers[i]/43 for i in range(len(numbers))]
m = ['hello'+str(numbers[i]) for i in range(len(numbers))]
numbers = [random.randint(1,100) for i in range(1000)]
lp = LineProfiler()
lp_wrapper = lp(do_stuff)
lp_wrapper(numbers)
lp.print_stats()
The results will be:
Timer unit: 1e-06 s
Total time: 0.000649 s
File: <ipython-input-2-2e060b054fea>
Function: do_stuff at line 4
Line # Hits Time Per Hit % Time Line Contents
==============================================================
4 def do_stuff(numbers):
5 1 10 10.0 1.5 s = sum(numbers)
6 1 186 186.0 28.7 l = [numbers[i]/43 for i in range(len(numbers))]
7 1 453 453.0 69.8 m = ['hello'+str(numbers[i]) for i in range(len(numbers))]
For the data folks using Jupyter Notebook
In a cell, you can use Jupyter’s %%time magic command to measure the execution time:
%%time
[ x**2 for x in range(10000)]
Output
CPU times: user 4.54 ms, sys: 0 ns, total: 4.54 ms
Wall time: 4.12 ms
This will only capture the execution time of a particular cell. If you’d like to capture the execution time of the whole notebook (i.e. program), you can create a new notebook in the same directory and in the new notebook execute all cells:
Suppose the notebook above is called example_notebook.ipynb. In a new notebook within the same directory:
# Convert your notebook to a .py script:
!jupyter nbconvert --to script example_notebook.ipynb
# Run the example_notebook with -t flag for time
%run -t example_notebook
Output
IPython CPU timings (estimated):
User : 0.00 s.
System : 0.00 s.
Wall time: 0.00 s.
I used a very simple function to time a part of code execution:
import time
def timing():
start_time = time.time()
return lambda x: print("[{:.2f}s] {}".format(time.time() - start_time, x))
And to use it, just call it before the code to measure to retrieve function timing, and then call the function after the code with comments. The time will appear in front of the comments. For example:
t = timing()
train = pd.read_csv('train.csv',
dtype={
'id': str,
'vendor_id': str,
'pickup_datetime': str,
'dropoff_datetime': str,
'passenger_count': int,
'pickup_longitude': np.float64,
'pickup_latitude': np.float64,
'dropoff_longitude': np.float64,
'dropoff_latitude': np.float64,
'store_and_fwd_flag': str,
'trip_duration': int,
},
parse_dates = ['pickup_datetime', 'dropoff_datetime'],
)
t("Loaded {} rows data from 'train'".format(len(train)))
Then the output will look like this:
[9.35s] Loaded 1458644 rows data from 'train'
If you want to measure time in microseconds, then you can use the following version, based completely on the answers of Paul McGuire and Nicojo – it’s Python 3 code. I’ve also added some colour to it:
import atexit
from time import time
from datetime import timedelta, datetime
def seconds_to_str(elapsed=None):
if elapsed is None:
return datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f")
else:
return str(timedelta(seconds=elapsed))
def log(txt, elapsed=None):
colour_cyan = ' 33[36m'
colour_reset = ' 33[0;0;39m'
colour_red = ' 33[31m'
print('n ' + colour_cyan + ' [TIMING]> [' + seconds_to_str() + '] ----> ' + txt + 'n' + colour_reset)
if elapsed:
print("n " + colour_red + " [TIMING]> Elapsed time ==> " + elapsed + "n" + colour_reset)
def end_log():
end = time()
elapsed = end-start
log("End Program", seconds_to_str(elapsed))
start = time()
atexit.register(end_log)
log("Start Program")
log() => function that prints out the timing information.
txt ==> first argument to log, and its string to mark timing.
atexit ==> Python module to register functions that you can call when the program exits.
You do this simply in Python. There is no need to make it complicated.
import time
start = time.localtime()
end = time.localtime()
"""Total execution time in minutes$ """
print(end.tm_min - start.tm_min)
"""Total execution time in seconds$ """
print(end.tm_sec - start.tm_sec)
This is the simplest way to get the elapsed time for the program:
Write the following code at the end of your program.
import time
print(time.clock())
I was having the same problem in many places, so I created a convenience package horology. You can install it with pip install horology and then do it in the elegant way:
from horology import Timing
with Timing(name='Important calculations: '):
prepare()
do_your_stuff()
finish_sth()
will output:
Important calculations: 12.43 ms
Or even simpler (if you have one function):
from horology import timed
@timed
def main():
...
will output:
main: 7.12 h
It takes care of units and rounding. It works with python 3.6 or newer.
Later answer, but I use the built-in timeit:
import timeit
code_to_test = """
a = range(100000)
b = []
for i in a:
b.append(i*2)
"""
elapsed_time = timeit.timeit(code_to_test, number=500)
print(elapsed_time)
# 10.159821493085474
- Wrap all your code, including any imports you may have, inside
code_to_test. numberargument specifies the amount of times the code should repeat.- Demo
Similar to the response from @rogeriopvl I added a slight modification to convert to hour minute seconds using the same library for long running jobs.
import time
start_time = time.time()
main()
seconds = time.time() - start_time
print('Time Taken:', time.strftime("%H:%M:%S",time.gmtime(seconds)))
Sample Output
Time Taken: 00:00:08
First, install humanfriendly package by opening Command Prompt (CMD) as administrator and type there –
pip install humanfriendly
Code:
from humanfriendly import format_timespan
import time
begin_time = time.time()
# Put your code here
end_time = time.time() - begin_time
print("Total execution time: ", format_timespan(end_time))
Output:
I tried and found time difference using the following scripts.
import time
start_time = time.perf_counter()
[main code here]
print (time.perf_counter() - start_time, "seconds")
For functions, I suggest using this simple decorator I created.
def timeit(method):
def timed(*args, **kwargs):
ts = time.time()
result = method(*args, **kwargs)
te = time.time()
if 'log_time' in kwargs:
name = kwargs.get('log_name', method.__name__.upper())
kwargs['log_time'][name] = int((te - ts) * 1000)
else:
print('%r %2.22f ms' % (method.__name__, (te - ts) * 1000))
return result
return timed
@timeit
def foo():
do_some_work()
# foo()
# 'foo' 0.000953 ms
Following this answer created a simple but convenient instrument.
import time
from datetime import timedelta
def start_time_measure(message=None):
if message:
print(message)
return time.monotonic()
def end_time_measure(start_time, print_prefix=None):
end_time = time.monotonic()
if print_prefix:
print(print_prefix + str(timedelta(seconds=end_time - start_time)))
return end_time
Usage:
total_start_time = start_time_measure()
start_time = start_time_measure('Doing something...')
# Do something
end_time_measure(start_time, 'Done in: ')
start_time = start_time_measure('Doing something else...')
# Do something else
end_time_measure(start_time, 'Done in: ')
end_time_measure(total_start_time, 'Total time: ')
The output:
Doing something...
Done in: 0:00:01.218000
Doing something else...
Done in: 0:00:01.313000
Total time: 0:00:02.672000
time.clock has been deprecated in Python 3.3 and will be removed from Python 3.8: use time.perf_counter or time.process_time instead
import time
start_time = time.perf_counter ()
for x in range(1, 100):
print(x)
end_time = time.perf_counter ()
print(end_time - start_time, "seconds")
I define the following Python decorator:
def profile(fct):
def wrapper(*args, **kw):
start_time = time.time()
ret = fct(*args, **kw)
print("{} {} {} return {} in {} seconds".format(args[0].__class__.__name__,
args[0].__class__.__module__,
fct.__name__,
ret,
time.time() - start_time))
return ret
return wrapper
and use it on functions or class/methods:
@profile
def main()
...
I use tic and toc from ttictoc.
pip install ttictoc
Then you can use in your script:
from ttictoc import tic,toc
tic()
# foo()
print(toc())
By default, Linux or Unix system (tested on macOS) comes with the time command on the terminal that you can use to run the Python script and get the real, user, sys time information for the execution of the running script.
However, the default output isn’t very clear (at least for me), and the default time command doesn’t even take any options as arguments to format the output. That’s because there are two versions of time – one is built into bash that provides just the minimal version and another one is located on /usr/bin/time.
The /usr/bin/time command accepts additional arguments like -al, -h, -p, and -o. My favorite is -p which shows the output in a new line like the following:
real 2.18
user 17.92
sys 2.71
time.clock has been deprecated in Python 3.3 and was removed from Python 3.8
You should use time.perf_counter or time.process_time instead.
time.perf_counter:
time.perf_counter returns the value (in fractional seconds) of a performance counter, i.e. a clock with the highest available resolution to measure a short duration. It does include time elapsed during sleep and is system-wide. The reference point of the returned value is undefined, so that only the difference between the results of two calls is valid.
time.process_time: time.process_time returns the value (in fractional seconds) of the sum of the system and user CPU time of the current process. It does not include time elapsed during sleep. It is process-wide by definition. The reference point of the returned value is undefined, so that only the difference between the results of two calls is valid.
You can see this documentation regarding the usage. Also, see the following example:
import time
s_time = time.perf_counter()
// call your function here
e_time = time.perf_counter()
print(e_time - s_time, "seconds")