Python Performance Optimization: Measuring Execution Time with timeit

Phase 1: Measuring the execution time

timeit is a built-in Python library that allows you to measure the execution time of a specific piece of code. The timeit module contains a timeit function that takes a snippet (as a string) as positional argument, and different keyword arguments such as the number of times you want to run the code. We usually want that number to be big. Indeed, the execution time of the snippet will vary almost every time you run it since it depends on the machine resources and other external variables. Hence the need to run it several times and compute the average execution time for better accuracy.

Here’s a basic example taken from the official doc leveraging multi-line strings, probably the most convenient way to call timeit on long snippets:

import timeit

statement = '''
try:
    str.__bool__
except AttributeError:
    pass
'''
timeit.timeit(stmt=statement, number=100000)

Another convenient way of calling timeit is via methods. Just make sure you pass a setup parameter with instructions to import the method you’ve defined.

def test():
    '''Stupid test function'''
    L = [i for i in range(100)]

timeit.timeit('test()', setup='from __main__ import test')

Phase 2: Identifying & optimizing the bottlenecks

The only purpose of this section is to remind you that in most cases, you should probably focus on the bottlenecks of your code. While it might be tempting and/or satisfying to optimize your whole codebase, a piece of code often needs optimizing when a specific number of methods are particularly expensive, and represent the majority of the overall execution time. Once again, the 80-20 rule for the win!