How to benchmark your python program?

theroyakash — Mon, 23 Nov 2020 10:48:39 +0000

Let's say you have a really slow program and you want to benchmark where your program is taking most of the time to run. If you can find that you can just optimize that part of the program to run faster.

There is couple of way of doing this going through this manually or using some kind of library like cProfile to generate a report on the function's workings.

Generating reports

I've written all necessary code to run this in my package AKDSFramework.
AKDSFramework can be found here. You can pretty much use this on any python function as you like, small-big-has other dependency anything.

If you install it you can get the benchmarking and implementation of several data structures and algorithms using best practices in it.

If you don't wish to use my package at the end of the blog I'll paste the source code for @benchmark decorator.

Example implementation

We gonna see an example of implementation of benchmarking by building a max heap and adding 2 numbers to the heap and again building it.

To make max heaps I'll use AKDSFramework, let's create a heap and build it now with around 600 elements.

from AKDSFramework.applications.decorators import benchmark
from AKDSFramework.structure import MaxHeap

@benchmark
def buildHeap(array):
    h = MaxHeap(array)
    h.build()

    h.add(68)
    h.add(13)
    h.build()

buildHeap([data**2 for data in range(601)])

Notice the @benchmark decorator at the beginning of the declaration of the function, that calls cProfile to start calculating what taking what.

Now running the code will output a report in the console like this:

       3597 function calls (3003 primitive calls) in 0.002 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.002    0.002 <ipython-input-18-1d08ee399432>:4(buildHeap)
        2    0.000    0.000    0.002    0.001 /opt/venv/lib/python3.7/site-packages/AKDSFramework/structure/heap.py:136(build)
 1195/601    0.001    0.000    0.001    0.000 /opt/venv/lib/python3.7/site-packages/AKDSFramework/structure/heap.py:153(heapify)
     1195    0.000    0.000    0.000    0.000 /opt/venv/lib/python3.7/site-packages/AKDSFramework/structure/heap.py:67(get_left_child)
     1195    0.000    0.000    0.000    0.000 /opt/venv/lib/python3.7/site-packages/AKDSFramework/structure/heap.py:53(get_right_child)
        2    0.000    0.000    0.000    0.000 /opt/venv/lib/python3.7/site-packages/AKDSFramework/structure/heap.py:26(add)
        1    0.000    0.000    0.000    0.000 /opt/venv/lib/python3.7/site-packages/AKDSFramework/structure/heap.py:128(__init__)
        1    0.000    0.000    0.000    0.000 /opt/venv/lib/python3.7/site-packages/AKDSFramework/structure/heap.py:21(__init__)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        2    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
        2    0.000    0.000    0.000    0.000 {built-in method builtins.len}

This has all the call's report and how much time it's taking. If you see the second last function call {method 'append' of 'list' objects} see that's called 2 times total as we are appending 2 elements.

So this way you can see how much each function taking time and how many times they are called. If you wish you can reduce the number of calls or use a different approach to solve the part where it's slow.

AKDSFramework's all implementations of data structures and algorithms are super optimized so you can't find any bottle neck when using @benchmark on our function calls.

If you don't wish to install AKDSFramework here is the @benchmark decorator source code:

import cProfile
import pstats
import io

def benchmark(func):
    """
    AKDSFramework default benchmark profiler. Implemented with cProfile and pstats.
    """
    def profiler(*args, **kwargs):
        profiler = cProfile.Profile()
        profiler.enable()
        returnvalue = func(*args, **kwargs)
        profiler.disable()

        stringIO = io.StringIO()

        ps = pstats.Stats(profiler, stream=stringIO).sort_stats("cumulative")
        ps.print_stats()
        print(stringIO.getvalue())

        return returnvalue

    return profiler

What is PyTorch `.detach()` method?

theroyakash — Sat, 14 Nov 2020 18:37:14 +0000

What is PyTorch .detach() method?

PyTorch's detach method works on the tensor class.

tensor.detach() creates a tensor that shares storage with tensor that does not require gradient. tensor.clone() creates a copy of tensor that imitates the original tensor's requires_grad field.

You should use detach() when attempting to remove a tensor from a computation graph, and clone as a way to copy the tensor while still keeping the copy as a part of the computation graph it came from.

Let's see that in an example here

X = torch.ones((28, 28), dtype=torch.float32, requires_grad=True)
y = X**2
z = X**2

result = (y+z).sum()

torchviz.make_dot(result).render('Attached', format='png')

And now one with the detach.

X = torch.ones((28, 28), dtype=torch.float32, requires_grad=True)
y = X**2
z = X.detach()**2

result = (y+z).sum()

torchviz.make_dot(result).render('Attached', format='png')

As you can see now that the branch of computation with x**2 is no longer tracked. This is reflected in the gradient of the result which no longer records the contribution of this branch

Forem: theroyakash

How to benchmark your python program?

Generating reports

Example implementation

What is PyTorch `.detach()` method?