Threading decorator is not callable

Question

I am implementing a thread as a decorator to a class which should apply threads to the method, all concurrently. However, I get the following error:

TypeError: ‘TEST_THREAD’ object is not callable

The example below should print out each iteration over the maximum thread number.

def start_workload(NUM_THREADS):
    def wrapper(fn, *args):
        thread = []
        for i in range(*args):
            t = threading.Thread(target=fn, args=(i,))
        #t = threading.Thread(target=do_query, args=(i,))
            t.start()
            thread.append(t)
        for i in range(*args):
            thread[i].join()
    return wrapper

class TEST_THREAD(object):
    def __init__(self, *args):
        super().__init__()
        self._args = args
    @start_workload
    def print(self, threads):
        print(threads, self._args)

if __name__ == '__main__':
    test = TEST_THREAD(*list([1, 2, 3, 4, 5]))
    test.print(5)

I was expecting the wrapper to perform the same functionality like the following approach:

class TEST_THREAD:
    def __init__(self, *args):
        super().__init__()
        self._args = args
    
    def print(self, threads):
        print(threads, self._args)

def start_workload(fn, num_thread):
    thread = []
    print(num_thread)
    for i in range(num_thread):
        t = threading.Thread(target=fn, args=(i,))
        t.start()
        thread.append(t)
    for i in range(num_thread):
        thread[i].join()

if __name__ == '__main__':
    test = TEST_THREAD(*list([1, 2, 3, 4, 5]))
    start_workload(test.print, 5)

Expected output:

0 (1, 2, 3, 4, 5)
1 (1, 2, 3, 4, 5)
2 (1, 2, 3, 4, 5)
3 (1, 2, 3, 4, 5)
4 (1, 2, 3, 4, 5)

Asked By: Emil11

||

Source

Answer 1

When using a decorator without parens @start_workload def ..., the function is used as the argument to the decorator function. So in this case you assigned TEST_THREAD.print as NUM_THREADS in start_workload.

This decorator then returns a function that takes two arguments. Because you are wrapping a class method, the first argument provided will be self, so calling test.print(5) is really calling wrapper(fn, *args) where fn is self and *args is (5,).

You then end up passing fn (which is self, the instance of TEST_THREAD) to threading.Thread(target=fn, args=(i,)). This results in the error because it is using that instance as a callable to threading.

Its unclear exactly what you want to do here, but something like this is probably closer.

Don’t use a parameterizable decorator, the number of threads is being passed through the function call as normal arguments.
You want to use the threads argument to be the total number of threads used. You also want to change the threads argument being passed to the method to be the index of the thread. This means we’ll need to modify the arguments being proxied to the call to threading.Thread.
You’re printing from multiple threads, so you probably want to use a method that wont have outputs being jumbled on top of eachother. The logging module would be good here.

import logging
import threading

def start_workload(fn):
    def wrapped(self, threads, *args, **kwargs):
        assert isinstance(threads, int)
        assert threads > 0

        ts = []
        for i in range(threads):
            new_args = (self, i, *args)
            t = threading.Thread(target=fn, args=new_args, kwargs=kwargs)
            t.start()
            ts.append(t)
        for t in ts:
            t.join()
    return wrapped

class TEST_THREAD(object):
    def __init__(self, *args):
        super().__init__()
        self._args = args

    @start_workload
    def print(self, threads):
        logging.info(f"{threads}, {self._args}")

if __name__ == '__main__':
    logging.basicConfig(level=logging.INFO)
    test = TEST_THREAD(1, 2, 3, 4, 5)
    test.print(5)

INFO:root:0, (1, 2, 3, 4, 5)
INFO:root:1, (1, 2, 3, 4, 5)
INFO:root:2, (1, 2, 3, 4, 5)
INFO:root:3, (1, 2, 3, 4, 5)
INFO:root:4, (1, 2, 3, 4, 5)

Answered By: flakes

Threading decorator is not callable

Question:

Answers: