How to call a function on a running Python thread

Question:

Say I have this class that spawns a thread:

import threading
class SomeClass(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)

    def run(self):
        while True:
            pass

    def doSomething(self):
        pass

    def doSomethingElse(self):
        pass

I want to

someClass = SomeClass()
someClass.start()
someClass.doSomething()
someClass.doSomethingElse()
someClass.doSomething()

How can I do this? I know I can call a function inside the run() function, but that’s not what I am aiming for.

Asked By: Chris F

||

Answers:

Look into concurrent.futures and its submit method, which does what you want when you limit the thread pool to one worker.

Answered By: jhermann

You can’t directly do what you want. The background thread is running its run function, which just loops forever, so it can’t possibly do anything else.

You can, of course, call the class’s methods on your own thread, but that presumably isn’t what you want here.


The reason frameworks like Qt, .NET, or Cocoa can offer runOnOtherThread-type methods is that each thread runs an “event loop”, so all they’re really doing is posting an event. You can do that yourself, if you rewrite the run method into an event loop. For example:

import queue
import threading

class SomeClass(threading.Thread):
    def __init__(self, q, loop_time = 1.0/60):
        self.q = q
        self.timeout = loop_time
        super(SomeClass, self).__init__()

    def onThread(self, function, *args, **kwargs):
        self.q.put((function, args, kwargs))

    def run(self):
        while True:
            try:
                function, args, kwargs = self.q.get(timeout=self.timeout)
                function(*args, **kwargs)
            except queue.Empty:
                self.idle()

    def idle(self):
        # put the code you would have put in the `run` loop here 

    def doSomething(self):
        pass

    def doSomethingElse(self):
        pass

Now, you can do this:

someClass = SomeClass()
someClass.start()
someClass.onThread(someClass.doSomething)
someClass.onThread(someClass.doSomethingElse)
someClass.onThread(someClass.doSomething)

If you want to simplify the calling interface a bit, at the cost of more code in the class, you can add wrapper methods like this:

    def _doSomething(self):
        # put the real code here
    def doSomething(self):
        self.onThread(self._doSomething)

However, unless your idle method has work to do, you’re really just building the equivalent of a single-thread thread pool here, and there are much easier ways to do this than to build it from scratch. For example, using the futures module off PyPI (a backport of the Python 3 concurrent.futures module):

import futures

class SomeClass(object):
    def doSomething(self):
        pass
    def doSomethingElse(self):
        pass

someClass = SomeClass()
with futures.ThreadPoolExecutor(1) as executor:
    executor.submit(someClass.doSomething)
    executor.submit(someClass.doSomethingElse)
    executor.submit(someClass.doSomething)

Or, with just the stdlib:

from multiprocessing import dummy as multithreading

class SomeClass(object):
    def doSomething(self):
        pass
    def doSomethingElse(self):
        pass

someClass = SomeClass()
pool = multithreading.Pool(1)
pool.apply(someClass.doSomething)
pool.apply(someClass.doSomethingElse)
pool.apply(someClass.doSomething)
pool.close()
pool.join()

Pools have some other advantages, and executors even more. For example, what if the methods returned values, and you want to kick off two functions, then wait for the results, then kick off a third with the results of the first two? Easy:

with futures.ThreadPoolExecutor(1) as executor:
    f1 = executor.submit(someClass.doSomething)
    f2 = executor.submit(someClass.doSomethingElse)
    futures.wait((f1, f2))
    f3 = executor.submit(someClass.doSomethingElser, f1.result(), f2.result())
    result = f3.result()

Even if you later switch to a pool of 4 threads, so f1 and f2 may be waiting concurrently and f2 may even return first, you’re guaranteed to kick off doSomethingElser as soon as both of them are finished, and no sooner.


There’s another possibility here. Do you really need the code to run in that thread, or do you just need it to modify variables that thread depends on? If it’s the latter, just synchronize access to the variables. For example:

class SomeClass(threading.Thread):
    def __init__(self):
        self.things_lock = threading.Lock()
        self.things = []
        while True:
            with self.lock:
                things = self.things[:]
            for thing in things:
                # pass
    def doSomething(self):
        with self.lock:
            self.things.append(0)

someClass = SomeClass()
someClass.start()
someClass.doSomething()

There’s nothing magical about being on the main thread here. If, in addition to needing to modify variables that SomeClass depends on, you also wanted to just kick doSomething off the main thread so you can do more important things than just waiting around for it to finish, you can create a short-lived extra thread just to doSomething:

someClass = SomeClass()
someClass.start()
somethingThread = threading.Thread(target=someClass.doSomething)
somethingThread.start()
doOtherImportantStuffWithSomethingIsHappening()
somethingThread.join()
Answered By: abarnert
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.