Multiple stdout w/ flush going on in Python threading

Question

I have a small piece of code that I made to test out and hopefully debug the problem without having to modify the code in my main applet in Python. This has let me to build this code:

#!/usr/bin/env python
import sys, threading, time

def loop1():
    count = 0
    while True:
        sys.stdout.write('r thread 1: ' + str(count))
        sys.stdout.flush()
        count = count + 1
        time.sleep(.3)
        pass
    pass

def loop2():
    count = 0
    print ""
    while True:
        sys.stdout.write('r thread 2: ' + str(count))
        sys.stdout.flush()
        count = count + 2
        time.sleep(.3)
    pass

if __name__ == '__main__':
    try:
        th = threading.Thread(target=loop1)
        th.start()

        th1 = threading.Thread(target=loop2)
        th1.start()
        pass
    except KeyboardInterrupt:
        print ""
        pass
    pass

My goal with this code is to be able to have both of these threads displaying output in stdout format (with flushing) at the same time and have then side by side or something. problem is that I assume since it is flushing each one, it flushes the other string by default. I don’t quite know how to get this to work if it is even possible.

If you just run one of the threads, it works fine. However I want to be able to run both threads with their own string running at the same time in the terminal output. Here is a picture displaying what I’m getting:

terminal screenshot

let me know if you need more info. thanks in advance.

Asked By: BlackVikingPro

||

Source

Answer 1

Instead of allowing each thread to output to stdout, a better solution is to have one thread control stdout exclusively. Then provide a threadsafe channel for the other threads to dispatch data to be output.

One good method to achieve this is to share a Queue between all threads. Ensure that only the output thread is accessing data after it has been added to the queue.

The output thread can store the last message from each other thread and use that data to format stdout nicely. This can include clearing output to display something like this, and update it as each thread generates new data.

Threads
#1: 0
#2: 0

Example

Some decisions were made to simplify this example:

There are gotchas to be wary of when giving arguments to threads.
Daemon threads terminate themselves when the main thread exits. They are used to avoid adding complexity to this answer. Using them on long-running or large applications can pose problems. Other
questions discuss how to exit a multithreaded application without leaking memory or locking system resources. You will need to think about how your program needs to signal an exit. Consider using asyncio to save yourself these considerations.
No newlines are used because r carriage returns cannot clear the whole console. They only allow the current line to be rewritten.

import queue, threading
import time, sys

q = queue.Queue()
keepRunning = True

def loop_output():
    thread_outputs = dict()

    while keepRunning:
        try:
            thread_id, data = q.get_nowait()
            thread_outputs[thread_id] = data
        except queue.Empty:
            # because the queue is used to update, there's no need to wait or block.
            pass

        pretty_output = ""
        for thread_id, data in thread_outputs.items():
            pretty_output += '({}:{}) '.format(thread_id, str(data))

        sys.stdout.write('r' + pretty_output)
        sys.stdout.flush()
        time.sleep(1)

def loop_count(thread_id, increment):
    count = 0
    while keepRunning:
        msg = (thread_id, count)
        try:
            q.put_nowait(msg)
        except queue.Full:
            pass

        count = count + increment
        time.sleep(.3)
        pass
    pass

if __name__ == '__main__':
    try:
        th_out = threading.Thread(target=loop_output)
        th_out.start()

        # make sure to use args, not pass arguments directly
        th0 = threading.Thread(target=loop_count, args=("Thread0", 1))
        th0.daemon = True
        th0.start()

        th1 = threading.Thread(target=loop_count, args=("Thread1", 3))
        th1.daemon = True
        th1.start()

        # Keep the main thread alive to wait for KeyboardInterrupt
        while True:
            time.sleep(.1)

    except KeyboardInterrupt:
        print("Ended by keyboard stroke")
        keepRunning = False
        for th in [th0, th1]:
            th.join()

Example Output:

(Thread0:110) (Thread1:330)

Answered By: Aaron3468

Multiple stdout w/ flush going on in Python threading

Question:

Answers:

Example