Python subprocess, realtime print with COLORS and save stdout

Question

Printing the output of a subprocess while saving the result is not a new problem, and has been answered many times before e.g.: https://stackoverflow.com/a/28319191/5506400
This does not work for me because I am trying to maintain the shell colours printed. E.g. when one goes systemctl status application, its prints running in green.
The above-mentioned methods all rely on reading a line one by one from subprocess, but it seems to me by then the colour information is stripped off and lost.

I tried to make an object which tee’s off the stdout prints and saves them into a variable:

from subprocess import *
import sys

class Tee():
    def __init__(self):
        self.content = ''
        self.stdout = sys.stdout
        sys.stdout = self
    def __enter__(self):
        return self
    def __exit__(self, *args):
        pass
    def __del__(self):
        sys.stdout = self.stdout
    def write(self, data):
        self.content += data
        self.stdout.write(data)
    def flush(self):
        self.content = ''

with Tee() as tee:
    # Saves print to tee.content
    print("Hello World")

    # This line does not save prints to tee.content    
    run(['apt-get', 'update'])

    # raises an error that tee.fileno is not supported
    run(['systemctl', 'status', 'nginx'], stdout=tee)

    content = tee.content

print("---------------------")
print(content)

But the problem is subprocess’s stdout requires an actual file: https://stackoverflow.com/a/2298003/5506400

Is there anyway to print realtime the output of a subprocess, while maintaining the colours, and store the value to a variable (without going through a temp file)?

Asked By: run_the_race

||

Source

Answer 1

systemctl checks where the output is going; if it’s a tty, it shows colorized output. If the STDOUT is not attached to a tty, it does not show the color.

So, essentially you need to do this from source i.e. make systemctl emit necessary escape codes when the STDOUT is not a tty.

There is a way, from man systemd:

$SYSTEMD_COLORS
    Controls whether colorized output should be generated.

So you need to pass the SYSTEMD_COLORS environment variable to make systmectl return output with color escapes.

You can do:

os.environ['SYSTEMD_COLORS'] = '1'
subprocess.run(['systemctl', 'status', 'application'], stdout=subprocess.PIPE)

To have the env var for this command only do del os.environ['SYSTEMD_COLORS'] afterwards.

Or run on shell directly for single command only:

subprocess.run('SYSTEMD_COLORS=1 systemctl status application', shell=True, stdout=subprocess.PIPE)

Answered By: heemayl

Answer 2

You can’t make it with subprocess, but pty can. pty creates pseudo-terminal so the command being executed detects that it’s running with tty and enables color output.

import pty, os

output_bytes = []

def read(fd):
    data = os.read(fd, 1024)
    output_bytes.append(data)
    return data

pty.spawn([command], read)
output = str(output_bytes)
# parse output as you need

Answered By: aleung

Answer 3

Execute command in list form with live colorised output:

cmd_parts = ['mycommand', 'arg1', 'arg2']

import os, pty
def exec_live_output(cmd_parts):
    pty.spawn(cmd_parts, lambda fd: os.read(fd, 1024))

Answered By: kwiknik

Python subprocess, realtime print with COLORS and save stdout

Question:

Answers: