How to get python logging output in shell script

Question

I would want to get the stdout of the python file into shell script. I initially used print and it worked fine but it is not working for logging.

ab.py

import logging
logger = logging.getLogger()
logging.basicConfig(level=logging.INFO)

logging.info('2222222222222')
print("111111111111")

cd.sh

#/bin/bash
set -e
set -x

communities=$(python3 ab.py) 
echo $communities

The output of executing cd.sh I am getting the output only as 111111111111 and not 2222222222222

Asked By: Sriram Chowdary

||

Source

Answer 1

logging prints to stderr and not stdout. These are 2 separate streams of data. You can manage these by learning up a bit on Input/Output Redirection. But, it is kind of a big topic to summarize here.

tl/dr: Make your command be python3 ab.py 2>&1 and you will get the output you want, kinda. You’ll see that the logging messages appear before the print messages. This is because stderr is unbuffered and stdout is buffered.

Answered By: Cargo23

Answer 2

By default, the Python logging system logs to stderr. You are capturing just stdout. To configure the Python logging system to output to stdout, which it sounds like is what you want, you can slightly modify your code, as follows:

import logging
import sys

logger = logging.getLogger()
logging.basicConfig(level=logging.INFO, stream=sys.stdout)

logging.info('2222222222222')
print("111111111111")

With this, both lines of output go to stdout and so your shell script will catch both of them.

Answered By: CryptoFool

How to get python logging output in shell script

Question:

Answers: