How to convert output from subprocess to list of dictionaries
Question:
I want to take the output of a Python script and convert it into list of dictionaries to perform some action. I have written a below code:
import subprocess
import json
chief_complaint = """ "Eernise, Randell is an 64-year-old male" """
command= """python predict_icd.py --cc """+str(chief_complaint)
command_proc=subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)
print("response - ",command_proc.stdout.read())
The output that I am getting of above code is:
command_proc.stdout.read() - b"[{'text': 'CHA', 'start': 51, 'end': 54, 'code': ''}, {'text': 'CHA', 'start': 221, 'end': 224, 'code': ''}, {'text': 'dog', 'start': 286, 'end': 289, 'code': ''}, {'text': 'Schizophrenia', 'start': 351, 'end': 364, 'code': 'F20'}, {'text': 'chronic pain', 'start': 366, 'end': 378, 'code': 'G89.29'}, {'text': 'chest pain', 'start': 599, 'end': 609, 'code': 'R07.89'}, {'text': 'palpitations', 'start': 613, 'end': 625, 'code': 'R00.2'}, {'text': 'cough', 'start': 829, 'end': 834, 'code': 'R05'}, {'text': 'wheezing', 'start': 836, 'end': 844, 'code': 'R06.2'}, {'text': 'pain', 'start': 966, 'end': 970, 'code': ''}, {'text': 'schizophrenia', 'start': 1078, 'end': 1091, 'code': 'F20'}]n"
Answers:
The absolutely best solution is to not run Python as a subprocess of itself. Instead, refactor predict_icd.py so you can import it and call its functions directly. See What is the best way to call a script from another script?
Your second best option is to change predict_icd.py so that it prints its results in a standard format like JSON. Then you can call it from any tool, not just Python.
The distant third option is to go with what you have and use ast.literal_eval() to parse out the results. I have also refactored to avoid the pesky and unnecessary shell=True and fix the orphaned Popen.
import subprocess
# import json # unused import
from ast import literal_eval
chief_complaint = """ "Eernise, Randell is an 64-year-old male" """
command_proc = subprocess.run(
['python', 'predict_icd.py', '--cc', chief_complaint],
capture_output=True, text=True, check=True)
response = literal_eval(command_proc.stdout)
print("response = ", response)
In some more detail, Popen will only start a process; you have to take care of it during its lifetime if you do that – but subprocess.run() already takes care of all this plumbing for you. Like its documentation says, avoid Popen when the higher-level functions from the standard library already do what you want.
Notice also the text=True which tells Python to send and receive text, not bytes to and from the subprocess.
I want to take the output of a Python script and convert it into list of dictionaries to perform some action. I have written a below code:
import subprocess
import json
chief_complaint = """ "Eernise, Randell is an 64-year-old male" """
command= """python predict_icd.py --cc """+str(chief_complaint)
command_proc=subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)
print("response - ",command_proc.stdout.read())
The output that I am getting of above code is:
command_proc.stdout.read() - b"[{'text': 'CHA', 'start': 51, 'end': 54, 'code': ''}, {'text': 'CHA', 'start': 221, 'end': 224, 'code': ''}, {'text': 'dog', 'start': 286, 'end': 289, 'code': ''}, {'text': 'Schizophrenia', 'start': 351, 'end': 364, 'code': 'F20'}, {'text': 'chronic pain', 'start': 366, 'end': 378, 'code': 'G89.29'}, {'text': 'chest pain', 'start': 599, 'end': 609, 'code': 'R07.89'}, {'text': 'palpitations', 'start': 613, 'end': 625, 'code': 'R00.2'}, {'text': 'cough', 'start': 829, 'end': 834, 'code': 'R05'}, {'text': 'wheezing', 'start': 836, 'end': 844, 'code': 'R06.2'}, {'text': 'pain', 'start': 966, 'end': 970, 'code': ''}, {'text': 'schizophrenia', 'start': 1078, 'end': 1091, 'code': 'F20'}]n"
The absolutely best solution is to not run Python as a subprocess of itself. Instead, refactor predict_icd.py so you can import it and call its functions directly. See What is the best way to call a script from another script?
Your second best option is to change predict_icd.py so that it prints its results in a standard format like JSON. Then you can call it from any tool, not just Python.
The distant third option is to go with what you have and use ast.literal_eval() to parse out the results. I have also refactored to avoid the pesky and unnecessary shell=True and fix the orphaned Popen.
import subprocess
# import json # unused import
from ast import literal_eval
chief_complaint = """ "Eernise, Randell is an 64-year-old male" """
command_proc = subprocess.run(
['python', 'predict_icd.py', '--cc', chief_complaint],
capture_output=True, text=True, check=True)
response = literal_eval(command_proc.stdout)
print("response = ", response)
In some more detail, Popen will only start a process; you have to take care of it during its lifetime if you do that – but subprocess.run() already takes care of all this plumbing for you. Like its documentation says, avoid Popen when the higher-level functions from the standard library already do what you want.
Notice also the text=True which tells Python to send and receive text, not bytes to and from the subprocess.