Python CSV module returning non string iterator when passed the STDOUT stream containing CSV data

Question

The following program calls subprocess.run() to invoke a program (ffprobe, but that’s not important) and return a CSV result in STDOUT. I am getting an error enumerating the CSV result.

import os
import subprocess
import csv

for file in os.listdir("/Temp/Video"):
    if file.endswith(".mkv"):
        print(os.path.join("/Temp/Video", file))
        ps = subprocess.run(["ffprobe", "-show_streams", "-print_format", "csv",  "-i", "/Temp/Video/" + file], capture_output = True)
        print(ps.stdout)
        reader = csv.reader(ps.stdout)
        results = []
        for row in reader:
            results.append(row)

Produces the following error:

Traceback (most recent call last):
  File "C:Usersmbmaseclipse-workspaceDolbyAtmosToFlaccommbmmain.py", line 12, in <module>
    for row in reader:
_csv.Error: iterator should return strings, not int (the file should be opened in text mode)

The output from the print(ps.stdout) statement produces:

b'stream,0,h264,H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10,High,video,[0][0][0][0],0x0000,1920,1080,1920,1080,0,0,2,1:1,16:9,yuv420p,40,unknown,unknown,unknown,unknown,left,progressive,1,true,4,N/A,24000/1001,24000/1001,1/1000,0,0.000000,N/A,N/A,N/A,N/A,8,N/A,N/A,N/A,46,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,eng,17936509,01:20:18.271791666,115523,10802870592,001011,MakeMKV v1.16.4 win(x64-release),2021-08-20 19:09:26,BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES SOURCE_ID,Lavc59.7.102 libx264,00:01:30.010000000rnstream,1,vorbis,Vorbis,unknown,audio,[0][0][0][0],0x0000,fltp,48000,3,3.0,0,0,N/A,0/0,0/0,1/1000,0,0.000000,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,3314,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,eng,Surround 3.0,2422660,01:20:18.272000000,451713,1459129736,001100,MakeMKV v1.16.4 win(x64-release),2021-08-20 19:09:26,BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES SOURCE_ID,Lavc59.7.102 libvorbis,00:01:30.003000000rn'

The suggestion Python provides, the file should be opened in text mode, is confusing. The above output looks like text to me. How do I use the csv module to read CSV data held in a stream?

Asked By: mbmast

||

Source

Answer 1

subprocess.run is returning bytes:

b'stream,0,h264,H.264 ,,,'

To have it return text, pass text=True:

ps = subprocess.run(
    ["ffprobe", "-show_streams", "-print_format", "csv",  "-i", "/Temp/Video/" + file],
    capture_output=True,
    text=True
)

Answered By: snakecharmerb

Python CSV module returning non string iterator when passed the STDOUT stream containing CSV data

Question:

Answers: