Unable to print TCPcump information using python subprocess

Question

I wanted to process tcpdump output in a python script and so far I was able to get to this implementation

from subprocess import Popen, PIPE, CalledProcessError
import os
import signal
import time

if __name__=="__main__":
    
    cmd = ["sudo","tcpdump", "-c","1000","-i","any","port","22","-n"]
    with Popen(cmd, stdout=PIPE, bufsize=1, universal_newlines=True) as p:
        try:
            for line in p.stdout:
                print(line,flush=True) # process line here   
    
        except KeyboardInterrupt:       
            print("Quitting")

This is what I uderstood from the second answer of this previously asked question.
While it is not waiting for the subprocess to complete to print the output of the tcpdump, I still get the output in chunks of 20-30 lines at a time. Is there a way to read even if there is a single line in stdout pf the subprocess?

PS: I am running this script on a raspberry Pi 4 with ubuntu server 22.04.1

Asked By: Vibhore Jain

||

Source

Answer 1

tcpdump uses a larger buffer if you connect its standard output to a pipe. You can easily see this by running the following two commands. (I changed the count from 1000 to 40 and removed port 22 in order to quickly get output on my system.)

$ sudo tcpdump -c 40 -i any -n
$ sudo tcpdump -c 40 -i any -n | cat

The first command prints one line at a time. The second collects everything in a buffer and prints everything when tcpdump exits. The solution is to tell tcpdump to use line buffering with the -l argument.

$ sudo tcpdump -l -c 40 -i any -n | cat

Do the same in your Python program.

import subprocess

cmd = ["sudo", "tcpdump", "-l", "-c", "40", "-i", "any", "-n"]
with subprocess.Popen(cmd, stdout=subprocess.PIPE, bufsize=0, text=True) as p:
    for line in p.stdout:
        print(line.strip())

When I run this, I get one line printed at a time.

Note that universal_newlines is a backward-compatible alias for text, so the latter should be preferred.

Answered By: tfpf

Unable to print TCPcump information using python subprocess

Question:

Answers: