How to read mp4 video to be processed by scikit-image?

Question

I would like to apply a scikit-image function (specifically the template matching function match_template) to the frames of a mp4 video, h264 encoding. It’s important for my application to track the time of each frame, but I know the framerate so I can easily calculate from the frame number.

Please note that I’m running on low resources, and I would like to keep dependencies as slim as possible: numpy is needed anyway, and since I’m planning to use scikit-image, I would avoid importing (and compiling) openCV just to read the video.

I see at the bottom of this page that scikit-image can seamleassly process video stored as a numpy array, obtaining that would thus be ideal.

Asked By: gaggio

||

Source

Answer 1

Imageio python package should do what you want. Here is a python snippet using this package:

import pylab
import imageio
filename = '/tmp/file.mp4'
vid = imageio.get_reader(filename,  'ffmpeg')
nums = [10, 287]
for num in nums:
    image = vid.get_data(num)
    fig = pylab.figure()
    fig.suptitle('image #{}'.format(num), fontsize=20)
    pylab.imshow(image)
pylab.show()

enter image description here

You can also directly iterate over the images in the file (see the documentation ):

for i, im in enumerate(vid):
    print('Mean of frame %i is %1.1f' % (i, im.mean()))

To install imageio you can use pip:

pip install imageio

An other solution would be to use moviepy (which use a similar code to read video), but I think imageio is lighter and does the job.

response to first comment

In order to check if the nominal frame rate is the same over the whole file, you can count the number of frame in the iterator:

count = 0
try:
    for _ in vid:
        count += 1
except RuntimeError:
    print('something went wront in iterating, maybee wrong fps number')
finally:
    print('number of frames counted {}, number of frames in metada {}'.format(count, vid.get_meta_data()['nframes']))


In [10]: something went wront in iterating, maybee wrong fps number
         number of frames counted 454, number of frames in metada 461

In order to display the timestamp of each frame:

try:
    for num, image in enumerate(vid.iter_data()):
        if num % int(vid._meta['fps']):
            continue
        else:
            fig = pylab.figure()
            pylab.imshow(image)
            timestamp = float(num)/ vid.get_meta_data()['fps']
            print(timestamp)
            fig.suptitle('image #{}, timestamp={}'.format(num, timestamp), fontsize=20)
            pylab.show()
except RuntimeError:
    print('something went wrong')

Answered By: head7

Answer 2

You could use scikit-video, like this:

from skvideo.io import VideoCapture

cap = VideoCapture(filename)
cap.open()

while True:
    retval, image = cap.read()
    # image is a numpy array containing the next frame
    # do something with image here
    if not retval:
        break

This uses avconv or ffmpeg under the hood. The performance is quite good, with a small overhead to move the data into python compared to just decoding the video in avconv.

The advantage of scikit-video is that the API is exactly the same as the video reading/writing API of OpenCV; just replace cv2.VideoCapture with skvideo.io.VideoCapture.

Answered By: Alex I

Answer 3

An easy way to read video in python is using skviode. A single line code can help to read entire video.

import skvideo.io  
videodata = skvideo.io.vread("video_file_name")  
print(videodata.shape)

http://mllearners.blogspot.in/2018/01/scikit-video-skvideo-tutorial-for.html

Answered By: Win GATE ECE

How to read mp4 video to be processed by scikit-image?

Question:

Answers: