Python ffmpeg: overlaying videos is dropping all audio

Question

I am using ffmpeg-python (source) to create the effect where you add a blurred background to fill in the sides of a tall vertical video as shown below:

The problem is that the output has no audio attached. Since the clips are the same, I want to keep the audio from one of the clips in the final output. How can I keep the audio? (I don’t want to overlay the audio from both and get an echo effect, however!)

This is the function I’m using:

import ffmpeg
def add_blurred_bg():
    HEIGHT = 720 
    WIDTH = 1280
    in_file = ffmpeg.input('input.mp4')
    probe = ffmpeg.probe('input.mp4')
    video_stream = next((stream for stream in probe['streams'] if stream['codec_type'] == 'video'), None)
    iw=int(video_stream['width'])
    ih=int(video_stream['height'])
    nw = HEIGHT*iw/ih
    (
        ffmpeg
        .overlay(
            in_file.filter('scale', WIDTH, -2).crop(0,(WIDTH*HEIGHT/nw-HEIGHT)/2,WIDTH,HEIGHT).filter('gblur', sigma=40),
            in_file.filter('scale', -2, HEIGHT),
            x=(WIDTH-nw)/2
        )
        .output('output.mp4')
        .run()
    )

Asked By: swagrov

||

Source

Answer 1

Do you have to do it keep the audio style? Can you mix the audio and video? If so, this is a messy, but functioning example:

import ffmpeg
import os

def add_blurred_bg():
    HEIGHT = 720 
    WIDTH = 1280
    inp = 'input.mp4'
    os.system("ffmpeg -i " + inp + " -f mp3 -ab 192000 -vn music.mp3")
    print("extracting audio...")
    in_file = ffmpeg.input(inp)
    probe = ffmpeg.probe('input.mp4')
    video_stream = next((stream for stream in probe['streams'] if stream['codec_type'] == 'video'), None)
    iw=int(video_stream['width'])
    ih=int(video_stream['height'])
    nw = HEIGHT*iw/ih
    (
       ffmpeg
       .overlay(
            in_file.filter('scale', WIDTH, -2).crop(0,(WIDTH*HEIGHT/nw-HEIGHT)/2,WIDTH,HEIGHT).filter('gblur', sigma=40),
            in_file.filter('scale', -2, HEIGHT),
            x=(WIDTH-nw)/2
        )
        .output('outputPartial.mp4')
        .run()
    )
    print("bluring...")
    os.system("ffmpeg -i outputPartial.mp4 -i music.mp3 -shortest -c:v copy -c:a aac -b:a 256k output.mp4")
    print("mixing...")
    os.remove("outputPartial.mp4")
    os.remove("music.mp3")
    print("cleaning up...")
    print("done!")

I don’t know why you have that problem, but here is a workaround.

STEP 1: Extract the music

STEP 2: Blur the video

STEP 3: Mix the audio and the video

STEP 4: Clean-Up

Answered By: xilpex

Answer 2

The other answer will work, but there’s a less complex approach that will fix the audio.

Change this line:

.output('output.mp4')

to this:

.output(in_file.audio, 'output.mp4')

Explanation

When ffmpeg is given a -map option, it drops all audio/video streams which are not explicitly part of the map statement. Internally, ffmpeg-python uses map to implement overlays, and does not explicitly map the audio. See also FFMPEG overlaying video with image removes audio.

So, the fix is to explicitly map the input audio to the output.

Full working code:

import ffmpeg
def add_blurred_bg():
    HEIGHT = 720 
    WIDTH = 1280
    in_file = ffmpeg.input('input.mp4')
    probe = ffmpeg.probe('input.mp4')
    video_stream = next((stream for stream in probe['streams'] if stream['codec_type'] == 'video'), None)
    iw=int(video_stream['width'])
    ih=int(video_stream['height'])
    nw = HEIGHT*iw/ih
    (
        ffmpeg
        .overlay(
            in_file.filter('scale', WIDTH, -2).crop(0,(WIDTH*HEIGHT/nw-HEIGHT)/2,WIDTH,HEIGHT).filter('gblur', sigma=40),
            in_file.filter('scale', -2, HEIGHT),
            x=(WIDTH-nw)/2
        )
        .output(in_file.audio, 'output.mp4')
        .run()
    )

Answered By: Nick ODell

Python ffmpeg: overlaying videos is dropping all audio

Question:

Answers:

Explanation