ffmpeg Python command only runs once in PM2 environment
Question:
PM2 is running as a web
user. ffmpeg was installed native to Ubuntu 16.04 LTS using sudo apt install ffmpeg
. The Python version is 3.6. The software uses [email protected].
The applications spawned produce no errors. When the ffmpeg code executes for the first time, we see an output and the ffmpeg process completes the task as expected.
All subsequent requests stall on the next ffmpeg execution. No output. No return from the ffmpeg process. No errors. The PM2 process does not error out. The application log stalls on the ffmpeg command as though it is hung.
What is the root cause? Any help is deeply appreciated.
Furthermore, what are the reasons PM2 hangs on a subprocess (like ffmpeg)?
Here is the code:
class ImageHelper:
def __init__(self):
pass
@classmethod
def create_thumb_from_video_ffmpeg(cls, input_video_file_path,
output_image_path,
scale_width,
scale_height
):
"""
This function is used to create the thumb image
from a source video file.
We are using a python wrapper/library for FFMPEG
"""
try:
if Functions.get_attribute_env('ENVIRONMENT') == 'prod':
out, err = (
ffmpeg
.input(input_video_file_path, ss="00:00:00.001")
.filter('scale', scale_width, scale_height)
.output(output_image_path, vframes=1, loglevel='quiet')
.overwrite_output()
.run(capture_stdout=True)
)
print("We only see this once!")
else:
out, err = (
ffmpeg
.input(input_video_file_path, ss="00:00:00.001")
.filter('scale', scale_width, scale_height)
.output(output_image_path, vframes=1)
.overwrite_output()
.run(capture_stdout=True)
)
print("We only see this once!")
if err:
if Functions.get_attribute_env('ENVIRONMENT') != 'prod':
print('ffmpeg video thumb', err)
else:
Functions.logger_function(str(err))
raise Exception(err)
else:
return output_image_path
except Exception as e:
if Functions.get_attribute_env('ENVIRONMENT') != 'prod':
print('in thumb exception', e)
else:
Functions.logger_function(str(e))
raise Exception(e)
Answers:
Check whether the ffmpeg
process is running while the sequential request is made or not. If it is, you may want to make sure the process is closed after the first completion so it can start again for the sequential request.
Processes spawned by apache
or nginx
will have limits on how long they execute and will be killed automatically. In those cases you’ll probably want to have the script triggered to run outside of the web process pool, eg. something like:
setsid /usr/bin/python3 my_script.py
For what it’s worth to anyone with a subprocess problem… the solution turned out to be attributed to the a badly done .env implementation. When we finally recreated the .env, the problem went away. I actually recommended to my team that we use Anaconda for our Python env and that did the trick. :’D
PM2 is running as a web
user. ffmpeg was installed native to Ubuntu 16.04 LTS using sudo apt install ffmpeg
. The Python version is 3.6. The software uses [email protected].
The applications spawned produce no errors. When the ffmpeg code executes for the first time, we see an output and the ffmpeg process completes the task as expected.
All subsequent requests stall on the next ffmpeg execution. No output. No return from the ffmpeg process. No errors. The PM2 process does not error out. The application log stalls on the ffmpeg command as though it is hung.
What is the root cause? Any help is deeply appreciated.
Furthermore, what are the reasons PM2 hangs on a subprocess (like ffmpeg)?
Here is the code:
class ImageHelper:
def __init__(self):
pass
@classmethod
def create_thumb_from_video_ffmpeg(cls, input_video_file_path,
output_image_path,
scale_width,
scale_height
):
"""
This function is used to create the thumb image
from a source video file.
We are using a python wrapper/library for FFMPEG
"""
try:
if Functions.get_attribute_env('ENVIRONMENT') == 'prod':
out, err = (
ffmpeg
.input(input_video_file_path, ss="00:00:00.001")
.filter('scale', scale_width, scale_height)
.output(output_image_path, vframes=1, loglevel='quiet')
.overwrite_output()
.run(capture_stdout=True)
)
print("We only see this once!")
else:
out, err = (
ffmpeg
.input(input_video_file_path, ss="00:00:00.001")
.filter('scale', scale_width, scale_height)
.output(output_image_path, vframes=1)
.overwrite_output()
.run(capture_stdout=True)
)
print("We only see this once!")
if err:
if Functions.get_attribute_env('ENVIRONMENT') != 'prod':
print('ffmpeg video thumb', err)
else:
Functions.logger_function(str(err))
raise Exception(err)
else:
return output_image_path
except Exception as e:
if Functions.get_attribute_env('ENVIRONMENT') != 'prod':
print('in thumb exception', e)
else:
Functions.logger_function(str(e))
raise Exception(e)
Check whether the ffmpeg
process is running while the sequential request is made or not. If it is, you may want to make sure the process is closed after the first completion so it can start again for the sequential request.
Processes spawned by apache
or nginx
will have limits on how long they execute and will be killed automatically. In those cases you’ll probably want to have the script triggered to run outside of the web process pool, eg. something like:
setsid /usr/bin/python3 my_script.py
For what it’s worth to anyone with a subprocess problem… the solution turned out to be attributed to the a badly done .env implementation. When we finally recreated the .env, the problem went away. I actually recommended to my team that we use Anaconda for our Python env and that did the trick. :’D