Regression with filter graph involving -shortest flag

[Wes Castro]

Summary of the bug:

A command line that worked with FFmpeg 5.1 produces a regressed output in FFmpeg 6.0, 6.1, and git-master. The expected output is a ~15 second MP4 with a video and audio stream. With git-master all video frames are dropped and only a 980ms audio stream is produced.

Apologies for the large filter graph... when attempting to reduce the chain I was unable to reproduce the issue in the same way.

How to reproduce:

Generate test inputs:

FFmpeg for all commands built with only ./configure --enable-zlib && make -j$(nproc)

ffmpeg -y -f lavfi -i testsrc=duration=15:size=1080x1920:rate=30 -f lavfi -i sine=frequency=1000:sample_rate=48000:duration=14.9 15secondvideoaudio.mp4
ffmpeg -y -f lavfi -i sine=frequency=1000:sample_rate=48000:duration=300 5minaudio.mp4
ffmpeg -y -f lavfi -i testsrc=size=1280x720 -frames:v 1 image.png

Run problematic filter graph:

ffmpeg -v 9 -loglevel 99 -hide_banner -y -i 15secondvideoaudio.mp4 -ss 21 -to 35 -i 5minaudio.mp4 -i 15secondvideoaudio.mp4 -framerate 30 -i image.png -framerate 30 -i image.png -filter_complex "[2:v] scale=w=1080:h=-2 [_fg_scaled] ; nullsrc=size=1080x1920:r=2:d=1, geq=red_expr='88+(Y/H)*(72-88)':green_expr='136+(Y/H)*(72-136)':blue_expr='184+(Y/H)*(72-184)', loop=-1:size=2, fps=fps=30 [_bg] ; [_bg][_fg_scaled] overlay=(main_w-overlay_w)/2:(main_h-overlay_h)/2:shortest=1 [uploaded_video_2_0_uploaded_video] ; [3:v][uploaded_video_2_0_uploaded_video] scale2ref=w=in_w*1.000000:h=in_h*1.000000:sws_flags=lanczos [_overlay][_video_in] ; [_video_in][_overlay] overlay=x=main_w*0.000000:y=main_h*0.000000:shortest=0:repeatlast=1 [video1] ; [4:v][video1] scale2ref=w=in_w*0.430556:h=in_h*0.075000:sws_flags=lanczos [_overlay][_video_in] ; [_video_in][_overlay] overlay=x=main_w*0.234429:y=main_h*-0.132120:shortest=0:repeatlast=1 [_video_out] ; [_video_out] setpts=PTS-STARTPTS [video]" -filter_complex "[1:a] volume=volume=1.000, afade=type=in:start_time=0:duration=0.4, afade=type=out:start_time=14.367:duration=0.6, aloop=loop=-1:size=2.14748e+09 [music] ; [0:a] volume=volume=0.000 [original_audio] ; [original_audio][music] amix=inputs=2:duration=longest, asetpts=PTS-STARTPTS [music_mix]" -map [music_mix] -map [video] -map_metadata -1 -shortest -f mp4 -y output.mp4

Results

• FFmpeg 5.1
  • Produces 15.04 second video+audio
• FFmpeg 6.0
  • Hangs indefinitely when encoding frame 446 (when I cancel it with Ctrl-C the last frame= line output is for 448)
• FFmpeg 6.1
  • Fails with Error while filtering: No space left
  • Workaround: Change loop=-1 to loop=99
    • Not ideal since loop is no longer infinite
    • Higher loop value (999) will cause conversion to fail with Error while filtering: No space left again
• FFmpeg master branch 3/26/2024 (9df11820650b1d93f6a49519aaa51afb5cc17086)
  • Warning is printed: [vost#0:1/mpeg4 @ 0x2e90500] No filtered frames for output stream, trying to initialize anyway.
  • Produces no video frames and 980ms of audio.

Notes

Changing the audio stream duration of 15secondvideoaudio.mp4 to 14.8 or 15.0 causes the hang to not occur in FFmpeg 6.0 or 6.1, but the master branch issue still occurs.

[Wes Castro]

Log files were uploaded with the VideoLAN file uploader.