"anull" insensibly eating memory

[Alfredo Di Napoli]

At work we recently upgraded from 5.0.1 to 7.1.0, and we couldn't figure out why some machines in our cluster were really struggling; these are big machines with 16 cores and something like 30GB of RAM. By a quick htop inspection it seemed like for a single ffmpeg command, all the RAM was being eventually consumed and the machines started to swap. Removing the anull filter to the command made the leak go away. We could still see the memory climbing ever so slightly, but not in the same aggressive way.

To reproduce, consider the following command:

% /usr/local/bin/ffmpeg-7.1.0 -i /foo/left/master -i /foo/right/master -i /foo/left/master.muted.mov -i /foo/right/master.muted.mov -aspect 32:9 -y -filter_complex [1:1]anull[hms-1-1]  -filter_complex [0:1]anull[hms-0-1] -filter_complex '[2:v]setpts=PTS-STARTPTS,pad=iw*2:ih:color=white,framerate=30[l];[3:v]setpts=PTS-STARTPTS[r];[l][r]overlay=x=W-w:shortest=1[vout]' -map [vout] -map [hms-1-1] -map [hms-0-1] -shortest /foo/central/master.mp4

Without getting into too much detail, anull here is used just as a way to pass the audio stream, unchanged, to the final video. I can't remember why I'm using anull and not just mapping the relevant audio streams directly, but case on point, this command leaks quite severely. If we amend the command this way:

% /usr/local/bin/ffmpeg-7.1.0 -i /foo/left/master -i /foo/right/master -i /foo/left/master.muted.mov -i /foo/right/master.muted.mov -aspect 32:9 -y -filter_complex '[2:v]setpts=PTS-STARTPTS,pad=iw*2:ih:color=white,framerate=30[l];[3:v]setpts=PTS-STARTPTS[r];[l][r]overlay=x=W-w:shortest=1[vout]' -map [vout] -shortest /foo/central/master.mp4

The leak goes away.

ffmpeg version 7.1 Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
configuration: --prefix=/usr/local/ic-ffmpeg-7.1.0 --extra-cflags=-I/usr/local/ic-ffmpeg-7.1.0/include --extra-ldflags=-L/usr/local/ic-ffmpeg-7.1.0/lib --bindir=/usr/local/ic-ffmpeg-7.1.0/bin --extra-libs=-ldl --ld=g++ --enable-gpl --enable-gnutls --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-frei0r
libavutil      59. 39.100 / 59. 39.100
libavcodec     61. 19.100 / 61. 19.100
libavformat    61.  7.100 / 61.  7.100
libavdevice    61.  3.100 / 61.  3.100
libavfilter    10.  4.100 / 10.  4.100
libswscale      8.  3.100 /  8.  3.100
libswresample   5.  3.100 /  5.  3.100
libpostproc    58.  3.100 / 58.  3.100

[MasterQuestionable]

͏    Refactored:
#1:
͏    ffmpeg -i "left" -i "right" -i "left.muted.mov" -i "right.muted.mov" -aspect 32:9 -y -lavfi "[1:1]anull[hms-1-1]" -lavfi "[0:1]anull[hms-0-1]" -lavfi "[2:v]setpts=(( PTS - STARTPTS )),pad=(( iw * 2 )):ih:color=white,framerate=30[l]; [3:v]setpts=(( PTS - STARTPTS ))[r]; [l][r]overlay=x=(( W - w )):shortest=1[vout]" -map [vout] -map [hms-1-1] -map [hms-0-1] -shortest "central.mp4"
#2:
͏    ffmpeg -i "left" -i "right" -i "left.muted.mov" -i "right.muted.mov" -aspect 32:9 -y -lavfi "[2:v]setpts=(( PTS - STARTPTS )),pad=(( iw * 2 )):ih:color=white,framerate=30[l]; [3:v]setpts=(( PTS - STARTPTS ))[r]; [l][r]overlay=x=(( W - w )):shortest=1[vout]" -map [vout] -shortest "central.mp4"

͏    Interpreted "filter_complex" common:
[[

        [2:v]setpts=(( PTS - STARTPTS )),
        pad=(( iw * 2 )):ih:color=white,
        framerate=30[l];
        [3:v]setpts=(( PTS - STARTPTS ))[r];

        [l][r]overlay=x=(( W - w )):shortest=1[vout]

]]

͏    Example #2 didn't include the problematic audio.
͏    Would the problem reproduce just with the audio?

͏    Try removing excessive options to ascertain the minimal one.

[Alfredo Di Napoli]

Hello there,

Thank you so much for looking into this.

Some (hopefully) extra pointers to track down the issue: I was able to reproduce the problem using the refactored command #1 and using the same input file for all the -i in the command (which also rules out the "muted" argument, i.e. it doesn't matter).

It's also worth pointing out that the issue starts at around frame 130 of my transcoding (~ after 30 seconds of processed video).

The input file is 76MB long and is nothing special (is a piece of a YoutTube video), which I'm happy to attach if the attachments supports such a large file.

[MasterQuestionable]

͏    Must include video to reproduce?
͏    Is the video filter-chain relevant?

͏    Try "testsrc" alike as input source? (`-f lavfi -i "..."`)
͏    See:
͏    https://ffmpeg.org/ffmpeg-filters.html#testsrc
͏    https://ffmpeg.org/ffmpeg-filters.html#anullsrc

͏    What about adding "-map 0:1 -map 1:1" in previous example #2?


͏    I tend to avoid processing large files.
͏    You may leave the YouTube video link whatsoever.