-map 0:v causes segfaults on 2pass libvpx-vp9 on multi video-track file

[kagami]

If file has multiple video tracks, -map 0:v causes segfault in case of 2pass libvpx-vp9 encode (probably other 2pass encoders will segfault too):

$ ffmpeg -i test.mp4 -map 0:v -c:v libvpx-vp9 -pass 1 -f null -           
ffmpeg version N-76952-g6b978da Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.9.3 (Gentoo 4.9.3 p1.4, pie-0.6.4)
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags=' ' --disable-static --enable-avfilter --enable-avresample --disable-stripping --enable-nonfree --enable-version3 --disable-indev=v4l2 --disable-outdev=v4l2 --disable-indev=oss --disable-indev=jack --disable-outdev=oss --enable-bzlib --enable-runtime-cpudetect --disable-debug --disable-doc --disable-gnutls --enable-gpl --enable-hardcoded-tables --enable-iconv --disable-lzma --enable-network --enable-openssl --enable-postproc --disable-libsmbclient --enable-ffplay --disable-vaapi --enable-vdpau --enable-xlib --enable-libxcb --enable-libxcb-shm --enable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --disable-openal --disable-opengl --disable-libv4l2 --disable-libpulse --disable-libopencore-amrwb --disable-libopencore-amrnb --disable-libfdk-aac --disable-libopenjpeg --disable-libbluray --disable-libcelt --disable-libgme --disable-libgsm --disable-libmodplug --enable-libopus --disable-libquvi --disable-librtmp --disable-libssh --disable-libschroedinger --disable-libspeex --enable-libvorbis --enable-libvpx --disable-libzvbi --disable-libbs2b --disable-chromaprint --disable-libflite --disable-frei0r --disable-libfribidi --enable-fontconfig --disable-ladspa --enable-libass --enable-libfreetype --disable-librubberband --disable-libzimg --disable-libsoxr --enable-pthreads --enable-libvo-aacenc --disable-libvo-amrwbenc --enable-libmp3lame --disable-libaacplus --disable-libfaac --disable-libkvazaar --disable-nvenc --disable-libopenh264 --disable-libsnappy --disable-libtheora --disable-libtwolame --disable-libwavpack --disable-libwebp --enable-libx264 --enable-libx265 --disable-libxvid --disable-amd3dnow --disable-amd3dnowext --disable-aesni --disable-avx2 --disable-fma3 --disable-fma4 --disable-xop --cpu=corei7-avx
  libavutil      55.  9.100 / 55.  9.100
  libavcodec     57. 16.101 / 57. 16.101
  libavformat    57. 19.100 / 57. 19.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 17.100 /  6. 17.100
  libavresample   3.  0.  0 /  3.  0.  0
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[mjpeg @ 0x17967e0] Changing bps to 8
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Keep Drawing
    artist          : Shelter Studio
    date            : 2011-11-27T00:00:00+04:00
    encoder         : Lavf57.19.100
    comment         : Anime: Original animation 
                    : Music: Park Taejoon - Keep Drawing 
                    : More at AMVNews.ru
    genre           : Original Animation, Instrumental, Sentimental
    description     : В ожидании новых AMV посмотрим немного оригинальной анимации. Короткометражка Keep Drawing от независимой корейской анимационной студии Shelter Studio
    network         : http://amvnews.ru/index.php?go=Files&in=view&id=7237
    media_type      : 6
  Duration: 00:00:06.00, start: 0.000000, bitrate: 2419 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 2355 kb/s, 24 fps, 24 tbr, 24k tbn, 48 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Video: mjpeg (mp4v / 0x7634706D), yuvj420p(pc, bt470bg/unknown/unknown), 500x281 [SAR 1:1 DAR 500:281], 90k tbn, 90k tbc
    Metadata:
      handler_name    : VideoHandler
[swscaler @ 0x17adc60] deprecated pixel format used, make sure you did set range correctly
[libvpx-vp9 @ 0x1794900] v1.5.0-154-g610b413
[libvpx-vp9 @ 0x1833760] v1.5.0-154-g610b413
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Keep Drawing
    artist          : Shelter Studio
    date            : 2011-11-27T00:00:00+04:00
    media_type      : 6
    comment         : Anime: Original animation 
                    : Music: Park Taejoon - Keep Drawing 
                    : More at AMVNews.ru
    genre           : Original Animation, Instrumental, Sentimental
    description     : В ожидании новых AMV посмотрим немного оригинальной анимации. Короткометражка Keep Drawing от независимой корейской анимационной студии Shelter Studio
    network         : http://amvnews.ru/index.php?go=Files&in=view&id=7237
    encoder         : Lavf57.19.100
    Stream #0:0(und): Video: vp9 (libvpx-vp9), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, pass 1, 200 kb/s, 24 fps, 24 tbn, 24 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc57.16.101 libvpx-vp9
    Stream #0:1(und): Video: vp9 (libvpx-vp9), yuv420p, 500x281 [SAR 1:1 DAR 500:281], q=-1--1, pass 1, 200 kb/s, 90k tbn, 90k tbc
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc57.16.101 libvpx-vp9
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> vp9 (libvpx-vp9))
  Stream #0:1 -> #0:1 (mjpeg (native) -> vp9 (libvpx-vp9))
Press [q] to stop, [?] for help
frame=  144 fps= 42 q=0.0 Lq=0.0 size=N/A time=00:00:00.00 bitrate=N/A    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded 

$ gdb --args ffmpeg -i test.mp4 -map 0:v -c:v libvpx-vp9 -pass 2 -f null -
GNU gdb (Gentoo 7.9.1 vanilla) 7.9.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ffmpeg...(no debugging symbols found)...done.
(gdb) run
Starting program: /usr/bin/ffmpeg -i test.mp4 -map 0:v -c:v libvpx-vp9 -pass 2 -f null -
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
ffmpeg version N-76952-g6b978da Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.9.3 (Gentoo 4.9.3 p1.4, pie-0.6.4)
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags=' ' --disable-static --enable-avfilter --enable-avresample --disable-stripping --enable-nonfree --enable-version3 --disable-indev=v4l2 --disable-outdev=v4l2 --disable-indev=oss --disable-indev=jack --disable-outdev=oss --enable-bzlib --enable-runtime-cpudetect --disable-debug --disable-doc --disable-gnutls --enable-gpl --enable-hardcoded-tables --enable-iconv --disable-lzma --enable-network --enable-openssl --enable-postproc --disable-libsmbclient --enable-ffplay --disable-vaapi --enable-vdpau --enable-xlib --enable-libxcb --enable-libxcb-shm --enable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --disable-openal --disable-opengl --disable-libv4l2 --disable-libpulse --disable-libopencore-amrwb --disable-libopencore-amrnb --disable-libfdk-aac --disable-libopenjpeg --disable-libbluray --disable-libcelt --disable-libgme --disable-libgsm --disable-libmodplug --enable-libopus --disable-libquvi --disable-librtmp --disable-libssh --disable-libschroedinger --disable-libspeex --enable-libvorbis --enable-libvpx --disable-libzvbi --disable-libbs2b --disable-chromaprint --disable-libflite --disable-frei0r --disable-libfribidi --enable-fontconfig --disable-ladspa --enable-libass --enable-libfreetype --disable-librubberband --disable-libzimg --disable-libsoxr --enable-pthreads --enable-libvo-aacenc --disable-libvo-amrwbenc --enable-libmp3lame --disable-libaacplus --disable-libfaac --disable-libkvazaar --disable-nvenc --disable-libopenh264 --disable-libsnappy --disable-libtheora --disable-libtwolame --disable-libwavpack --disable-libwebp --enable-libx264 --enable-libx265 --disable-libxvid --disable-amd3dnow --disable-amd3dnowext --disable-aesni --disable-avx2 --disable-fma3 --disable-fma4 --disable-xop --cpu=corei7-avx
  libavutil      55.  9.100 / 55.  9.100
  libavcodec     57. 16.101 / 57. 16.101
  libavformat    57. 19.100 / 57. 19.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 17.100 /  6. 17.100
  libavresample   3.  0.  0 /  3.  0.  0
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[mjpeg @ 0x6587e0] Changing bps to 8
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Keep Drawing
    artist          : Shelter Studio
    date            : 2011-11-27T00:00:00+04:00
    encoder         : Lavf57.19.100
    comment         : Anime: Original animation 
                    : Music: Park Taejoon - Keep Drawing 
                    : More at AMVNews.ru
    genre           : Original Animation, Instrumental, Sentimental
    description     : В ожидании новых AMV посмотрим немного оригинальной анимации. Короткометражка Keep Drawing от независимой корейской анимационной студии Shelter Studio
    network         : http://amvnews.ru/index.php?go=Files&in=view&id=7237
    media_type      : 6
  Duration: 00:00:06.00, start: 0.000000, bitrate: 2419 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 2355 kb/s, 24 fps, 24 tbr, 24k tbn, 48 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Video: mjpeg (mp4v / 0x7634706D), yuvj420p(pc, bt470bg/unknown/unknown), 500x281 [SAR 1:1 DAR 500:281], 90k tbn, 90k tbc
    Metadata:
      handler_name    : VideoHandler
[swscaler @ 0x6d70a0] deprecated pixel format used, make sure you did set range correctly
[libvpx-vp9 @ 0x656900] v1.5.0-154-g610b413
[libvpx-vp9 @ 0x6f52e0] v1.5.0-154-g610b413
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Keep Drawing
    artist          : Shelter Studio
    date            : 2011-11-27T00:00:00+04:00
    media_type      : 6
    comment         : Anime: Original animation 
                    : Music: Park Taejoon - Keep Drawing 
                    : More at AMVNews.ru
    genre           : Original Animation, Instrumental, Sentimental
    description     : В ожидании новых AMV посмотрим немного оригинальной анимации. Короткометражка Keep Drawing от независимой корейской анимационной студии Shelter Studio
    network         : http://amvnews.ru/index.php?go=Files&in=view&id=7237
    encoder         : Lavf57.19.100
    Stream #0:0(und): Video: vp9 (libvpx-vp9), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, pass 2, 200 kb/s, 24 fps, 24 tbn, 24 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc57.16.101 libvpx-vp9
    Stream #0:1(und): Video: vp9 (libvpx-vp9), yuv420p, 500x281 [SAR 1:1 DAR 500:281], q=-1--1, pass 2, 200 kb/s, 90k tbn, 90k tbc
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc57.16.101 libvpx-vp9
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> vp9 (libvpx-vp9))
  Stream #0:1 -> #0:1 (mjpeg (native) -> vp9 (libvpx-vp9))
Press [q] to stop, [?] for help
[New Thread 0x7fffde746700 (LWP 5641)]/A time=00:00:05.08 bitrate=N/A    
[New Thread 0x7fffdef47700 (LWP 5640)]
[New Thread 0x7fffdf748700 (LWP 5639)]
[New Thread 0x7fffdff49700 (LWP 5638)]
[New Thread 0x7fffe074a700 (LWP 5637)]
[New Thread 0x7fffe0f4b700 (LWP 5636)]
[New Thread 0x7fffe174c700 (LWP 5634)]
[New Thread 0x7fffe1f4d700 (LWP 5633)]
[New Thread 0x7fffe274e700 (LWP 5632)]
[New Thread 0x7fffe6194700 (LWP 5631)]
[New Thread 0x7fffe6995700 (LWP 5630)]
[New Thread 0x7fffe7196700 (LWP 5629)]
[New Thread 0x7fffe7997700 (LWP 5628)]
[New Thread 0x7fffe8198700 (LWP 5627)]
[New Thread 0x7fffe8999700 (LWP 5626)]
[New Thread 0x7fffe919a700 (LWP 5625)]
[New Thread 0x7fffe999b700 (LWP 5624)]
[New Thread 0x7fffea19c700 (LWP 5623)]
[New Thread 0x7fffea99d700 (LWP 5622)]
[New Thread 0x7fffeb19e700 (LWP 5621)]
[New Thread 0x7fffeb99f700 (LWP 5620)]
[New Thread 0x7fffec1a0700 (LWP 5619)]
[New Thread 0x7fffec9a1700 (LWP 5618)]
[New Thread 0x7fffed1a2700 (LWP 5617)]
[New Thread 0x7fffed9a3700 (LWP 5616)]
[New Thread 0x7fffee1a4700 (LWP 5615)]
[New Thread 0x7fffee9a5700 (LWP 5614)]

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff15ee0e7 in vp9_frame_type_qdelta () from /usr/lib64/libvpx.so.3
(gdb) bt full
#0  0x00007ffff15ee0e7 in vp9_frame_type_qdelta () from /usr/lib64/libvpx.so.3
No symbol table info available.
#1  0x00007ffff15ee414 in vp9_rc_pick_q_and_bounds () from /usr/lib64/libvpx.so.3
No symbol table info available.
#2  0x00007ffff15e5369 in set_size_dependent_vars () from /usr/lib64/libvpx.so.3
No symbol table info available.
#3  0x00007ffff15e92cc in encode_frame_to_data_rate () from /usr/lib64/libvpx.so.3
No symbol table info available.
#4  0x00007ffff15ea7be in vp9_get_compressed_data () from /usr/lib64/libvpx.so.3
No symbol table info available.
#5  0x00007ffff15a931a in encoder_encode () from /usr/lib64/libvpx.so.3
No symbol table info available.
#6  0x00007ffff14f62c8 in vpx_codec_encode () from /usr/lib64/libvpx.so.3
No symbol table info available.
#7  0x00007ffff6529d92 in vp8_encode () from /usr/lib64/libavcodec.so.57
No symbol table info available.
#8  0x00007ffff6682f66 in avcodec_encode_video2 () from /usr/lib64/libavcodec.so.57
No symbol table info available.
#9  0x000000000040867c in main ()
No symbol table info available.

[Carl Eugen Hoyos]

No crash here with an older version of libvpx and your backtrace indicates an issue within libvpx so I am closing this ticket.

[kagami]

What version of libvpx do you have? I tried with 1.4.0 and still getting the same segfault. I think the problem is that ffmpeg uses same logfile for two different video tracks which is incorrent; segfault is just the consequence of passing corrupted logfile. Even if your version of libvpx can tolerate that, ffmpeg's behaviour is still wrong.

[Carl Eugen Hoyos]

Replying to kagami:

What version of libvpx do you have? I tried with 1.4.0 and still getting the same segfault.

1.3.0

I think the problem is that ffmpeg uses same logfile for two different video tracks

This is possible but unavoidable in other cases, think of a user mistake.

which is incorrent; segfault is just the consequence of passing corrupted logfile.

So if I understand correctly (and please do correct me if I am wrong) you argue that it is perfectly ok for libvpx to crash if the logfile that is passed does not match the source file? Or in other words: If a user (accidentally or not) passes a different source file for first and second pass the expected result is that libvpx crashes?

To be more precise: Generally, libraries should never crash on invalid input. In this specific case (second pass input file does not match the log file) I have argued in the past (#2846) that you are lucky if the encoding library crashes: If it does not crash and you get an output file, it is most likely broken / very bad quality and you will not notice. In any case, all this is outside of the scope of FFmpeg.

[kagami]

you argue that it is perfectly ok for libvpx to crash if the logfile that is passed does not match the source file?

No, segfault is just the symptom here.

Generally, libraries should never crash on invalid input

Completely agree. This crash should be fixed in libvpx and this issue should have a bit different name and description. I just don't have enough knowledge to analyze whether ffmpeg correctly encodes two VP9 tracks with 2pass at the same time (corrupted logfile is just an assumption), so I demonstated the most obvious consequence from the user point of view in this ticket. It still would be bad if you get output but in bad quality if e.g. libvpx can tolerate wrong input.

I think it might be related to the logfile processing because:

• I encode two tracks but get only single log file
• If I encode only 1 track on a second pass, I still get segfault

I may be complete wrong here though. So ideally I should file crash bug to the libvpx first, wait for the fix, make sure library explicitly forbids incorrect logs/input, try again. I just thought it would be easier if someone at ffmpeg bugtracker could tell whether it's related to ffmpeg or just libvpx issue.