Opened 7 years ago
Closed 6 years ago
#7264 closed defect (wontfix)
libvmaf segmentation fault at ffmpeg 4.0
Reported by: | George Stoyanov | Owned by: | |
---|---|---|---|
Priority: | important | Component: | avfilter |
Version: | git-master | Keywords: | libvmaf |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
I am trying to compare two videos using the libvmaf in ffmpeg 4.0. I have compiled and installed from source VMAF and FFMPEG and FFMPEG has built-in support for libvmaf
. That's the ffmpeg configuration:
$ ffmpeg -version ffmpeg version 4.0 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.9) 20160609 configuration: --pkg-config-flags=--static --extra-libs='-lpthread -lm' --toolchain=hardened --enable-gpl --enable-nonfree --enable-version3 --disable-stripping --enable-avisynth --enable-libass --enable-libfontconfig --enable-libfreetype --enable-libvmaf --enable-libfribidi --enable-libgme --enable-libgsm --enable-librubberband --enable-libshine --enable-libsnappy --enable-libssh --enable-libtwolame --enable-libwavpack --enable-libzvbi --enable-openal --enable-sdl2 --enable-libdrm --enable-frei0r --enable-ladspa --enable-libpulse --enable-libsoxr --enable-libspeex --enable-avfilter --enable-postproc --enable-pthreads --enable-libfdk-aac --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --disable-ffplay --enable-libopenjpeg --enable-libsrt --enable-libwebp --enable-libxvid --enable-libvidstab --enable-libopenh264 --enable-zlib --enable-openssl libavutil 56. 14.100 / 56. 14.100 libavcodec 58. 18.100 / 58. 18.100 libavformat 58. 12.100 / 58. 12.100 libavdevice 58. 3.100 / 58. 3.100 libavfilter 7. 16.100 / 7. 16.100 libswscale 5. 1.100 / 5. 1.100 libswresample 3. 1.100 / 3. 1.100 libpostproc 55. 1.100 / 55. 1.100
My VMAF is version:
VMAF Development Kit (VDK) Version 1.3.5 VMAF Version 0.6.3
When I try to run the VMAF on both files using the following command I get a segmentation fault error and error about unknown format and segmentation fault. The syntax of my command is the following:
`
ffmpeg -i distorted.nut -i reference.nut -lavfi libvmaf -f null -
`
The NUT files in this case are both in raw format. I have tried to convert the nut to mp4 using libx264
but the result was exactly the same. I have uploaded the log file to https://we.tl/gUNY9Adoky with the debug information.
If I run the VMAF manually from the VMAF install directory I got the results:
./run_vmaf yuv420p 1280 720 reference.nut distorted.nut --out-fmt json
This is the aggregated VMAF score:
"aggregate": { "VMAF_feature_adm2_score": 0.989985319391465, "VMAF_feature_motion2_score": 12.065062245699426, "VMAF_feature_vif_scale0_score": 0.6955497705160056, "VMAF_feature_vif_scale1_score": 0.956442334512976, "VMAF_feature_vif_scale2_score": 0.9785004960696057, "VMAF_feature_vif_scale3_score": 0.9873306412919364, "VMAF_score": 99.37877231666236, "method": "mean" }
I have checked and the library header libvmaf.h
is under /usr/local/include
, the library libvmaf.a
under /user/local/lib
and all the model files under /usr/local/share
, so everything looks OK.
Attachments (2)
Change History (18)
comment:1 by , 7 years ago
Keywords: | libvmaf added; vmaf removed |
---|
comment:2 by , 7 years ago
I have tried to compile the git head with libvmaf support but I got an error:
libavfilter/vf_libvmaf.c: In function ‘compute_vmaf_score’: libavfilter/vf_libvmaf.c:165:16: error: too few arguments to function ‘compute_vmaf’ s->error = compute_vmaf(&s->vmaf_score, format, s->width, s->height, ^ In file included from libavfilter/vf_libvmaf.c:28:0: /usr/local/include/libvmaf.h:26:5: note: declared here int compute_vmaf(double* vmaf_score, char* fmt, int width, int height, int (*read_frame)(float *ref_data, float *main_data, float *temp_data, int stride_byte, void *user_data), ^ ffbuild/common.mak:60: recipe for target 'libavfilter/vf_libvmaf.o' failed make: *** [libavfilter/vf_libvmaf.o] Error 1
My compilation code is:
git clone https://github.com/FFmpeg/FFmpeg.git cd FFmpeg ./configure \ --pkg-config-flags="--static" \ --extra-libs="-lpthread -lm" \ --toolchain=hardened \ --enable-gpl \ --enable-nonfree \ --enable-version3 \ --disable-stripping \ --enable-avisynth \ --enable-libass \ --enable-libfontconfig \ --enable-libfreetype \ --enable-libvmaf \ --enable-libfribidi \ --enable-libgme \ --enable-libgsm \ --enable-librubberband \ --enable-libshine \ --enable-libsnappy \ --enable-libssh \ --enable-libtwolame \ --enable-libwavpack \ --enable-libzvbi \ --enable-openal \ --enable-sdl2 \ --enable-libdrm \ --enable-frei0r \ --enable-ladspa \ --enable-libpulse \ --enable-libsoxr \ --enable-libspeex \ --enable-avfilter \ --enable-postproc \ --enable-pthreads \ --enable-libfdk-aac \ --enable-libmp3lame \ --enable-libopus \ --enable-libtheora \ --enable-libvorbis \ --enable-libvpx \ --enable-libx264 \ --enable-libx265 \ --disable-ffplay \ --enable-libopenjpeg \ --enable-libsrt \ --enable-libwebp \ --enable-libxvid \ --enable-libvidstab \ --enable-libopenh264 \ --enable-zlib \ --enable-openssl make -j16 sudo make install sudo make distclean
I have uploaded the compilation log in the attachment section if you want I can also provide the detailed bug reports in ffmpeg 4.0 but you said you are not supporting it.
comment:4 by , 7 years ago
you are right I have tried to recompile ffmpeg 4.0 right now and got the same error. I think it is because I have re-compiled the VMAF yesterday and this changed the VMAF VDK version from 1.3.5 to 1.3.6 so apparently the latest VMAF git build is not working with ffmpeg. But the segmentation fault error was happening on the previous VMAF version as well.
comment:5 by , 7 years ago
I have also opened a ticket in Github regarding this and some other issues: https://github.com/Netflix/vmaf/issues/174
comment:6 by , 7 years ago
And with a bit of delay I am posting the uncut command with the segmentation fault error:
ffmpeg -i distorted.nut -i reference.nut -lavfi libvmaf -f null - ffmpeg version 4.0 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.9) 20160609 configuration: --pkg-config-flags=--static --extra-libs='-lpthread -lm' --toolchain=hardened --enable-gpl --enable-nonfree --enable-version3 --disable-stripping --enable-avisynth --enable-libass --enable-libfontconfig --enable-libfreetype --enable-libvmaf --enable-libfribidi --enable-libgme --enable-libgsm --enable-librubberband --enable-libshine --enable-libsnappy --enable-libssh --enable-libtwolame --enable-libwavpack --enable-libzvbi --enable-openal --enable-sdl2 --enable-libdrm --enable-frei0r --enable-ladspa --enable-libpulse --enable-libsoxr --enable-libspeex --enable-avfilter --enable-postproc --enable-pthreads --enable-libfdk-aac --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --disable-ffplay --enable-libopenjpeg --enable-libsrt --enable-libwebp --enable-libxvid --enable-libvidstab --enable-libopenh264 --enable-zlib --enable-openssl libavutil 56. 14.100 / 56. 14.100 libavcodec 58. 18.100 / 58. 18.100 libavformat 58. 12.100 / 58. 12.100 libavdevice 58. 3.100 / 58. 3.100 libavfilter 7. 16.100 / 7. 16.100 libswscale 5. 1.100 / 5. 1.100 libswresample 3. 1.100 / 3. 1.100 libpostproc 55. 1.100 / 55. 1.100 Input #0, nut, from 'distorted.nut': Metadata: encoder : Lavf58.12.100 Duration: 00:04:59.94, start: 0.000000, bitrate: 553011 kb/s Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 51200 tbn, 51200 tbc Metadata: encoder : Lavc58.18.100 rawvideo Input #1, nut, from 'reference.nut': Metadata: encoder : Lavf58.12.100 Duration: 00:04:59.96, start: 0.000000, bitrate: 553011 kb/s Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 51200 tbn, 51200 tbc Metadata: encoder : Lavc58.18.100 rawvideo Stream mapping: Stream #0:0 (rawvideo) -> libvmaf:main Stream #1:0 (rawvideo) -> libvmaf:reference libvmaf -> Stream #0:0 (wrapped_avframe) Press [q] to stop, [?] for help Start calculating VMAF score... Output #0, null, to 'pipe:': Metadata: encoder : Lavf58.12.100 Stream #0:0: Video: wrapped_avframe, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc (default) Metadata: encoder : Lavc58.18.100 wrapped_avframe unknown format ▒h▒rU. Segmentation fault (core dumped)
follow-up: 8 comment:7 by , 7 years ago
Please test current FFmpeg git head (nothing else is supported here) and provide complete, uncut console output, backtrace, disassembly and register dump as explained on https://ffmpeg.org/bugreports.html
comment:8 by , 7 years ago
Replying to cehoyos:
Please test current FFmpeg git head (nothing else is supported here) and provide complete, uncut console output, backtrace, disassembly and register dump as explained on https://ffmpeg.org/bugreports.html
I have already tested it and reported my findings. The most recent git head doesn't work with the most recent VMAF build. It doesn't want to compile, I have attached the compilation error log alongside.
comment:10 by , 7 years ago
With latest FFMPEG nightly and VDK 1.3.5
ffmpeg -i distcut.nut -i refcut.nut -lavfi libvmaf="log_fmt=json:log_path=ffvmaf.json:psnr=1:ssim=1:ms_ssim=1" -f null - ffmpeg version N-91330-ga990184 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.9) 20160609 configuration: --pkg-config-flags=--static --extra-libs='-lpthread -lm' --toolchain=hardened --enable-gpl --enable-nonfree --enable-version3 --disable-stripping --enable-avisynth --enable-libass --enable-libfontconfig --enable-libfreetype --enable-libvmaf --enable-libfribidi --enable-libgme --enable-libgsm --enable-librubberband --enable-libshine --enable-libsnappy --enable-libssh --enable-libtwolame --enable-libwavpack --enable-libzvbi --enable-openal --enable-sdl2 --enable-libdrm --enable-frei0r --enable-ladspa --enable-libpulse --enable-libsoxr --enable-libspeex --enable-avfilter --enable-postproc --enable-pthreads --enable-libfdk-aac --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --disable-ffplay --enable-libopenjpeg --enable-libsrt --enable-libwebp --enable-libxvid --enable-libvidstab --enable-libopenh264 --enable-zlib --enable-openssl libavutil 56. 18.102 / 56. 18.102 libavcodec 58. 20.103 / 58. 20.103 libavformat 58. 17.100 / 58. 17.100 libavdevice 58. 4.101 / 58. 4.101 libavfilter 7. 25.100 / 7. 25.100 libswscale 5. 2.100 / 5. 2.100 libswresample 3. 2.100 / 3. 2.100 libpostproc 55. 2.100 / 55. 2.100 Input #0, nut, from 'distcut.nut': Metadata: encoder : Lavf58.12.100 Duration: 00:00:39.98, start: 0.000000, bitrate: 553250 kb/s Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 51200 tbn, 51200 tbc Metadata: encoder : Lavc58.18.100 rawvideo Input #1, nut, from 'refcut.nut': Metadata: encoder : Lavf58.12.100 Duration: 00:00:39.98, start: 0.000000, bitrate: 553250 kb/s Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 51200 tbn, 51200 tbc Metadata: encoder : Lavc58.18.100 rawvideo Stream mapping: Stream #0:0 (rawvideo) -> libvmaf:main Stream #1:0 (rawvideo) -> libvmaf:reference libvmaf -> Stream #0:0 (wrapped_avframe) Press [q] to stop, [?] for help Start calculating VMAF score... Output #0, null, to 'pipe:': Metadata: encoder : Lavf58.17.100 Stream #0:0: Video: wrapped_avframe, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc (default) Metadata: encoder : Lavc58.20.103 wrapped_avframe unknown format ▒I▒/V. Segmentation fault (core dumped)
If I downgrade the VMAF to v.1.3.4 it works:
$ ffmpeg -i distcut.nut -i refcut.nut -lavfi libvmaf="log_fmt=json:log_path=ffvmaf_nut.json:psnr=1:ssim=1:ms_ssim=1" -f null - ffmpeg version N-91330-ga990184 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.9) 20160609 configuration: --pkg-config-flags=--static --extra-libs='-lpthread -lm' --toolchain=hardened --enable-gpl --enable-nonfree --enable-version3 --disable-stripping --enable-avisynth --enable-libass --enable-libfontconfig --enable-libfreetype --enable-libvmaf --enable-libfribidi --enable-libgme --enable-libgsm --enable-librubberband --enable-libshine --enable-libsnappy --enable-libssh --enable-libtwolame --enable-libwavpack --enable-libzvbi --enable-openal --enable-sdl2 --enable-libdrm --enable-frei0r --enable-ladspa --enable-libpulse --enable-libsoxr --enable-libspeex --enable-avfilter --enable-postproc --enable-pthreads --enable-libfdk-aac --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --disable-ffplay --enable-libopenjpeg --enable-libsrt --enable-libwebp --enable-libxvid --enable-libvidstab --enable-libopenh264 --enable-zlib --enable-openssl libavutil 56. 18.102 / 56. 18.102 libavcodec 58. 20.103 / 58. 20.103 libavformat 58. 17.100 / 58. 17.100 libavdevice 58. 4.101 / 58. 4.101 libavfilter 7. 25.100 / 7. 25.100 libswscale 5. 2.100 / 5. 2.100 libswresample 3. 2.100 / 3. 2.100 libpostproc 55. 2.100 / 55. 2.100 Input #0, nut, from 'distcut.nut': Metadata: encoder : Lavf58.12.100 Duration: 00:00:39.98, start: 0.000000, bitrate: 553250 kb/s Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 51200 tbn, 51200 tbc Metadata: encoder : Lavc58.18.100 rawvideo Input #1, nut, from 'refcut.nut': Metadata: encoder : Lavf58.12.100 Duration: 00:00:39.98, start: 0.000000, bitrate: 553250 kb/s Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 51200 tbn, 51200 tbc Metadata: encoder : Lavc58.18.100 rawvideo Stream mapping: Stream #0:0 (rawvideo) -> libvmaf:main Stream #1:0 (rawvideo) -> libvmaf:reference libvmaf -> Stream #0:0 (wrapped_avframe) Press [q] to stop, [?] for help Start calculating VMAF score... Output #0, null, to 'pipe:': Metadata: encoder : Lavf58.17.100 Stream #0:0: Video: wrapped_avframe, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc (default) Metadata: encoder : Lavc58.20.103 wrapped_avframe frame= 2000 fps=2.6 q=-0.0 Lsize=N/A time=00:00:40.00 bitrate=N/A speed=0.0522x video:1047kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown Exec FPS: 2.606700 VMAF score = 91.079469 [libvmaf @ 0x5622212ba880] VMAF score: 91.079469
So apparently something changed in VDK 1.3.5 which is causing segmentation fault when you try to run the filter and to evaluate the VMAF of two files and from VDK 1.3.6 onwards, latest is 1.3.7, ffmpeg is unable to compile with libvmaf support.
comment:11 by , 7 years ago
Version: | unspecified → git-master |
---|
comment:12 by , 7 years ago
Priority: | normal → important |
---|
While this is apparently a libvmaf regression, there is an issue: libvmaf is not only breaking API as in ticket #7248 but possibly also ABI. I wonder if such an unstable wrapper makes sense in FFmpeg.
Is there a warning shown when compiling libavfilter/vf_libvmaf.o
?
by , 7 years ago
Attachment: | compilation_ffmpeg_nightly_VDK_1_3_4.log added |
---|
compilation log of the latest nightly FFMPEG with libvmaf VDK 1.3.5
comment:13 by , 7 years ago
As far as I can see from the log there aren't any errors with libvmaf library in it.
I have added the compilation_ffmpeg_nightly_VDK_1_3_4.log to the attachment list, it is the compilation log of the latest ffmpeg nightly with VMAF VDK 1.3.5. Please note that the name of the file is wrong and it should be 1.3.5, not 1.3.4.
comment:14 by , 7 years ago
As in: Making the answer to a question as difficult to parse as possible?
comment:15 by , 7 years ago
I have modified my comment.
On top of that ffmpeg libvmaf filter is calcuating the PSNR, SSIM, MS-SSIM differently so the average scores when using libvmaf are:
ffmpeg -i distcut.nut -i refcut.nut -lavfi libvmaf="log_fmt=json:log_path=/home/ubuntu/ffvmaf_nut.json:psnr=1:ssim=1:ms_ssim=1" -f null -
ADM2: 0.9828 Motion2: 0.0464 VIF Scale0: 0.6618 VIF Scale1: 0.9437 VIF Scale2: 0.9708 VIF Scale3: 0.9815 VMAF: 91.0795 PSNR: 40.7593dB SSIM: 0.9919 MS-SSIM: 0.9919
The average scores I am getting while running a python script which calculates the average of all the scores from the json log file.
if I run the same metrics in ffmpeg:
ffmpeg -i distcut.nut -i refcut.nut -lavfi "ssim;[0:v][1:v]psnr" -f null – ffmpeg version N-91330-ga990184 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.9) 20160609 configuration: --pkg-config-flags=--static --extra-libs='-lpthread -lm' --toolchain=hardened --enable-gpl --enable-nonfree --enable-version3 --disable-stripping --enable-avisynth --enable-libass --enable-libfontconfig --enable-libfreetype --enable-libvmaf --enable-libfribidi --enable-libgme --enable-libgsm --enable-librubberband --enable-libshine --enable-libsnappy --enable-libssh --enable-libtwolame --enable-libwavpack --enable-libzvbi --enable-openal --enable-sdl2 --enable-libdrm --enable-frei0r --enable-ladspa --enable-libpulse --enable-libsoxr --enable-libspeex --enable-avfilter --enable-postproc --enable-pthreads --enable-libfdk-aac --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --disable-ffplay --enable-libopenjpeg --enable-libsrt --enable-libwebp --enable-libxvid --enable-libvidstab --enable-libopenh264 --enable-zlib --enable-openssl libavutil 56. 18.102 / 56. 18.102 libavcodec 58. 20.103 / 58. 20.103 libavformat 58. 17.100 / 58. 17.100 libavdevice 58. 4.101 / 58. 4.101 libavfilter 7. 25.100 / 7. 25.100 libswscale 5. 2.100 / 5. 2.100 libswresample 3. 2.100 / 3. 2.100 libpostproc 55. 2.100 / 55. 2.100 Input #0, nut, from 'distcut.nut': Metadata: encoder : Lavf58.12.100 Duration: 00:00:39.98, start: 0.000000, bitrate: 553250 kb/s Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 51200 tbn, 51200 tbc Metadata: encoder : Lavc58.18.100 rawvideo Input #1, nut, from 'refcut.nut': Metadata: encoder : Lavf58.12.100 Duration: 00:00:39.98, start: 0.000000, bitrate: 553250 kb/s Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 51200 tbn, 51200 tbc Metadata: encoder : Lavc58.18.100 rawvideo Stream mapping: Stream #0:0 (rawvideo) -> ssim:main Stream #0:0 (rawvideo) -> psnr:main Stream #1:0 (rawvideo) -> ssim:reference Stream #1:0 (rawvideo) -> psnr:reference ssim -> Stream #0:0 (wrapped_avframe) psnr -> Stream #0:1 (wrapped_avframe) Press [q] to stop, [?] for help Output #0, null, to '–': Metadata: encoder : Lavf58.17.100 Stream #0:0: Video: wrapped_avframe, yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc Metadata: encoder : Lavc58.20.103 wrapped_avframe Stream #0:1: Video: wrapped_avframe, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc Metadata: encoder : Lavc58.20.103 wrapped_avframe frame= 2000 fps=764 q=-0.0 Lq=-0.0 size=N/A time=00:00:40.00 bitrate=N/A speed=15.3x video:2094kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown [Parsed_ssim_0 @ 0x55ffa5e9f8c0] SSIM Y:0.969197 (15.114140) U:0.973418 (15.754072) V:0.974990 (16.018821) All:0.970866 (15.356039) [Parsed_psnr_1 @ 0x55ffa5e81700] PSNR y:38.856330 u:43.963544 v:43.765811 average:39.980715 min:27.518272 max:50.428166
So as you can see there is a difference between the average PSNR and SSIM while using the built-in PSNR and SSIM filters and libvmaf.
Please test current FFmpeg git head (nothing else is supported here) and provide complete, uncut console output, backtrace, disassembly and register dump as explained on https://ffmpeg.org/bugreports.html