h264_nvenc / b_ref_mode middle with refs parameter is not working

[smallishzulu]

Hello,

b_ref_mode option with refs parameter is not working, giving out below error:
InitializeEncoder failed: invalid param (8)

Is it normal ? ref frame number should be bf/2 b setting b_ref_mode to middle.

/opt/ffmpeg/bin/ffmpeg -loglevel verbose -re -i /home/admintv/big-buck-bunny_294_1280x720.mp4  -aspect 16:9 -s 720x576 -map 0:0  -c:v:0 h264_nvenc -temporal-aq 1 -spatial-aq 1 -aq-strength 15 -2pass 1 -preset slow  -cbr 1 -rc cbr  -profile:v main   -vb 1550k -minrate 1550k -maxrate 1550k -bufsize 1550k -muxrate 1790k    -flags +ilme+ildct -top 1  -r 25 -pix_fmt yuv420p -map 0:1 -c:a:0 aac  -b:a:0 128k -g 50 -f mpegts -bf 4 -refs 2 -b_ref_mode middle 'udp://192.168.2.100:5000?pkt_size=188'
ffmpeg version N-91062-gf995aa8 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
  configuration: --prefix=/opt/ffmpeg --enable-nonfree --enable-gpl --extra-cflags='-I/opt/ffmpeg/include -I/usr/local/include -I/usr/local/cuda/include -I/opt/ndi/sdk/include' --extra-ldflags='-L/opt/ffmpeg/lib -L/usr/local/cuda/lib64 -L/opt/ndi/sdk/lib' --bindir=/opt/ffmpeg/bin --extra-libs=-ldl --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl --enable-nvenc --enable-libzvbi --enable-libfdk-aac --enable-libzimg --enable-libzmq --enable-libfreetype --enable-static --enable-shared --enable-hardcoded-tables --enable-vdpau --enable-cuda --enable-cuvid --enable-libvpx --enable-libmp3lame --enable-libndi_newtek --enable-openssl --enable-ffnvcodec --enable-libfontconfig --enable-libfribidi --enable-cuda-sdk --enable-libnpp
  libavutil      56. 18.102 / 56. 18.102
  libavcodec     58. 19.101 / 58. 19.101
  libavformat    58. 13.102 / 58. 13.102
  libavdevice    58.  4.100 / 58.  4.100
  libavfilter     7. 22.100 /  7. 22.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
[h264 @ 0x1cbf640] Reinit context to 1280x720, pix_fmt: yuv420p
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/admintv/big-buck-bunny_294_1280x720.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: mp41
    creation_time   : 1970-01-01T00:00:00.000000Z
    title           : Big Buck Bunny
    encoder         : Lavf52.13.0
  Duration: 00:09:56.46, start: 0.000000, bitrate: 2353 kb/s
    Stream #0:0(und): Video: h264 (Constrained Baseline), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 2220 kb/s, 24 fps, 24 tbr, 24 tbn, 48 tbc (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[h264 @ 0x1d2dcc0] Reinit context to 1280x720, pix_fmt: yuv420p
[graph_1_in_0_1 @ 0x2023600] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:0x3
[graph 0 input from stream 0:0 @ 0x2642b00] w:1280 h:720 pixfmt:yuv420p tb:1/24 fr:24/1 sar:1/1 sws_param:flags=2
[scaler_out_0_0 @ 0x2643e40] w:720 h:576 flags:'bicubic' interl:0
[scaler_out_0_0 @ 0x2643e40] w:1280 h:720 fmt:yuv420p sar:1/1 -> w:720 h:576 fmt:yuv420p sar:64/45 flags:0x4
[h264_nvenc @ 0x1ce7280] Loaded Nvenc version 8.1
[h264_nvenc @ 0x1ce7280] Nvenc initialized successfully
[h264_nvenc @ 0x1ce7280] 1 CUDA capable devices found
[h264_nvenc @ 0x1ce7280] [ GPU #0 - < GeForce GTX 1080 > has Compute SM 6.1 ]
[h264_nvenc @ 0x1ce7280] supports NVENC
[h264_nvenc @ 0x1ce7280] AQ enabled.
[h264_nvenc @ 0x1ce7280] Temporal AQ enabled.
[h264_nvenc @ 0x1ce7280] InitializeEncoder failed: invalid param (8)
[h264_nvenc @ 0x1ce7280] Nvenc unloaded
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x1cc2340] Qavg: 188.848
[aac @ 0x1cc2340] 2 frames left in the queue on closing
Conversion failed!

[Carl Eugen Hoyos]

Why do you think this can be fixed in FFmpeg?

[smallishzulu]

In libavcodec/h264_nvenc.c, b_ref_mode "middle" explanation is:

Only (number of B frames)/2 will be used for reference

What I understand from here is like ref frame number will be b frame number divided by two.
Like if b frame number is 4, than ref frame number should be 2.

A normal user can try to pass -bf 4 -refs 2, I just opened this ticket to ignore -refs parameter if b_ref_mode & bf is used. (This is what I think)

[smallishzulu]

Official NVIDIA reply to this ticket over their ticket system:

The combination of options "-bf 4 -refs 2 -b_ref_mode middle" is invalid. The -refs option is a generic ffmpeg option for controlling the maximum DPB size; it does not specify the index of the B-frame that should be used as reference. The B-frame to be used as a reference frame is selected by the driver (for the "middle" mode, it will be number_of_b_frames / 2). Further, using B-frames as reference requires a DPB size of at least 3 (2 slots for the reference frames on which the B-frame would depend and one slot for the B-frame itself), so in this specific case (-refs 2), the DPB is not large enough for use with the B-frames as reference feature.

However if I set -refs 3, than FFmpeg give out Invalid DTS errors:

/opt/ffmpeg/bin/ffmpeg -loglevel verbose -re -i /home/admintv/big-buck-bunny_294_1280x720.mp4 -aspect 16:9 -s 720x576 -map 0:0 -c:v:0 h264_nvenc -temporal-aq 1 -spatial-aq 1 -aq-strength 15 -2pass 1 -preset slow -cbr 1 -rc cbr -profile:v main -vb 1550k -minrate 1550k -maxrate 1550k -bufsize 1550k -muxrate 1790k -flags +ilme+ildct -top 1 -r 25 -pix_fmt yuv420p -map 0:1 -c:a:0 aac -b:a:0 128k -g 50 -f mpegts -bf 4 -refs 3 -b_ref_mode middle 'udp://192.168.2.100:5000?pkt_size=188'
ffmpeg version N-91062-gf995aa8 Copyright (c) 2000-2018 the FFmpeg developers

built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
configuration: --prefix=/opt/ffmpeg --enable-nonfree --enable-gpl --extra-cflags='-I/opt/ffmpeg/include -I/usr/local/include -I/usr/local/cuda/include -I/opt/ndi/sdk/include' --extra-ldflags='-L/opt/ffmpeg/lib -L/usr/local/cuda/lib64 -L/opt/ndi/sdk/lib' --bindir=/opt/ffmpeg/bin --extra-libs=-ldl --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl --enable-nvenc --enable-libzvbi --enable-libfdk-aac --enable-libzimg --enable-libzmq --enable-libfreetype --enable-static --enable-shared --enable-hardcoded-tables --enable-vdpau --enable-cuda --enable-cuvid --enable-libvpx --enable-libmp3lame --enable-libndi_newtek --enable-openssl --enable-ffnvcodec --enable-libfontconfig --enable-libfribidi --enable-cuda-sdk --enable-libnpp
libavutil 56. 18.102 / 56. 18.102
libavcodec 58. 19.101 / 58. 19.101
libavformat 58. 13.102 / 58. 13.102
libavdevice 58. 4.100 / 58. 4.100
libavfilter 7. 22.100 / 7. 22.100
libswscale 5. 2.100 / 5. 2.100
libswresample 3. 2.100 / 3. 2.100
libpostproc 55. 2.100 / 55. 2.100

[h264 @ 0x920640] Reinit context to 1280x720, pix_fmt: yuv420p
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/admintv/big-buck-bunny_294_1280x720.mp4':

Metadata:

major_brand : isom
minor_version : 512
compatible_brands: mp41
creation_time : 1970-01-01T00:00:00.000000Z
title : Big Buck Bunny
encoder : Lavf52.13.0

Duration: 00:09:56.46, start: 0.000000, bitrate: 2353 kb/s

Stream #0:0(und): Video: h264 (Constrained Baseline), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 2220 kb/s, 24 fps, 24 tbr, 24 tbn, 48 tbc (default)
Metadata:

creation_time : 1970-01-01T00:00:00.000000Z
handler_name : VideoHandler

Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (default)
Metadata:

creation_time : 1970-01-01T00:00:00.000000Z
handler_name : SoundHandler

Stream mapping:

Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
Stream #0:1 -> #0:1 (aac (native) -> aac (native))

Press [q] to stop, ? for help
[h264 @ 0x98ecc0] Reinit context to 1280x720, pix_fmt: yuv420p
[graph_1_in_0_1 @ 0xc84600] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:0x3
[graph 0 input from stream 0:0 @ 0x12a3b00] w:1280 h:720 pixfmt:yuv420p tb:1/24 fr:24/1 sar:1/1 sws_param:flags=2
[scaler_out_0_0 @ 0x12a4e40] w:720 h:576 flags:'bicubic' interl:0
[scaler_out_0_0 @ 0x12a4e40] w:1280 h:720 fmt:yuv420p sar:1/1 -> w:720 h:576 fmt:yuv420p sar:64/45 flags:0x4
[h264_nvenc @ 0x948280] Loaded Nvenc version 8.1
[h264_nvenc @ 0x948280] Nvenc initialized successfully
[h264_nvenc @ 0x948280] 1 CUDA capable devices found
[h264_nvenc @ 0x948280] [ GPU #0 - < GeForce GTX 1080 > has Compute SM 6.1 ]
[h264_nvenc @ 0x948280] supports NVENC
[h264_nvenc @ 0x948280] AQ enabled.
[h264_nvenc @ 0x948280] Temporal AQ enabled.
[mpegts @ 0x948d40] muxrate 1790000, pcr every 23 pkts, sdt every 595, pat/pmt every 119 pkts
Output #0, mpegts, to 'udp://192.168.2.100:5000?pkt_size=188':

Metadata:

major_brand : isom
minor_version : 512
compatible_brands: mp41
title : Big Buck Bunny
encoder : Lavf58.13.102
Stream #0:0(und): Video: h264 (h264_nvenc) (Main), 1 reference frame, yuv420p(top first, left), 720x576 [SAR 64:45 DAR 16:9], q=-1--1, 1550 kb/s, 25 fps, 90k tbn, 25 tbc (default)
Metadata:

creation_time : 1970-01-01T00:00:00.000000Z
handler_name : VideoHandler
encoder : Lavc58.19.101 h264_nvenc

Side data:

cpb: bitrate max/min/avg: 1550000/0/1550000 buffer size: 1550000 vbv_delay: -1

Stream #0:1(und): Audio: aac (LC), 48000 Hz, stereo, fltp, delay 1024, 128 kb/s (default)
Metadata:

creation_time : 1970-01-01T00:00:00.000000Z
handler_name : SoundHandler
encoder : Lavc58.19.101 aac

[mpegts @ 0x948d40] Invalid DTS: 7200 PTS: 3600 in output stream 0:0, replacing by guess.468x
[mpegts @ 0x948d40] Invalid DTS: 10800 PTS: 7200 in output stream 0:0, replacing by guess
[mpegts @ 0x948d40] Invalid DTS: 25200 PTS: 21600 in output stream 0:0, replacing by guess
[mpegts @ 0x948d40] Invalid DTS: 28800 PTS: 25200 in output stream 0:0, replacing by guess
[mpegts @ 0x948d40] Invalid DTS: 46800 PTS: 39600 in output stream 0:0, replacing by guess957x
[mpegts @ 0x948d40] Invalid DTS: 50400 PTS: 46800 in output stream 0:0, replacing by guess
[mpegts @ 0x948d40] Invalid DTS: 64800 PTS: 61200 in output stream 0:0, replacing by guess
[mpegts @ 0x948d40] Invalid DTS: 68400 PTS: 64800 in output stream 0:0, replacing by guess
[mpegts @ 0x948d40] Invalid DTS: 82800 PTS: 79200 in output stream 0:0, replacing by guess97x
[mpegts @ 0x948d40] Invalid DTS: 86400 PTS: 82800 in output stream 0:0, replacing by guess
[mpegts @ 0x948d40] Invalid DTS: 100800 PTS: 97200 in output stream 0:0, replacing by guess
[mpegts @ 0x948d40] Invalid DTS: 104400 PTS: 100800 in output stream 0:0, replacing by guess
[mpegts @ 0x948d40] Invalid DTS: 118800 PTS: 115200 in output stream 0:0, replacing by guess
[mpegts @ 0x948d40] Invalid DTS: 122400 PTS: 118800 in output stream 0:0, replacing by guess
[mpegts @ 0x948d40] Invalid DTS: 140400 PTS: 136800 in output stream 0:0, replacing by guess
[mpegts @ 0x948d40] Invalid DTS: 144000 PTS: 140400 in output stream 0:0, replacing by guess
[mpegts @ 0x948d40] Invalid DTS: 158400 PTS: 154800 in output stream 0:0, replacing by guess
frame= 44 fps= 18 q=28.0 Lsize= 519kB time=00:00:02.43 bitrate=1747.7kbits/s speed=0.99x
video:404kB audio:39kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 17.295065%
Input file #0 (/home/admintv/big-buck-bunny_294_1280x720.mp4):

Input stream #0:0 (video): 59 packets read (631613 bytes); 45 frames decoded;
Input stream #0:1 (audio): 114 packets read (38584 bytes); 113 frames decoded (115712 samples);
Total: 173 packets (670197 bytes) demuxed

Output file #0 (udp://192.168.2.100:5000?pkt_size=188):

Output stream #0:0 (video): 44 frames encoded; 44 packets muxed (413320 bytes);
Output stream #0:1 (audio): 113 frames encoded (115712 samples); 114 packets muxed (39630 bytes);
Total: 158 packets (452950 bytes) muxed

[h264_nvenc @ 0x948280] Nvenc unloaded
[aac @ 0x923340] Qavg: 163.924
Exiting normally, received signal 2.

And video is broken as attached.

[Balling]

Lets merge this to #7303. As -refs is already renamed, etc. ​https://github.com/FFmpeg/FFmpeg/commit/e929b2f248a9c49761475e07d1a089e05912d1c6