Ffmpeg H264 decoder delays I-frames for some streams

[Montes Claros]

Summary of the bug:
Some third party streams don't set num_reorder_frames in SPS. This is causing ffmpeg H264 decoder to delay frames, even I-frames. Those streams don't have B-frames and there is no need for delaying. It would be nice if we could configure Ffmpeg H264 decoder context to signal the streams have no B frames (sps->num_reorder_frames = 0).

[Carl Eugen Hoyos]

Please provide at least a sample and ffmpeg -i console output for the sample.

[Montes Claros]

Thx, cehoyos. Please, see attachments for a sample stream from which frames get delayed although it only has I and P-frames. The workaround below avoids frame delays in streams without B-frames. Some times SPS also has num_reorder_frames != 0, so I think the configuration should precede SPS configuration values. I'm not an expert on this but I understand that somehow this needs to be set in decoder context.

--- a/ffmpeg-3.1.4/libavcodec/h264_ps.c	Fri Nov 25 17:20:08 2016 -0800
+++ b//ffmpeg-3.1.4/libavcodec/h264_ps.c Tue Dec 06 16:48:08 2016 -0800
@@ -566,6 +566,8 @@
         }
     }
 
+	sps->num_reorder_frames = 0;
+
     if (!sps->sar.den)
         sps->sar.den = 1;

Result from ffmpeg -i:

ffmpeg -i test.h264
ffmpeg version 3.1.4 Copyright (c) 2000-2016 the FFmpeg developers
  built with Microsoft (R) C/C++ Optimizing Compiler Version 18.00.40629 for x64
  configuration: --arch=x86_64 --target-os=win64 --enable-ffmpeg --enable-shared --disable-gpl --enable-encoder='mjpeg,mpeg4,pcm_mulaw' --enable-bsfs
--enable-decoder='mjpeg,mpeg4,h264,pcm_mulaw' --enable-filter='amix,aresample,rotate,crop,scale' --enable-muxer=mp4 --enable-demuxer=mov --enable-pars
er=h264 --enable-protocol=file --prefix=/c/ffmpeg-3.1.4/out --toolchain=msvc --extra-ldflags=user32
.lib
  libavutil      55. 28.100 / 55. 28.100
  libavcodec     57. 48.101 / 57. 48.101
  libavformat    57. 41.100 / 57. 41.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 47.100 /  6. 47.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
Input #0, h264, from 'test.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1200k tbn, 60 tbc
At least one output file must be specified

[Montes Claros]

Stream with I and P frame (sps->num_reorder_frames = 2)

[Carl Eugen Hoyos]

Is the issue reproducible with current FFmpeg git head?

[Hendrik]

If the stream specifies num_reorder_frames then clearly the H264 decoder is behaving as expected by obeying that value. It has no way to know that this value is wrong.

[Montes Claros]

Hi heleppkes, this stream is actually not the best sample. If you have sps->bitstream_restriction_flag set to false you would also get delayed frames. It's not really a bug in the decoder but it's convenient when you don't have delayed frames and want to get snapshots of the stream. Cheers, Montez.

[Carl Eugen Hoyos]

Replying to mont3z:

Hi heleppkes, this stream is actually not the best sample.

Please provide a better sample and test current FFmpeg git head to make this a valid ticket.

[Carl Eugen Hoyos]

Please reopen this ticket if you can provide a sample file.