Skipping frames for GoPro videos recorder with Ambarella

[Den-ffmpeg]

Hi! I am trying to work with GoPro videos made by Ambarella AVC. I tried to convert any video from that camera and output video was lagging.

The problem is old, it was reproduced in version 2.8.1 and has been existing at least from version 2.2.4

How to reproduce:

ffmpeg -i "D:\videos\h264_gopro_ambarella.mp4" -vcodec h264 "D:\videos\lags_on_output_video.avi"
ffmpeg version 2.8.1 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 5.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b
 --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpe
g --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis
--enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100

[Den-ffmpeg]

Input and ouput videos:
​http://www.datafilehost.com/d/314e84e3
​http://www.datafilehost.com/d/dc890fb6

Firstly, I thought the problem was related to the conversion. However, the ffplay plays the original video with lags as well. So I think, the problem is in decoder. VLC and Media Player Classic play video with lags too. But QuickTime and Windows Media Player play the video correctly.

[Carl Eugen Hoyos]

Please test current FFmpeg git head and please provide the command line that allows to reproduce the issue together with the complete, uncut console output to make this a valid ticket.

[Den-ffmpeg]

Thanks for responding! Here is a full log from commit a5202bc made by Ganesh Ajjanagadde:
​http://www.datafilehost.com/d/fe621ab2

The command line is specified in log, but I'll write it here too:

ffmpeg -i "D:\videos\h264_gopro_ambarella.mp4" -vcodec h264 "D:\videos\lags_on_output_video.avi"

[Carl Eugen Hoyos]

The output file you uploaded works fine here, please reopen this ticket if you can provide command line together with the complete, uncut console output and if you can explain what the issue is.

[Den-ffmpeg]

Did you compare the input and output file? I definitely see the difference between them. Please, try to watch them again, for example, in QuickTime or Windows Media Player. Did you look through the attached log? There are command line and uncut console output. And you might see something like that "[h264 @ 05ccd4e0] no picture ooo" there.

I think, the problem is in frames' order. Specified line comes from \libavcodec\h264.c

    if (!out_of_order && pics > h->avctx->has_b_frames) {
        h->next_output_pic = out;
        if (out_idx == 0 && h->delayed_pic[0] && (h->delayed_pic[0]->f->key_frame || h->delayed_pic[0]->mmco_reset)) {
            h->next_outputed_poc = INT_MIN;
        } else
            h->next_outputed_poc = out->poc;
    } else {
        av_log(h->avctx, AV_LOG_DEBUG, "no picture %s\n", out_of_order ? "ooo" : "");
    }

[Carl Eugen Hoyos]

Please provide the command line that allows to reproduce the issue together with its complete, uncut console output here in the bug tracker, do not use external resources.
Please explain what differences you see when "comparing" the output files: Can the difference also be seen with vlc, MPlayer or FFplay?

[Den-ffmpeg]

console output

[Den-ffmpeg]

Sorry for incomprehension, done.
Command line:

ffmpeg -i "D:\\videos\\h264_gopro_ambarella.mp4" -vcodec h264 "D:\\videos\\lags_on_output_video.avi"

Full log in attach

I see that the output video is lagging. If you open both videos, for example, in QuickTime, you will see the difference. On output video is more shaking, that effect is created by the fact, that about 25% frames were skipped. On original file the video is more smooth.

No, there is no difference by playing it with VLC, MPlayer or FFplay (also with Media Player Classic). It is explained that these players use the same ffmpeg h264 decoder. But on QuickTime or Windows Media Player the difference is obvious.

[Den-ffmpeg]

Could you confirm the problem exists? Will there be any action to fix it?

[Carl Eugen Hoyos]

Unfortunately I am unable to reproduce any issue. Please test with -vcodec mpeg4 -qscale 2 to rule out an issue with an external library (x264).

[Den-ffmpeg]

Have the same problem. Full log is attached. Please note that the problem is in decoder, I can see message from it again:

[h264 @ 05680900] no picture ooo

[carriganjohn68]

Hi! I have the same issue: the output video is strongly lagging.
I have a lot of GoPro video files from my customers with the same lagging problem as described in that ticket. I’d like to mention that the popularity of GoPro cameras is growing constantly and it would be great if you fix it asap.

[Carl Eugen Hoyos]

I cannot reproduce an issue with the sample h264_gopro_ambarella.mp4
Please test with current FFmpeg git head (not a month-old version) and provide command line including complete, uncut console output. Please do not use an external library (x264) if this is not necessary to reproduce the issue.

$ md5sum h264_gopro_ambarella.mp4
b7bfb6524ae5cb74ac38f21ae8245c44  h264_gopro_ambarella.mp4
$ ffmpeg -i h264_gopro_ambarella.mp4 -qscale 2 out.avi
ffmpeg version N-77160-g9aebea0 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      55. 10.100 / 55. 10.100
  libavcodec     57. 17.100 / 57. 17.100
  libavformat    57. 19.100 / 57. 19.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 20.100 /  6. 20.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'h264_gopro_ambarella.mp4':
  Metadata:
    major_brand     : avc1
    minor_version   : 0
    compatible_brands: avc1isom
    creation_time   : 2013-01-05 15:25:38
  Duration: 00:00:16.48, start: 0.000000, bitrate: 12270 kb/s
    Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 12126 kb/s, 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc (default)
    Metadata:
      creation_time   : 2013-01-05 15:25:38
      handler_name    :  Ambarella AVC
      encoder         : Ambarella AVC encoder
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2013-01-05 15:25:38
      handler_name    :  Ambarella AAC
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, avi, to 'out.avi':
  Metadata:
    major_brand     : avc1
    minor_version   : 0
    compatible_brands: avc1isom
    ISFT            : Lavf57.19.100
    Stream #0:0(eng): Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 59.94 fps, 59.94 tbn, 59.94 tbc (default)
    Metadata:
      creation_time   : 2013-01-05 15:25:38
      handler_name    :  Ambarella AVC
      encoder         : Lavc57.17.100 mpeg4
    Stream #0:1(eng): Audio: ac3 ([0] [0][0] / 0x2000), 48000 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      creation_time   : 2013-01-05 15:25:38
      handler_name    :  Ambarella AAC
      encoder         : Lavc57.17.100 ac3
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mpeg4 (native))
  Stream #0:1 -> #0:1 (aac (native) -> ac3 (native))
Press [q] to stop, [?] for help
frame=  741 fps=284 q=2.0 Lsize=   31664kB time=00:00:16.48 bitrate=15736.9kbits/s
video:31233kB audio:386kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.143361%

[Carl Eugen Hoyos]

Please ignore, I can reproduce the issue now, will add more information.

[Michael Niedermayer]

Replying to cehoyos:

Please ignore, I can reproduce the issue now, will add more information.

cant reproduce either, how did you reproduce it ?

[Elon Musk]

No samples, no way reproduce this.

[alex_hitman]

Hi everyone!

I'v done some research of this bug.

Sample video has 988 video frames, but h264 decoder outputs only 741 (every 4th is dropped).
Header has VUI parameters with bitstream restrictions, where num_reordered_frames == 1.
But pts is differ from dts by 2. So decoder drops frames because it does not increase reordered buffer to 2 due to bitstream restrictions.

h264.c:decode_postinit()

    } else if(h->avctx->has_b_frames < out_of_order && !sps->bitstream_restriction_flag){
        av_log(h->avctx, AV_LOG_INFO, "Increasing reorder buffer to %d\n", out_of_order);
        h->avctx->has_b_frames = out_of_order;
    }

Here has_b_frames == num_reordered_frames because bitstream_restriction_flag is set.

After dropping VUI bitstream restrictions decoder outputs all frames.

So, the question is what to do.

h264 standard says that VUI parameters are optional and in our case they are probably wrong.
But may exists other cases where it can be helpful.

May be decoder can do more soft checking. For example, use max_dec_frame_buffering. From standard:

The value of num_reorder_frames shall be in the range of 0 to max_dec_frame_buffering,
inclusive. When the num_reorder_frames syntax element is not present,
the value of num_reorder_frames value shall be inferred to be equal
to max_dec_frame_buffering.

So we can trace warning but drop the frames only after reaching max_dec_frame_buffering.

Or may be don't use num_reordered_frames at all. Seems player are not using ffmpeg for decoding (wmp, quicktime), don't pay much attention to bitstream restrictions.

Reuploaded file: ​https://www.datafilehost.com/d/7d7b159e

[Balling]

​https://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket5000/lags_on_output_video.avi is now all 990 frames perfectly (checked with yuview)

and

​https://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket5000/h264_gopro_ambarella.mp4 988 frames but it has

[mov,mp4,m4a,3gp,3g2,mj2 @ 000001e4f14338c0] st: 0 edit list: 1 Missing key frame while searching for timestamp: 3003
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001e4f14338c0] st: 0 edit list 1 Cannot find an index entry before timestamp: 3003.

so with -ignore_editlist 1 -an it is also 990 frames.