PMW-EX3 camera raw videos (mpeg2video/pcm_s16be/MP4): A/V out of sync in quicktime after transcoding to h264 with ffmpeg

[ls]

• When transcoding raw videos from a PMW-EX3 camera the resulting videos are about 1-2 frames out of sync when playing them with Quicktime 7 (tested on OSX Snow Leopard and Windows).
• Since the problem does not seem to occur in ffplay, mplayer, or windows media player it might be a Quicktime specific problem and not a ffmpeg bug.
• Maybe it is of interest that after importing the camera data with Final Cut (FCP 6.0.6) and compressing to h264 there is no A/V sync issue in Quicktime or ffplay.

ffmpeg call and output

ffmpeg -i camera-raw.mp4 -async 1 -codec:v libx264 -preset medium -profile:v baseline -threads 0 -crf 20 -codec:a libfaac -ab 256k ffmpeg-result.mp4
ffmpeg version N-31257-g6a3f172 Copyright (c) 2000-2012 the FFmpeg developers
  built on Feb 20 2012 22:08:11 with gcc 4.4.3
  configuration: --prefix=/homes/lschilli/ffmpeg --enable-shared --disable-static --enable-gpl --enable-nonibmp3lame --enable-libx264 --enable-libxvid --enable-sram --cpu=core2 --enable-hwaccel=h264_vaapi --enable-aapi --enable-hwaccel=mpeg4_vaapi --enable-libfaac --enable-x11grab --disable-ffserver --enable-libass --ene
  libavutil      51. 39.100 / 51. 39.100
  libavcodec     54.  3.101 / 54.  3.101
  libavformat    54.  1.100 / 54.  1.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 62.101 /  2. 62.101
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  7.100 /  0.  7.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'camera-raw.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42
    creation_time   : 2012-02-13 23:21:42
  Duration: 00:13:40.80, start: 0.000000, bitrate: 1022 kb/s
    Stream #0:0(eng): Video: mpeg2video (Main) (mp4v / 0x7634706D), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], fps, 25 tbr, 25k tbn, 50 tbc
    Metadata:
      creation_time   : 2012-02-13 23:21:42
      handler_name    : Video Media Handler
    Stream #0:1(eng): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, 2 channels, s16, 1536 kb/s
    Metadata:
      creation_time   : 2012-02-13 23:21:42
      handler_name    : NRT Metadata Handler
[buffer @ 0x84ce6e0] w:1920 h:1080 pixfmt:yuv420p tb:1/1000000 sar:1/1 sws_param:
[libx264 @ 0x84cefe0] using SAR=1/1
[libx264 @ 0x84cefe0] using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cache64
[libx264 @ 0x84cefe0] profile Constrained Baseline, level 4.0
[libx264 @ 0x84cefe0] 264 - core 120 r0+2 a3f4407 - H.264/MPEG-4 AVC codec - Copyleft 2003-2011 - http://wwx264.html - options: cabac=0 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1 psy_rd=1.00:0.00 minge=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=3 slice0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 ra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=20.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq
Output #0, mp4, to 'ffmpeg-result.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42
    creation_time   : 2012-02-13 23:21:42
    encoder         : Lavf54.1.100
    Stream #0:0(eng): Video: h264 (![0][0][0] / 0x0021), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 25 tbn, 25 tbc
    Metadata:
      creation_time   : 2012-02-13 23:21:42
      handler_name    : Video Media Handler
    Stream #0:1(eng): Audio: aac (@[0][0][0] / 0x0040), 48000 Hz, 2 channels, s16, 256 kb/s
    Metadata:
      creation_time   : 2012-02-13 23:21:42
      handler_name    : NRT Metadata Handler
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video -> libx264)
  Stream #0:1 -> #0:1 (pcm_s16be -> libfaac)
Press [q] to stop, [?] for help
[mpeg2video @ 0x84a4440] warning: first frame is no keyframe
    Last message repeated 1 times
Truncating packet of size 124464 to 34592B time=00:00:21.04 bitrate=8901.7kbits/s dup=1 drop=0     
[mpeg2video @ 0x84a4440] ac-tex damaged at 63 16
[mpeg2video @ 0x84a4440] Warning MVs not available
[mpeg2video @ 0x84a4440] concealing 6240 DC, 6240 AC, 6240 MV errors
Truncating packet of size 125431 to 1
frame=  574 fps=  6 q=-1.0 Lsize=   24571kB time=00:00:22.96 bitrate=8766.9kbits/s dup=2 drop=0    
video:24092kB audio:465kB global headers:0kB muxing overhead 0.056445%
[libx264 @ 0x84cefe0] frame I:4     Avg QP:17.82  size:132915
[libx264 @ 0x84cefe0] frame P:570   Avg QP:20.57  size: 42347
[libx264 @ 0x84cefe0] mb I  I16..4: 33.5%  0.0% 66.5%
[libx264 @ 0x84cefe0] mb P  I16..4:  1.5%  0.0%  1.8%  P16..4: 63.2% 12.3%  6.1%  0.0%  0.0%    skip:15.1%
[libx264 @ 0x84cefe0] coded y,uvDC,uvAC intra: 60.9% 73.0% 16.5% inter: 22.1% 50.9% 0.5%
[libx264 @ 0x84cefe0] i16 v,h,dc,p: 16% 18% 27% 39%
[libx264 @ 0x84cefe0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 23% 23%  5%  7%  6%  6%  4%  6%
[libx264 @ 0x84cefe0] i8c dc,h,v,p: 57% 22% 18%  3%
[libx264 @ 0x84cefe0] ref P L0: 55.8% 24.6% 19.6%
[libx264 @ 0x84cefe0] kb/s:8595.64

Sample files

​Camera raw data
​ffmpeg result
​Final Cut import
​H264 created by Final Cut / Compressor

• Large (MPEG2) samples have been processed with qt-faststart and truncated at 100MB.
• Side note: The truncated file after-fcp-import.mov seems to playable in ffplay but transcoding with ffmpeg stops after 0.96s (Truncating packet of size 4 to 1).

Workaround

Adding "-vf setpts=PTS-STARTPTS" as workaround seems to reduce the problem:

ffmpeg -i camera-raw.mp4 -vf setpts=PTS-STARTPTS -async 1 -codec:v libx264 -preset medium -profile:v baseline -threads 0 -crf 20 -codec:a libfaac -ab 256k ffmpeg-result.mp4

The first frame ffmpeg reads has a pts_time of 0.04 according to ffmpeg:

[showinfo @ 0x1a2c120] n:0 pts:1000 pts_time:0.04 pos:746083 fmt:yuv420p sar:1/1 s:1920x1080 i:P iskey:0 type:B checksum:0C66BC18 plane_checksum:[8FDB61DF 549CD599 349F8491]

Maybe this is related to the problem.