Opened 3 years ago

Last modified 3 weeks ago

#9210 new enhancement

Concat images produces wrong timings when duration is in milliseconds

Reported by: demet64 Owned by:
Priority: wish Component: avformat
Version: git-master Keywords: concat
Cc: sycxyc Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description (last modified by demet64)

This issue was also mentioned here: https://www.reddit.com/r/ffmpeg/comments/e1o3hv/ffconcat_filter_with_millisecond_precision_how/ and here: https://forum.videohelp.com/threads/397470-Converting-a-sequence-of-images-to-video-with-variable-framerate

Summary of the bug:
How to reproduce:
A concat file like this

file 000000.jpg
duration 116ms
file 000001.jpg
duration 116ms
file 000002.jpg
duration 116ms
file 000003.jpg
duration 116ms
file 000004.jpg
duration 116ms
file 000005.jpg
duration 116ms
file 000006.jpg
duration 116ms
file 000007.jpg
duration 116ms
file 000008.jpg
duration 116ms
file 000009.jpg
duration 1002ms
file 000010.jpg
duration 116ms
file 000011.jpg
duration 116ms
file 000012.jpg
duration 116ms
file 000013.jpg
duration 1129ms
file 000014.jpg
duration 73ms
file 000015.jpg
duration 73ms
file 000016.jpg
duration 116ms
file 000017.jpg
duration 116ms
file 000018.jpg
duration 2184ms
file 000018.jpg

The output:

ffprobe -f concat -i concat.txt -show_frames | findstr "pkt_pts"
pkt_pts=0
pkt_pts_time=0.000000
pkt_pts=3
pkt_pts_time=0.120000
pkt_pts=6
pkt_pts_time=0.240000
pkt_pts=9
pkt_pts_time=0.360000
pkt_pts=12
pkt_pts_time=0.480000
pkt_pts=15
pkt_pts_time=0.600000
pkt_pts=17
pkt_pts_time=0.680000
pkt_pts=20
pkt_pts_time=0.800000
pkt_pts=23
pkt_pts_time=0.920000
pkt_pts=26
pkt_pts_time=1.040000
pkt_pts=51
pkt_pts_time=2.040000
pkt_pts=54
pkt_pts_time=2.160000
pkt_pts=57
pkt_pts_time=2.280000
pkt_pts=60
pkt_pts_time=2.400000
pkt_pts=88
pkt_pts_time=3.520000
pkt_pts=90
pkt_pts_time=3.600000
pkt_pts=92
pkt_pts_time=3.680000
pkt_pts=95
pkt_pts_time=3.800000
pkt_pts=98
pkt_pts_time=3.920000
pkt_pts=152
pkt_pts_time=6.080000

The output looks correct at first, but timestamps are rounded, and for some reasons the duration here

pkt_pts_time=0.600000
pkt_pts=17
pkt_pts_time=0.680000

is 80ms instead of 120ms (and expected 116ms)

It's not the problem of ffprobe, when converting to webm, for example, or gif

ffmpeg -i concat.txt -c:v libvpx-vp9 -lossless 1 out.webm

timings would be like showed above.

The other case:

file 000000.jpg
duration 50ms
file 000001.jpg
duration 60ms
file 000002.jpg
duration 50ms
file 000003.jpg
duration 60ms
file 000004.jpg
duration 50ms
file 000005.jpg
duration 60ms
file 000006.jpg
duration 50ms
file 000007.jpg
duration 60ms
file 000008.jpg
duration 60ms
file 000009.jpg
duration 50ms
file 000010.jpg
duration 60ms
file 000011.jpg
duration 50ms
file 000012.jpg
duration 60ms
file 000013.jpg
duration 50ms
file 000014.jpg
duration 60ms
file 000015.jpg
duration 50ms
file 000016.jpg
duration 60ms
file 000017.jpg
duration 60ms
file 000018.jpg
duration 50ms
file 000019.jpg
duration 60ms
file 000020.jpg
duration 50ms
file 000021.jpg
duration 60ms
file 000022.jpg
duration 50ms
file 000023.jpg
duration 60ms
file 000023.jpg

Output:

pkt_pts=0
pkt_pts_time=0.000000
pkt_pts=1
pkt_pts_time=0.040000
pkt_pts=3
pkt_pts_time=0.120000
pkt_pts=4
pkt_pts_time=0.160000
pkt_pts=6
pkt_pts_time=0.240000
pkt_pts=7
pkt_pts_time=0.280000
pkt_pts=8
pkt_pts_time=0.320000
pkt_pts=10
pkt_pts_time=0.400000
pkt_pts=11
pkt_pts_time=0.440000
pkt_pts=13
pkt_pts_time=0.520000
pkt_pts=14
pkt_pts_time=0.560000
pkt_pts=15
pkt_pts_time=0.600000
pkt_pts=17
pkt_pts_time=0.680000
pkt_pts=18
pkt_pts_time=0.720000
pkt_pts=19
pkt_pts_time=0.760000
pkt_pts=21
pkt_pts_time=0.840000
pkt_pts=22
pkt_pts_time=0.880000
pkt_pts=24
pkt_pts_time=0.960000
pkt_pts=25
pkt_pts_time=1.000000
pkt_pts=26
pkt_pts_time=1.040000
pkt_pts=28
pkt_pts_time=1.120000
pkt_pts=29
pkt_pts_time=1.160000
pkt_pts=31
pkt_pts_time=1.240000
pkt_pts=32
pkt_pts_time=1.280000
pkt_pts=33
pkt_pts_time=1.320000

Attachments (2)

ffmpeg-20210501-201727.log (111.7 KB ) - added by demet64 3 years ago.
concat.txt (643 bytes ) - added by demet64 3 years ago.

Download all attachments as: .zip

Change History (12)

comment:1 by demet64, 3 years ago

Description: modified (diff)

comment:2 by mkver, 3 years ago

It seems as if this is due to rounding to a timebase of 1/25 somewhere. The exact timestamps in your example would be 0, 116, 232, 348, 464, 580, 696... which get rounded to 0, 120, 240, 360, 480, 600, 680... when using a timebase of 1/25.

comment:3 by demet64, 3 years ago

It seems as if this is due to rounding to a timebase of 1/25 somewhere.

This is what person from reddit thread I mentioned says too. The problem is that it happens on input stream and it seems there's no way you can specify timebase of the input stream

Last edited 3 years ago by demet64 (previous) (diff)

comment:4 by Carl Eugen Hoyos, 3 years ago

Please (test current FFmpeg git head and) provide the command line you tested together with the complete, uncut console output and the necessary input files to make this a valid ticket.

comment:5 by Carl Eugen Hoyos, 3 years ago

Keywords: concat added

by demet64, 3 years ago

Attachment: ffmpeg-20210501-201727.log added

by demet64, 3 years ago

Attachment: concat.txt added

comment:6 by demet64, 3 years ago

I don't know if input images are necessary. I tried the same concat file with different sets of pictures and it didn't have any impact on timecodes.

comment:7 by Andre LaBranche, 2 years ago

The unwanted 25 fps does seem to be coming from concat. I tested that along with image2 and found that image2 honors the frame rate passed as a cli option.

Here's a script that creates some sample images, does a concat run and an image2 run, and collects reports for each.

#!/bin/zsh

OUT=$(mktemp -d)
CATFILE=${OUT}/catfile.txt
pushd ${OUT}
echo "generating some sample images with Imagemagik's 'convert' tool"
for i in {1..60}
do
    IMG=image-$(printf "%03d" ${i}).jpg
    convert -size 600x300 -gravity center -background black \
        -fill purple label:${i} -units PixelsPerInch -density 254 ${IMG}
    echo "file '${IMG}'" >> ${CATFILE}
    echo "duration 0.016" >> ${CATFILE}
done

echo "Wrote sample ffmpeg concat input file: ${CATFILE}"

CONCAT_CMD='ffmpeg -f concat -i ${CATFILE} -report concat.mkv > /dev/null'
export FFREPORT=file=ffprobe-concat.log
echo "Probing concat with:\n${CONCAT_CMD}"
eval ${CONCAT_CMD}
echo "Wrote video ${OUT}/concat.mkv"
echo "Reading frame rate of concat.mkv..."
unset FFREPORT
ffprobe -v error -show_format -show_streams concat.mkv | grep 'frame_rate'

IMAGE2_CMD="ffmpeg -f image2 -framerate 60 -i image-%03d.jpg -report image2.mkv > /dev/null"
echo "\nProbing image2 with:\n${IMAGE2_CMD}"
export FFREPORT=file=ffprobe-image2.log
eval ${IMAGE2_CMD}
echo "Wrote video ${OUT}/image2.mkv"
echo "Reading frame rate of concat.mkv..."
unset FFREPORT
ffprobe -v error -show_format -show_streams image2.mkv | grep 'frame_rate'
echo "\nFull reports for each run: "
ls -1 ${OUT}/ffprobe*.log
open ${OUT}
popd

When run on my machine, it looks like this:

(base) andre@boom matlab_image_tsp % ./duration_snooping.sh 
generating some sample images with Imagemagik's 'convert' tool
Wrote sample ffmpeg concat input file: /var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/catfile.txt
Probing concat with:
ffmpeg -f concat -i ${CATFILE} -report concat.mkv > /dev/null
ffmpeg started on 2021-12-06 at 04:07:22
Report written to "ffprobe-concat.log"
Log level: 48
ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 13.0.0 (clang-1300.0.29.3)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/4.4.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-avresample --enable-videotoolbox
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, concat, from '/var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/catfile.txt':
  Duration: 00:00:00.96, start: 0.000000, bitrate: 18 kb/s
  Stream #0:0: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 600x300 [SAR 254:254 DAR 2:1], 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x12b815400] using SAR=1/1
[libx264 @ 0x12b815400] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x12b815400] profile High 4:4:4 Predictive, level 2.1, 4:4:4, 8-bit
[libx264 @ 0x12b815400] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=9 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'concat.mkv':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Video: h264 (H264 / 0x34363248), yuvj444p(pc, bt470bg/unknown/unknown, progressive), 600x300 [SAR 254:254 DAR 2:1], q=2-31, 25 fps, 1k tbn
    Metadata:
      encoder         : Lavc58.134.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=   26 fps=0.0 q=-1.0 Lsize=      14kB time=00:00:00.92 bitrate= 121.6kbits/s dup=0 drop=34 speed=13.4x    
video:13kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 6.463387%
[libx264 @ 0x12b815400] frame I:1     Avg QP:19.46  size:   495
[libx264 @ 0x12b815400] frame P:12    Avg QP:17.97  size:   657
[libx264 @ 0x12b815400] frame B:13    Avg QP:22.41  size:   314
[libx264 @ 0x12b815400] consecutive B-frames: 11.5% 61.5% 11.5% 15.4%
[libx264 @ 0x12b815400] mb I  I16..4:  4.2% 93.6%  2.2%
[libx264 @ 0x12b815400] mb P  I16..4:  1.6%  2.6%  1.4%  P16..4:  8.6%  3.2%  0.5%  0.0%  0.0%    skip:82.1%
[libx264 @ 0x12b815400] mb B  I16..4:  0.8%  1.3%  0.3%  B16..8:  9.7%  2.8%  0.1%  direct: 0.0%  skip:84.9%  L0:38.7% L1:60.7% BI: 0.7%
[libx264 @ 0x12b815400] 8x8 transform intra:71.4% inter:27.3%
[libx264 @ 0x12b815400] coded y,u,v intra: 5.0% 4.1% 4.5% inter: 0.5% 0.3% 0.4%
[libx264 @ 0x12b815400] i16 v,h,dc,p: 53% 38%  8%  0%
[libx264 @ 0x12b815400] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 31%  9% 58%  1%  0%  0%  0%  0%  0%
[libx264 @ 0x12b815400] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 21% 47%  3%  2%  2%  2%  1%  2%
[libx264 @ 0x12b815400] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x12b815400] ref P L0: 30.0%  7.6% 51.8% 10.6%
[libx264 @ 0x12b815400] ref B L0: 75.2% 24.0%  0.7%
[libx264 @ 0x12b815400] ref B L1: 96.8%  3.2%
[libx264 @ 0x12b815400] kb/s:95.87
Wrote video /var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/concat.mkv
Reading frame rate of concat.mkv...
r_frame_rate=25/1
avg_frame_rate=25/1

Probing image2 with:
ffmpeg -f image2 -framerate 60 -i image-%03d.jpg -report image2.mkv > /dev/null
ffmpeg started on 2021-12-06 at 04:07:22
Report written to "ffprobe-image2.log"
Log level: 48
ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 13.0.0 (clang-1300.0.29.3)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/4.4.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-avresample --enable-videotoolbox
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, image2, from 'image-%03d.jpg':
  Duration: 00:00:01.00, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 600x300 [SAR 254:254 DAR 2:1], 60 fps, 60 tbr, 60 tbn, 60 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x14a01e400] using SAR=1/1
[libx264 @ 0x14a01e400] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x14a01e400] profile High 4:4:4 Predictive, level 3.1, 4:4:4, 8-bit
[libx264 @ 0x14a01e400] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=9 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'image2.mkv':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Video: h264 (H264 / 0x34363248), yuvj444p(pc, bt470bg/unknown/unknown, progressive), 600x300 [SAR 254:254 DAR 2:1], q=2-31, 60 fps, 1k tbn
    Metadata:
      encoder         : Lavc58.134.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=   60 fps=0.0 q=-1.0 Lsize=      26kB time=00:00:00.95 bitrate= 221.8kbits/s speed=13.7x    
video:25kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 4.316689%
[libx264 @ 0x14a01e400] frame I:1     Avg QP:22.02  size:   415
[libx264 @ 0x14a01e400] frame P:21    Avg QP:21.83  size:   617
[libx264 @ 0x14a01e400] frame B:38    Avg QP:26.79  size:   295
[libx264 @ 0x14a01e400] consecutive B-frames:  6.7% 10.0% 50.0% 33.3%
[libx264 @ 0x14a01e400] mb I  I16..4:  3.9% 93.9%  2.2%
[libx264 @ 0x14a01e400] mb P  I16..4:  2.0%  3.2%  1.7%  P16..4:  5.9%  3.4%  0.5%  0.0%  0.0%    skip:83.3%
[libx264 @ 0x14a01e400] mb B  I16..4:  1.1%  1.5%  0.3%  B16..8:  9.9%  2.0%  0.0%  direct: 0.0%  skip:85.2%  L0:58.0% L1:41.8% BI: 0.2%
[libx264 @ 0x14a01e400] 8x8 transform intra:61.5% inter:34.4%
[libx264 @ 0x14a01e400] coded y,u,v intra: 5.6% 4.4% 5.2% inter: 0.3% 0.3% 0.4%
[libx264 @ 0x14a01e400] i16 v,h,dc,p: 62% 34%  5%  0%
[libx264 @ 0x14a01e400] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23%  7% 67%  1%  0%  0%  0%  0%  0%
[libx264 @ 0x14a01e400] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 20% 41%  5%  2%  2%  2%  2%  1%
[libx264 @ 0x14a01e400] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x14a01e400] ref P L0: 43.2% 11.0% 31.0% 14.9%
[libx264 @ 0x14a01e400] ref B L0: 62.0% 28.3%  9.7%
[libx264 @ 0x14a01e400] ref B L1: 95.4%  4.6%
[libx264 @ 0x14a01e400] kb/s:196.69
Wrote video /var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/image2.mkv
Reading frame rate of concat.mkv...
r_frame_rate=60/1
avg_frame_rate=60/1

Full reports for each run: 
/var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/ffprobe-concat.log
/var/folders/sc/q27bbsbn105gr0d08r3nd2jr0000gn/T/tmp.PU7y4SbK/ffprobe-image2.log

Here are the two reports from the above run:

Last edited 2 years ago by Andre LaBranche (previous) (diff)

in reply to:  7 comment:8 by Cigaes, 2 years ago

Replying to Andre LaBranche:

The unwanted 25 fps does seem to be coming from concat. I tested that along with image2 and found that image2 honors the frame rate passed as a cli option.

image2 honors the frame_rate option, but you never gave that option to the image2 instances below the concat demuxer.

I would not oppose a patch to concat to add a time_base option.

comment:9 by Carl Eugen Hoyos, 17 months ago

Component: undeterminedavformat
Priority: normalwish
Type: defectenhancement
Version: unspecifiedgit-master

comment:10 by sycxyc, 3 weeks ago

Cc: sycxyc added
Note: See TracTickets for help on using tickets.