Matroska muxer doesn't support H.264/H.265 without global header

[bubbleguuum]

Using FFmpeg 6.1 Android custom build.

Using the MediaCodec h264 encoder for output in a matroska container on stdout fails:

./ffmpeg -hide_banner -i /sdcard/Movies/bigbuckbunny1080p.mp4 -c:v h264_mediacodec 
-profile:v high -level 4.1 -b:v 2000k -an -f matroska - > /dev/null
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/sdcard/Movies/bigbuckbunny1080p.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42isomavc1
    creation_time   : 2013-03-23T02:39:38.000000Z
    encoder         : HandBrake 0.9.8 2012071800
  Duration: 00:09:56.50, start: 0.000000, bitrate: 4421 kb/s
  Stream #0:0[0x1](und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 4087 kb/s, 24 fps, 24 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2013-03-23T02:39:38.000000Z
      vendor_id       : [0][0][0][0]
      encoder         : JVT/AVC Coding
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 329 kb/s (default)
    Metadata:
      creation_time   : 2013-03-23T02:39:38.000000Z
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_mediacodec))
Press [q] to stop, [?] for help
[h264_mediacodec @ 0xb400007959a1bef0] Use 1 as the default MediaFormat i-frame-interval, please set gop_size properly (>= fps)
[h264_mediacodec @ 0xb400007959a1bef0] Mediacodec encoder doesn't support AV_CODEC_FLAG_GLOBAL_HEADER. Use extract_extradata bsf when necessary.
[out#0/matroska @ 0xb4000078f9a1bfd0] Could not write header (incorrect codec parameters ?): Invalid data found when processing input
Error while filtering: Invalid data found when processing input
[out#0/matroska @ 0xb4000078f9a1bfd0] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=N/A bitrate=N/A speed=N/A    
Conversion failed!

It works fine using libx264 encoder:

 ./ffmpeg -i /sdcard/Movies/bigbuckbunny1080p.mp4  -c:v libx264 -preset veryfast 
-profile:v high -level 4.1 -b:v 2000k -an -f matroska - > /dev/null 
...
output omitted
...

It would be great to have it working with h264_mediacodec. Use case is generating an in-memory MKV on the fly for live transcoding.

[quinkblack]

MediaCodec doesn't support global header, both mp4 and flv can handle that, but mkv muxer doesn't.

There is already a workaround in mkv to support AV1 without global header:

    case AV_CODEC_ID_AV1:
        if (extradata_size)
            return ff_isom_write_av1c(dyn_cp, extradata,
                                      extradata_size, 1);
        else
            *size_to_reserve = (AV1_SANE_SEQUENCE_HEADER_MAX_BITS + 7) / 8 + 100;
        break;

We need to the same for H.264/H.265.

However, I'm wondering how to fix it in a generic way, instead of rework in each format.

[bubbleguuum]

Thank you for the detailed reply.

I thought the problem was specific to outputting to stdout, but it looks it is more general than that since outputting to mkv file does not work either.

[bubbleguuum]

Speaking of mp4, although it does not have this issue for outputting to file, it cannot be used to output to stdout as it requires a seekable output by design and stdout is not seekable.
Hence the need for my use case to have mkv working.

If you have a quick and dirty patch that I could use in my custom FFmpeg build, I would gladly take it !

[bubbleguuum]

Inspired by the AV1 snippet, I've tried this hack (using an arbitrary value of 10000 for size_to_reserve) and it seem to produce playable MKV:

case AV_CODEC_ID_H264:
	    if (extradata_size) 
		    return ff_isom_write_avcc(dyn_cp, extradata,
					      extradata_size);
	    else
		    *size_to_reserve = 10000;
	    break;

This will do it for me, until there is eventually a proper fix.

[quinkblack]

Replying to bubbleguuum:

Speaking of mp4, although it does not have this issue for outputting to file, it cannot be used to output to stdout as it requires a seekable output by design and stdout is not seekable.

Yes it can

ffmpeg -i ~/Movies/cctv.mp4 -c copy -movflags cmaf -f mp4 - |ffplay -

Hence the need for my use case to have mkv working.

If you have a quick and dirty patch that I could use in my custom FFmpeg build, I would gladly take it !

[bubbleguuum]

Thanks for this incredible tip, I confirm it works fine in my app as a substitute to mastroka as container (which also now works fine with the hack I mentioned above).

I was not aware of this cmaf flag, which is not documented in the FFmpeg web documentation page, along with other supposedly newer movflags flags.

If audio is transcoded (ie '-c:a libmp3lame'), the delay_moov flag must be specified as well. ffmpeg issues a warning about it otherwise:

[mp4 @ 0xb400007a769146d0] Track 1 starts with a nonzero dts -1105, while the moov already has been written. Set the delay_moov flag to handle this case.

[quinkblack]

FFmepg MediaCodec encoder wrapper can generate extradata now.

9e4991519575b3642e5fa63635f22de8f7c61b8b