Opened 5 years ago

Closed 8 weeks ago

#7621 closed enhancement (fixed)

Add .avif support (AV1 Image File Format

Reported by: Ewout Owned by: Vignesh Venkatasubramanian
Priority: wish Component: avformat
Version: git-master Keywords: mov
Cc: aros@gmx.com, Mika Fischer Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Netflix released samples of the new AV1 Image File Format (AVIF), it would be nice if support could be added for this format.

Change History (34)

comment:1 by Carl Eugen Hoyos, 5 years ago

Component: undeterminedavformat
Keywords: mov av01 added
Priority: normalwish
Version: unspecifiedgit-master

comment:3 by Ewout, 5 years ago

Current debug commandline if .avif is used as input:

Opening an input file: C:\Users\Ewout\AVIF\Chimera-AV1-8bit-1920x1080-6736kbps-6.avif.
[NULL @ 000001c464c4a780] Opening 'C:\Users\Ewout\AVIF\Chimera-AV1-8bit-1920x1080-6736kbps-6.avif' for reading
[file @ 000001c464c4b900] Setting default whitelist 'file,crypto'
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001c464c4a780] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001c464c4a780] ISO: File Type Major Brand: mif1
    Last message repeated 1 times
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001c464c4a780] moov atom not found
[AVIOContext @ 000001c464c53bc0] Statistics: 66192 bytes read, 3 seeks
C:\Users\Ewout\AVIF\Chimera-AV1-8bit-1920x1080-6736kbps-6.avif: Invalid data found when processing input

Current debug commandline if .avif is used as output:

Opening an output file: png-to-avif.avif.
[NULL @ 0000021f3f4ecc40] Unable to find a suitable output format for 'png-to-avif.avif'
png-to-avif.avif: Invalid argument
[AVIOContext @ 0000021f3f4f3b00] Statistics: 115667 bytes read, 0 seeks

comment:4 by Ewout, 5 years ago

Blocked By: #7600

Depends somewhat on #7600 for lossless encoding.

Microsoft is going to support AVIF in Windows 10.

comment:5 by Carl Eugen Hoyos, 5 years ago

Blocked By: #7600

Definitely independent of ticket #7600.

comment:6 by Ewout, 5 years ago

Aside from using libaom directly for encoding and muxing into an .avif file, libavif could also be considered.

comment:7 by Carl Eugen Hoyos, 5 years ago

Keywords: av01 removed

comment:8 by Artem S. Tashkinov, 4 years ago

Cc: aros@gmx.com added

AV1 decoding support is already there, the container format is probably already supported as well, why doesn't ffmpeg understand AVIF files? ;-)

in reply to:  8 comment:9 by av1masterrace, 3 years ago

Replying to birdie:

AV1 decoding support is already there, the container format is probably already supported as well, why doesn't ffmpeg understand AVIF files? ;-)

It doesn't seem like it. HEIC images aren't supported yet either. See #6521
Would be nice if it got support because it supports animations but doesn't look as trashy as WebP and GIF.

comment:10 by Balling, 3 years ago

Status: newopen

Ticket #7600 is fixed in 288ca1c66740640d5742f2921efbf502d83f8b6a and thus we now have no blocking stuff, AVIF container is implemented here: https://github.com/Swaraj1998/FFmpeg/commit/9a885cddb3550ab863a60d02c5fb78e4ae206cf1 and here: https://patchwork.ffmpeg.org/project/ffmpeg/patch/20170819072456.11276-1-rodger.combs@gmail.com/

Microsoft is going to support AVIF in Windows 10.

That will not help, cause AVIF extensions use MFF and we do not.

Last edited 3 years ago by Balling (previous) (diff)

comment:11 by bat_guano, 3 years ago

Hi

These avif files are showing up a lot now on the web.
I can see them in Chrome Developer tools.
This is an example website ---> https://besteltaart.nl/

As Ewout said 2 years ago, there is a decoder available at ---> https://github.com/AOMediaCodec/libavif

But it would be useful if FFmpeg could decode them too.

~ $ avifdec --version
Version: 0.9.1 (dav1d [dec]:185194b

~ $ avifdec --help
Syntax: avifdec [options] input.avif output.[jpg|jpeg|png|y4m]
        avifdec --info    input.avif

comment:13 by Mika Fischer, 2 years ago

Cc: Mika Fischer added

comment:14 by Balling, 2 years ago

[flac @ 000002129c2059c0] Unknown attached picture mimetype: image/avif.

For this sample https://github.com/ktmf01/flac-test-files/blob/main/subset/59%20-%20AVIF%20PICTURE.flac should be also fixed.

comment:15 by Balling, 23 months ago

It now mostly works except for decoding some samples with error

iloc: item_count > 1 not supported

but that should be fixed in https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220424183514.986374-1-vigneshv@google.com/

Sample file with this error is here:
https://bugzilla.mozilla.org/attachment.cgi?id=9241562

Ecoding should be good, lossless does work with same md5 as original frame!

Last edited 23 months ago by Balling (previous) (diff)

comment:16 by Vignesh Venkatasubramanian, 23 months ago

Resolution: fixed
Status: openclosed

@Balling, thanks for verifying the AVIF support!

Support for still and animated AVIF image muxing/parsing has been merged. There are some patches under review to support more AVIF files.

I think this ticket can be marked as fixed for now since basic support has been implemented. Please feel free to re-open if there are any other issues with AVIF muxing/parsing.

in reply to:  16 ; comment:17 by Balling, 22 months ago

Resolution: fixed
Status: closedreopened

@Vignesh

We need

a) the chroma siting support for top left of 420 right here: https://github.com/AOMediaCodec/av1-avif/issues/88
b) avif MIME
c) the patch for for item_count merged
d) alpha patch support merged
e) HEIC container patches reworked and applied (see HEIF ticket) and HEIF integrated too, since it is so close
f) maybe cpu-used set to 0 and of course defaulting for lossless (crf 0).

Thank you and sorry for reopening, but I found out that AVIF is more effective than JPEG XL since JPEG XL requires R'G'B' convertion which is lossy, dither is hard to compress, but with it turned off rounding of YCbCr is even harder to compress apparently, does not support subsampling like in old JPEG, no support for superblack/superwhite in limited range and out-of-gamut like sYCC, no support for BT.709 matrix like in JPEG 2000, which is just 💩. JPEG XL is useless for me.

Last edited 22 months ago by Balling (previous) (diff)

comment:18 by Balling, 22 months ago

Owner: set to Vignesh Venkatasubramanian
Status: reopenedopen

Assigning to you, see previous comment.

comment:19 by Vignesh Venkatasubramanian, 21 months ago

Hi Balling,

a) the chroma siting support for top left of 420 right here: ​https://github.com/AOMediaCodec/av1-avif/issues/88

What exactly do you need ffmpeg to do for this issue? I am not sure i understand the requirement.

b) avif MIME

Same here. What mime type support do you want in ffmpeg?

c) the patch for for item_count merged
d) alpha patch support merged

These two are now merged. I am working on some follow-up patches for still image alpha and grid image support.

e) HEIC container patches reworked and applied (see HEIF ticket) and HEIF integrated too, since it is so close

I am not very familiar with the HEIC features. I will try to look into it.

f) maybe cpu-used set to 0 and of course defaulting for lossless (crf 0).

You can pass these options now to libaom (or other av1 encoders availabvle on ffmpeg) using the command line options.

comment:20 by Balling, 21 months ago

What exactly do you need ffmpeg to do for this issue? I am not sure i understand the requirement.

All HDR movies use top-left siting for chroma. When you extract a plane 420 samples should be tagged as top-left in AVIF. Use any HDR10 movie. You already pass transfer, matrix and primaries.

What mime type support do you want in ffmpeg?

image/avif, see comment 14.

comment:21 by Vignesh Venkatasubramanian, 21 months ago

image/avif, see comment 14.

I see. It seems like the flac picture parser does not support images with container formats and only works with images that can be directly parsed (like png, jpeg, etc). Since AVIF always lives in a container + codec set up, it might be non-trivial to add support for that in the flac picture parser.

I will leave this as an open item.

comment:22 by Vignesh Venkatasubramanian, 21 months ago

All HDR movies use top-left siting for chroma. When you extract a plane 420 samples should be tagged as top-left in AVIF. Use any HDR10 movie. You already pass transfer, matrix and primaries.

If you are using libaom-av1 encoder for creating AVIF images, you can pass -aom-params "chroma-sample-location=xx" for whatever value you need.

comment:23 by Balling, 21 months ago

If you are using libaom-av1 encoder for creating AVIF images, you can pass -aom-params "chroma-sample-location=xx" for whatever value you need.

It should work automatically.

comment:24 by Vignesh Venkatasubramanian, 21 months ago

It should work automatically.

When encoding, parameters like transfer, matrix, primaries and chroma_sample_positiona are usually set by the user (according to libavcodec documentation).

comment:25 by Balling, 21 months ago

When encoding, parameters like transfer, matrix, primaries and chroma_sample_positiona are usually set by the user (according to libavcodec documentation).

That is false ever since the auto insertion got implemented in 7369595c556516b90bf3acdf85043dc11add7d89 (in practice not everyone migrated to it, like libx264 encoder wrapper and there is a bug when you go from RGB to some of 3 types of YCbCr). The point is it does work though for AVIF. You just forgot to implement chroma_sample_position syntax element (encoded using two bits). See this, looks like center will be implemented using CSP_UNKNOWN. https://github.com/AOMediaCodec/av1-avif/issues/88

in reply to:  17 comment:26 by diltuepromil, 21 months ago

Replying to Balling:

JPEG XL requires R'G'B' convertion which is lossy, dither is hard to compress, but with it turned off rounding of YCbCr is even harder to compress apparently, does not support subsampling like in old JPEG, no support for superblack/superwhite in limited range and out-of-gamut like sYCC, no support for BT.709 matrix like in JPEG 2000

On JXL Discord, BlueSwordM said:

  1. JXL requires RGB conversion

Nope, it converts stuff to XYB in lossy mode at very high bit depth internally, which is basically the best thing psycho-visually speaking, especially since no modern lossy video codecs use RGB(for a good reason).

  1. Dither is hard to compress.

With its current featureset, libjxl with its encoder cjxl is the absolute best encoder I've found to encode any kind of dither efficiently.

  1. Does not support subsampling.

Not a bad thing honestly. We have to move with the times.

  1. No support for BT709 matrix.

Is that even really needed these days since you can just attach the relevant metadata or use something like BT1886?

comment:27 by Balling, 21 months ago

Nope, it converts stuff to XYB in lossy mode at very high

I was only testing lossless mode that does not support XYB.

libjxl with its encoder cjxl is the absolute best encoder I've found to encode any kind of dither efficiently.

No dither needed in AVIF. Dither is applied on decode as it should be.

Not a bad thing honestly. We have to move with the times.

Will not be lossless.

just attach the relevant metadata or use something like BT1886?

Has nothing to do with BT.709 matrix. That is EOTF, transfer.

Last edited 21 months ago by Balling (previous) (diff)

comment:28 by Balling, 21 months ago

So, can you implement chroma_sample_position syntax element?

comment:29 by Balling, 13 months ago

Another ping. chroma_sample_position syntax element must be implemented.

comment:30 by hydra3333, 6 months ago

Just a thought ...
https://trac.ffmpeg.org/ticket/6521#comment:21

which asks about any possibility of ffmpeg supporting libheif
https://github.com/strukturag/libheif

in reply to:  30 comment:31 by Vignesh Venkatasubramanian, 6 months ago

Replying to hydra3333:

Just a thought ...
https://trac.ffmpeg.org/ticket/6521#comment:21

which asks about any possibility of ffmpeg supporting libheif
https://github.com/strukturag/libheif

The ffmpeg MOV parser (and the muxer) has the capability to handle HEIF with minimal changes. I have a patch locally for parsing HEIF. i will try to clean it up and upload it.

I don't think we need to add a seperate dependency for something that ffmpeg almost already natively supports. :)

comment:32 by hydra3333, 4 months ago

Thank you.
I'll keep an eye out for a libheif patch (if it's not already implemented).

comment:34 by Artem S. Tashkinov, 8 weeks ago

Resolution: fixed
Status: openclosed

FFMpeg Next is likely to support it.

Thanks a ton.

Note: See TracTickets for help on using tickets.