Opened 13 years ago

Last modified 8 weeks ago

#979 open defect

Abnormal colorspace conversion of BGR -> YUV comparing the RGB variant

Reported by: natt Owned by:
Priority: normal Component: swscale
Version: git-master Keywords: scale
Cc: human.peng@gmail.com, MasterQuestionable Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Input file is bgr24. Output file is yuv420p. If I instruct ffmpeg to convert to rgb24 first (instead of bgr24->yuv420p directly), the colors are slightly different.

Test input, exact command lines, and console output attached.

Attachments (5)

testcase.c (5.5 KB ) - added by natt 13 years ago.
testin.raw (192.0 KB ) - added by natt 13 years ago.
output of testcase executable
gfajfa.bmp (192.1 KB ) - added by Balling 4 months ago.
white.bmp.zip (7.3 KB ) - added by fireattack 4 months ago.
bgr.bmp (1.8 MB ) - added by Balling 4 months ago.

Change History (29)

by natt, 13 years ago

Attachment: testcase.c added

by natt, 13 years ago

Attachment: testin.raw added

output of testcase executable

comment:1 by Elon Musk, 13 years ago

Whatever issue you are reporting I can not reproduce it.
rgb24(->bgr24)->yuv is fine here

in reply to:  1 comment:2 by natt, 13 years ago

Replying to richardpl:

Whatever issue you are reporting I can not reproduce it.
rgb24(->bgr24)->yuv is fine here

The two conversions should be identical, right? Since rgb24<->bgr24 is lossless, bgr->rgb->yuv should produce the same output as bgr->yuv. If it doesn't, it's not working right.

ffmpeg -f rawvideo -video_size 256x256 -pix_fmt bgr24 -i testin.raw -sws_flags lanczos -f rawvideo -pix_fmt yuv420p md5:
0ba2a7494930baaa64869462d54b85cc

ffmpeg -f rawvideo -video_size 256x256 -pix_fmt bgr24 -i testin.raw -filter format=rgb24 -sws_flags lanczos -f rawvideo -pix_fmt yuv420p md5:
d5bd2d94675b67d23eb2661ca462966d

comment:3 by Carl Eugen Hoyos, 13 years ago

Status: newopen
Version: unspecifiedgit-master

If there is an issue, is is reproducible with this command:

$ ./ffmpeg -i tests/lena.pnm -pix_fmt yuv420p -f crc - -vf format=bgr24 -pix_fmt yuv420p -f crc -
ffmpeg version N-37568-g8162c6f Copyright (c) 2000-2012 the FFmpeg developers
  built on Feb  5 2012 16:31:32 with gcc 4.5.3
  configuration: --cc='/usr/local/gcc-4.5.3/bin/gcc -m32'
  libavutil      51. 37.100 / 51. 37.100
  libavcodec     54.  0.102 / 54.  0.102
  libavformat    54.  0.100 / 54.  0.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 61.100 /  2. 61.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  6.100 /  0.  6.100
Input #0, image2, from 'tests/lena.pnm':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: ppm, rgb24, 256x256, 25 tbr, 25 tbn, 25 tbc
[buffer @ 0x8dd20a0] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 0x8dd2360] auto-inserting filter 'auto-inserted scale 0' between the filter 'src' and the filter 'out'
[scale @ 0x8dd28e0] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:yuv420p flags:0x4
[buffer @ 0x8de20e0] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 0x8de1780] auto-inserting filter 'auto-inserted scale 1' between the filter 'Parsed_format_0' and the filter 'out'
[format @ 0x8de0a80] auto-inserting filter 'auto-inserted scale 2' between the filter 'src' and the filter 'Parsed_format_0'
[scale @ 0x8df3960] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:bgr24 flags:0x4
[scale @ 0x8de0b40] w:256 h:256 fmt:bgr24 -> w:256 h:256 fmt:yuv420p flags:0x4
Output #0, crc, to 'pipe:':
  Metadata:
    encoder         : Lavf54.0.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31, 200 kb/s, 90k tbn, 25 tbc
Output #1, crc, to 'pipe:':
  Metadata:
    encoder         : Lavf54.0.100
    Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31, 200 kb/s, 90k tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (ppm -> rawvideo)
  Stream #0:0 -> #1:0 (ppm -> rawvideo)
Press [q] to stop, [?] for help
CRC=0x7db3d981
CRC=0x74eb2924
frame=    1 fps=  0 q=0.0 Lq=0.0 size=       0kB time=00:00:00.04 bitrate=   3.0kbits/s
video:192kB audio:0kB global headers:0kB muxing overhead -99.992371%

in reply to:  3 comment:4 by natt, 13 years ago

Replying to cehoyos:

If there is an issue, is is reproducible with this command:

Yes, I get exactly that.

>ffmpeg -i lena.pnm -pix_fmt yuv420p -f crc - -vf format=bgr24
-pix_fmt yuv420p -f crc -
ffmpeg version N-37541-g670229e Copyright (c) 2000-2012 the FFmpeg developers
  built on Feb  3 2012 20:07:47 with gcc 4.6.2
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-ru
ntime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libope
ncore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --en
able-libmp3lame --enable-libopenjpeg --enable-librtmp --enable-libschroedinger -
-enable-libspeex --enable-libtheora --enable-libvo-aacenc --enable-libvo-amrwben
c --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-
libxvid --enable-zlib
  libavutil      51. 37.100 / 51. 37.100
  libavcodec     54.  0.102 / 54.  0.102
  libavformat    54.  0.100 / 54.  0.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 61.100 /  2. 61.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  6.100 /  0.  6.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, image2, from 'lena.pnm':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: ppm, rgb24, 256x256, 25 tbr, 25 tbn, 25 tbc
[buffer @ 003DA880] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 01B9D2A0] auto-inserting filter 'auto-inserted scale 0' between th
e filter 'src' and the filter 'out'
[scale @ 01B9D680] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:yuv420p flags:0x4
[buffer @ 01B9DB20] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 01C1CCA0] auto-inserting filter 'auto-inserted scale 1' between th
e filter 'Parsed_format_0' and the filter 'out'
[format @ 01C1BE40] auto-inserting filter 'auto-inserted scale 2' between the fi
lter 'src' and the filter 'Parsed_format_0'
[scale @ 01C1AAA0] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:bgr24 flags:0x4
[scale @ 01C1BDE0] w:256 h:256 fmt:bgr24 -> w:256 h:256 fmt:yuv420p flags:0x4
Output #0, crc, to 'pipe:':
  Metadata:
    encoder         : Lavf54.0.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31,
200 kb/s, 90k tbn, 25 tbc
Output #1, crc, to 'pipe:':
  Metadata:
    encoder         : Lavf54.0.100
    Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31,
200 kb/s, 90k tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (ppm -> rawvideo)
  Stream #0:0 -> #1:0 (ppm -> rawvideo)
Press [q] to stop, [?] for help
CRC=0x7db3d981
CRC=0x74eb2924
frame=    1 fps=  0 q=0.0 Lq=0.0 size=       0kB time=00:00:00.04 bitrate=   3.0
kbits/s
video:192kB audio:0kB global headers:0kB muxing overhead -99.992371%

comment:5 by Carl Eugen Hoyos, 12 years ago

Component: undeterminedswscale

comment:6 by Roger Pack, 11 years ago

anybody have any clues on this one?

comment:7 by fireattack, 5 years ago

So I assume pix_fmt routine is part of swscale and the problem is only limited to that?

The same problem doesn't seem to happen if format conversion is done in colorspace.

Test:

ffmpeg -i reference.pnm -vf colorspace=iall=bt601-6-625:all=bt709:format=yuv420p -f crc - -vf format=bgr24,colorspace=iall=bt601-6-625:all=bt709:format=yuv420p -f crc -
Last edited 5 years ago by fireattack (previous) (diff)

comment:8 by fireattack, 5 years ago

Cc: human.peng@gmail.com added

comment:9 by fireattack, 5 years ago

Just some additional info about the impact of this issue. It seems to affect quite a lot downstream software from my observation lately.

I've noticed this "color shifted to yellow" issue in many recording utilities and programs, a noble one being nVidia's ShadowPlay.

Hopefully someone could find the root cause and fix this long standing issue.

comment:10 by gdgsdg123, 5 years ago

So... the problem can be concluded as -pix_fmt interfering with the format video filter?

Or will this be related?.. (one of the swscale hazards)

comment:11 by fireattack, 5 years ago

They're not interfering anything, -pix_fmt, filter format all have the same discoloration bug as soon as you are converting bgr24->yuv (but NOT when rgb24->yuv, so you can work around by (losslessly) converting to rgb24 first).

format argument in colorspace filter (NOT format filter, two different things), however, does NOT have this issue.

Summary (I tested with additional bt601 to bt709 conversion, but the principle is the same):

-vf scale=out_color_matrix=bt709,format=yuv420p BAD
-vf scale=out_color_matrix=bt709 -pix_fmt yuv420p BAD (-pix_fmt is basically same as format filter from my testing. So you can replace one to the other below)

-vf colormatrix=bt601:bt709,format=yuv420p BAD

-vf colorspace=iall=bt601-6-625:all=bt709:format=yuv420p GOOD (this is using format option in colorspace filter)
-vf colorspace=iall=bt601-6-625:all=bt709,format=yuv420p BAD (this is to chain a format filter)
-vf zscale=matrix=709,format=yuv420p GOOD

Last edited 5 years ago by fireattack (previous) (diff)

comment:12 by Balling, 4 years ago

Keywords: bmp added
Summary: Unexpected Color Conversion (bgr->yuv vs rgb->yuv)Unexpected Color Conversion (bgr->yuv vs rgb->yuv); first is bad, bmp affected

This is fixed by -vf scale=flags=accurate_rnd (https://ffmpeg.org/ffmpeg-all.html#Scaler-Options), or more accurately accurate_rnd+whatever algorithm you want. WRONG documentation, BTW, you can use them together.

255, 255, 255 BGR value is converted to (as can be checked by ffplay -i file -vf extractplanes=y) YCbCr values 234, 127, 127 (instead of good 235, 128, 128). So it is just an off-by-one somewhere. And no, it is not out-of-gamut. I suppose the bug is indeed somewhere in second line about special "unscaled" converter, whatever that means (you can print those with +print_info):

[swscaler @ 00000275e7aa00c0] bicubic scaler, from bgr24 to yuv420p using MMXEXT
[swscaler @ 00000275e7aa00c0] using unscaled bgr24 -> yuv420p special converter

as there is no this second line with print_info+accurate_rnd.

What I am concerned with are JPEGs: they are also affected (?), need to check RGB one.

Also see duplicate #8056: bmp is affected too, since it uses BGR. (DPX can be affected too.)

I also clarified what is the bad one: bgr, not rgb.

P.S. -vf scale=out_range=pc is not affected.
P.P.S. If accurate_rnd is bit perfect, we just need to switch as default for only bgr formats. Simple as that.

Last edited 4 years ago by Balling (previous) (diff)

comment:13 by Balling, 4 months ago

Resolution: fixed
Status: openclosed

This is fixed now in 255, 255, 255 color...

But other colors are still broken.

Last edited 4 months ago by Balling (previous) (diff)

comment:14 by Balling, 4 months ago

Resolution: fixed
Status: closedreopened

Actually still an issue outside of color white, lol

comment:15 by MasterQuestionable, 4 months ago

Cc: MasterQuestionable added

͏    "-sws_flags +accurate_rnd" alike should work, unless there was some recent regression?
͏    Or perhaps passing in this manner somehow failed certain cases..?

Last edited 4 months ago by MasterQuestionable (previous) (diff)

comment:16 by MasterQuestionable, 4 months ago

Keywords: bmp removed
Status: reopenedopen
Summary: Unexpected Color Conversion (bgr->yuv vs rgb->yuv); first is bad, bmp affectedAbnormal colorspace conversion of BGR -> YUV comparing the RGB variant

comment:17 by Balling, 4 months ago

-sws_flags +accurate_rnd

sws_flags does not work at all, stop inserting it everywhere, it has no affect. -vf scale=flags=accurate_rnd is not the real fix for this, even if it works fine. The better fix is conversion to rgb24 first if you go from bgr to yuv420p. The reason being accurate_rnd has effect on rgb to yuv420p too.

Also bmp is very much affected, since it uses bgr24.

comment:18 by MasterQuestionable, 4 months ago

͏    "sws_flags" indeed worked, per:
͏    https://github.com/MasterInQuestion/attach/commit/e18e5cd26b9d27bacefa685ba70d0d41bc7d7cbc#diff-75a0130f1814ba26c23e3a6fc6ce5046620163061d392086bc55e2c13c145bf9L264

͏    Probably the "-sws_flags" was problematic..?
͏    https://trac.ffmpeg.org/ticket/10993
͏    .
͏    But to my experience it adequately worked.


͏    I know this is somewhat off-topic, but discussion right on the constrained wiki page...
͏    Likely troublesome.

͏    I believe the issue affects everything uses BGR: more than just BMP.


͏    I also tried to add a comment on:
͏    https://github.com/FFmpeg/FFmpeg/commit/3e064f52eb368a373ded6e3704fcf29f1db3ff12
͏    .
͏    But somehow failed with obscure error?

͏    I'm even unable to add comment on:
͏    https://github.com/FFmpeg/FFmpeg/commit/2f2f73dc3ae3fadbf040eb73f217213b1c99ff39
͏    ; that of my own...

͏    Seems to be sort of auth issue, as I tested in my own repository and succeeded.
͏    But per: https://api.github.com/repos/FFmpeg/FFmpeg/comments?per_page=100&page=4
͏    Commit comments should be permitted..?

͏    ----

https://github.com/FFmpeg/FFmpeg/pull/390
͏    "An owner of this repository has limited the ability to comment to users that are collaborators on this repository."
͏    GitHub issue... The UI for commit comment didn't properly reflect.

Last edited 4 months ago by MasterQuestionable (previous) (diff)

comment:19 by Balling, 4 months ago

Accurate_rnd does not work, that other flag may work.

Commit comments should be permitted..?

People on gothub disabled it after this https://github.com/FFmpeg/FFmpeg/commit/8d0fea81c736280ff19283b05cd0ad935d1e35c5

comment:20 by MasterQuestionable, 4 months ago

͏    "accurate_rnd" passed in alike manner shall work? Per my observation in <colorspace>.
͏    Probably just failed this case..?

͏    Whatsoever would be another bug if you can confirm.


͏    This is the last commit comment indicated from the API:
͏    https://github.com/FFmpeg/FFmpeg/commit/7b8445f03d10faf7ed232e6201bf04ba73d980d7#commitcomment-123092236

͏    Rather interesting thing is:
͏    https://github.com/FFmpeg/FFmpeg/commit/eb0455d64690eed0068e5cb202f72ecdf899837c#commitcomment-122074348
͏    ; the auto-spam...

͏    Both succeed the mentioned event.

comment:21 by Balling, 4 months ago

͏>"accurate_rnd" passed in alike manner shall work? Per my observation in <colorspace>. Probably just failed this case..?

Okay, apparently -sws_flags accurate_rnd only works if you use it after -i option. Good to know. Still it is more simple to use it inside vf scale.

Let's have some reference md5 here. I will convert the strange raw file ffmpeg -f rawvideo -video_size 256x256 -pix_fmt bgr24 -i testin.raw gfajfa.bmp and attach this file. This file has the same issue with

ffplay -i gfajfa.bmp -vf scale=out_color_matrix=bt709,format=yuv420p

Let's have some reference md5 here:

ffmpeg -i gfajfa.bmp -vf scale=out_color_matrix=bt709,format=yuv420p -f framemd5 -

says md5 34e64aa3a7268a6002381af53366ad09

Now convert bmp to png, this now has rgb24:
ffmpeg -i gfajfa.bmp anfca.png

and

ffmpeg -i anfca.png -vf scale=out_color_matrix=bt709,format=yuv420p -f framemd5 -

fef01c4f4721b3ab650088081ac6616d

yep, md5 is not the same, this is the bug. But if you add -sws_flags accurate_rnd after -i both md5 become

52c2faebb1c31fba70487f43887b46de.

Now we need to understand where is the problem is and which is closer to 52c2faebb1c31fba70487f43887b46de: 255, 255, 255 color problem is gone, so it is harder to tell where the issue is. Indeed, in older version from 2020 the md5 of png file to yuv420p frame is the same, but bmp to yuv420p differs in old and in new, so this issue is partially fixed. Let's compare the files between accurate_rnd and without (two files without) in hex editor.

INDEED, the accurate_rnd and ffmpeg -i anfca.png -vf scale=out_color_matrix=bt709,format=yuv420p filee.yuv are mostly the same (in the start of the file), yet ffmpeg -i gfajfa.bmp -vf scale=out_color_matrix=bt709,format=yuv420p file.yuv is not even close.

Last edited 4 months ago by Balling (previous) (diff)

by Balling, 4 months ago

Attachment: gfajfa.bmp added

comment:22 by fireattack, 4 months ago

I don't think even white color is fixed; without accurate_rnd, it still produces yellowish color.

ffmpeg240821.exe -v error -i white.bmp -vf scale=out_color_matrix=bt709,format=yuv420p -color_primaries 1 -color_trc 1 -colorspace 1  -qp 0 "output/white.bmp__ffmpeg240821.exe.mp4" -y
ffmpeg240821.exe -v error -i "output/white.bmp__ffmpeg240821.exe.mp4" "output/img_white.bmp__ffmpeg240821.exe.png" -y

(Note: I convert the image to a proper mp4 video with all the color-related tags first, and then grab the frame from it. Otherwise different versions of FFMPEG often have different implied color matrix/space/etc., and end up with different results.)

I can also confirm there is a hash difference bwtween older version (I use 4.2.1 to test) and current version when converting BGR to YUV (without accurate_rnd) as Balling observed. But perceptually the difference is so small they're visually the same (as in, not fixed at all.)

by fireattack, 4 months ago

Attachment: white.bmp.zip added

comment:23 by Balling, 4 months ago

I don't think even white color is fixed

It is not fixed with your file, indeed, but with a white bmp file of different size it is fixed. Hilarious.

ffplay.exe -i bgr.bmp -vf scale=out_color_matrix=bt709,format=yuv420p

and use a color picker. Same for your file.

by Balling, 4 months ago

Attachment: bgr.bmp added

comment:24 by Balling, 3 months ago

Keywords: scale added

I was thinking about this issue, since dither cannot be removed in scale in this without BITDEPTH change transform (sws_dither none indeed has no effect, see Paul comment https://trac.ffmpeg.org/ticket/9407#comment:10), only in zscale (by default no dither) that is what causes this issue, I imagine. Off-by-one in colors must be dither. And again even the better result is still

"are mostly the same (in the start of the file), " in the end of the file it is different anyway, so that means it is dither.

Last edited 8 weeks ago by Balling (previous) (diff)
Note: See TracTickets for help on using tickets.