Opened 13 years ago
Last modified 8 weeks ago
#979 open defect
Abnormal colorspace conversion of BGR -> YUV comparing the RGB variant
Reported by: | natt | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | swscale |
Version: | git-master | Keywords: | scale |
Cc: | human.peng@gmail.com, MasterQuestionable | Blocked By: | |
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Input file is bgr24. Output file is yuv420p. If I instruct ffmpeg to convert to rgb24 first (instead of bgr24->yuv420p directly), the colors are slightly different.
Test input, exact command lines, and console output attached.
Attachments (5)
Change History (29)
by , 13 years ago
Attachment: | testcase.c added |
---|
by , 13 years ago
Attachment: | testin.raw added |
---|
follow-up: 2 comment:1 by , 13 years ago
Whatever issue you are reporting I can not reproduce it.
rgb24(->bgr24)->yuv is fine here
comment:2 by , 13 years ago
Replying to richardpl:
Whatever issue you are reporting I can not reproduce it.
rgb24(->bgr24)->yuv is fine here
The two conversions should be identical, right? Since rgb24<->bgr24 is lossless, bgr->rgb->yuv should produce the same output as bgr->yuv. If it doesn't, it's not working right.
ffmpeg -f rawvideo -video_size 256x256 -pix_fmt bgr24 -i testin.raw -sws_flags lanczos -f rawvideo -pix_fmt yuv420p md5:
0ba2a7494930baaa64869462d54b85cc
ffmpeg -f rawvideo -video_size 256x256 -pix_fmt bgr24 -i testin.raw -filter format=rgb24 -sws_flags lanczos -f rawvideo -pix_fmt yuv420p md5:
d5bd2d94675b67d23eb2661ca462966d
follow-up: 4 comment:3 by , 13 years ago
Status: | new → open |
---|---|
Version: | unspecified → git-master |
If there is an issue, is is reproducible with this command:
$ ./ffmpeg -i tests/lena.pnm -pix_fmt yuv420p -f crc - -vf format=bgr24 -pix_fmt yuv420p -f crc - ffmpeg version N-37568-g8162c6f Copyright (c) 2000-2012 the FFmpeg developers built on Feb 5 2012 16:31:32 with gcc 4.5.3 configuration: --cc='/usr/local/gcc-4.5.3/bin/gcc -m32' libavutil 51. 37.100 / 51. 37.100 libavcodec 54. 0.102 / 54. 0.102 libavformat 54. 0.100 / 54. 0.100 libavdevice 53. 4.100 / 53. 4.100 libavfilter 2. 61.100 / 2. 61.100 libswscale 2. 1.100 / 2. 1.100 libswresample 0. 6.100 / 0. 6.100 Input #0, image2, from 'tests/lena.pnm': Duration: 00:00:00.04, start: 0.000000, bitrate: N/A Stream #0:0: Video: ppm, rgb24, 256x256, 25 tbr, 25 tbn, 25 tbc [buffer @ 0x8dd20a0] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param: [buffersink @ 0x8dd2360] auto-inserting filter 'auto-inserted scale 0' between the filter 'src' and the filter 'out' [scale @ 0x8dd28e0] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:yuv420p flags:0x4 [buffer @ 0x8de20e0] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param: [buffersink @ 0x8de1780] auto-inserting filter 'auto-inserted scale 1' between the filter 'Parsed_format_0' and the filter 'out' [format @ 0x8de0a80] auto-inserting filter 'auto-inserted scale 2' between the filter 'src' and the filter 'Parsed_format_0' [scale @ 0x8df3960] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:bgr24 flags:0x4 [scale @ 0x8de0b40] w:256 h:256 fmt:bgr24 -> w:256 h:256 fmt:yuv420p flags:0x4 Output #0, crc, to 'pipe:': Metadata: encoder : Lavf54.0.100 Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31, 200 kb/s, 90k tbn, 25 tbc Output #1, crc, to 'pipe:': Metadata: encoder : Lavf54.0.100 Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31, 200 kb/s, 90k tbn, 25 tbc Stream mapping: Stream #0:0 -> #0:0 (ppm -> rawvideo) Stream #0:0 -> #1:0 (ppm -> rawvideo) Press [q] to stop, [?] for help CRC=0x7db3d981 CRC=0x74eb2924 frame= 1 fps= 0 q=0.0 Lq=0.0 size= 0kB time=00:00:00.04 bitrate= 3.0kbits/s video:192kB audio:0kB global headers:0kB muxing overhead -99.992371%
comment:4 by , 13 years ago
Replying to cehoyos:
If there is an issue, is is reproducible with this command:
Yes, I get exactly that.
>ffmpeg -i lena.pnm -pix_fmt yuv420p -f crc - -vf format=bgr24 -pix_fmt yuv420p -f crc - ffmpeg version N-37541-g670229e Copyright (c) 2000-2012 the FFmpeg developers built on Feb 3 2012 20:07:47 with gcc 4.6.2 configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-ru ntime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libope ncore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --en able-libmp3lame --enable-libopenjpeg --enable-librtmp --enable-libschroedinger - -enable-libspeex --enable-libtheora --enable-libvo-aacenc --enable-libvo-amrwben c --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable- libxvid --enable-zlib libavutil 51. 37.100 / 51. 37.100 libavcodec 54. 0.102 / 54. 0.102 libavformat 54. 0.100 / 54. 0.100 libavdevice 53. 4.100 / 53. 4.100 libavfilter 2. 61.100 / 2. 61.100 libswscale 2. 1.100 / 2. 1.100 libswresample 0. 6.100 / 0. 6.100 libpostproc 52. 0.100 / 52. 0.100 Input #0, image2, from 'lena.pnm': Duration: 00:00:00.04, start: 0.000000, bitrate: N/A Stream #0:0: Video: ppm, rgb24, 256x256, 25 tbr, 25 tbn, 25 tbc [buffer @ 003DA880] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param: [buffersink @ 01B9D2A0] auto-inserting filter 'auto-inserted scale 0' between th e filter 'src' and the filter 'out' [scale @ 01B9D680] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:yuv420p flags:0x4 [buffer @ 01B9DB20] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param: [buffersink @ 01C1CCA0] auto-inserting filter 'auto-inserted scale 1' between th e filter 'Parsed_format_0' and the filter 'out' [format @ 01C1BE40] auto-inserting filter 'auto-inserted scale 2' between the fi lter 'src' and the filter 'Parsed_format_0' [scale @ 01C1AAA0] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:bgr24 flags:0x4 [scale @ 01C1BDE0] w:256 h:256 fmt:bgr24 -> w:256 h:256 fmt:yuv420p flags:0x4 Output #0, crc, to 'pipe:': Metadata: encoder : Lavf54.0.100 Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31, 200 kb/s, 90k tbn, 25 tbc Output #1, crc, to 'pipe:': Metadata: encoder : Lavf54.0.100 Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31, 200 kb/s, 90k tbn, 25 tbc Stream mapping: Stream #0:0 -> #0:0 (ppm -> rawvideo) Stream #0:0 -> #1:0 (ppm -> rawvideo) Press [q] to stop, [?] for help CRC=0x7db3d981 CRC=0x74eb2924 frame= 1 fps= 0 q=0.0 Lq=0.0 size= 0kB time=00:00:00.04 bitrate= 3.0 kbits/s video:192kB audio:0kB global headers:0kB muxing overhead -99.992371%
comment:5 by , 12 years ago
Component: | undetermined → swscale |
---|
comment:7 by , 5 years ago
So I assume pix_fmt routine is part of swscale and the problem is only limited to that?
The same problem doesn't seem to happen if format conversion is done in colorspace
.
Test:
ffmpeg -i reference.pnm -vf colorspace=iall=bt601-6-625:all=bt709:format=yuv420p -f crc - -vf format=bgr24,colorspace=iall=bt601-6-625:all=bt709:format=yuv420p -f crc -
comment:8 by , 5 years ago
Cc: | added |
---|
comment:9 by , 5 years ago
Just some additional info about the impact of this issue. It seems to affect quite a lot downstream software from my observation lately.
I've noticed this "color shifted to yellow" issue in many recording utilities and programs, a noble one being nVidia's ShadowPlay.
Hopefully someone could find the root cause and fix this long standing issue.
comment:10 by , 5 years ago
So... the problem can be concluded as -pix_fmt
interfering with the format
video filter?
Or will this be related?.. (one of the swscale hazards)
comment:11 by , 5 years ago
They're not interfering anything, -pix_fmt, filter format
all have the same discoloration bug as soon as you are converting bgr24->yuv (but NOT when rgb24->yuv, so you can work around by (losslessly) converting to rgb24 first).
format
argument in colorspace
filter (NOT format
filter, two different things), however, does NOT have this issue.
Summary (I tested with additional bt601 to bt709 conversion, but the principle is the same):
-vf scale=out_color_matrix=bt709,format=yuv420p
BAD
-vf scale=out_color_matrix=bt709 -pix_fmt yuv420p
BAD (-pix_fmt
is basically same as format
filter from my testing. So you can replace one to the other below)
-vf colormatrix=bt601:bt709,format=yuv420p
BAD
-vf colorspace=iall=bt601-6-625:all=bt709:format=yuv420p
GOOD (this is using format option in colorspace
filter)
-vf colorspace=iall=bt601-6-625:all=bt709,format=yuv420p
BAD (this is to chain a format
filter)
-vf zscale=matrix=709,format=yuv420p
GOOD
comment:12 by , 4 years ago
Keywords: | bmp added |
---|---|
Summary: | Unexpected Color Conversion (bgr->yuv vs rgb->yuv) → Unexpected Color Conversion (bgr->yuv vs rgb->yuv); first is bad, bmp affected |
This is fixed by -vf scale=flags=accurate_rnd (https://ffmpeg.org/ffmpeg-all.html#Scaler-Options), or more accurately accurate_rnd+whatever algorithm you want. WRONG documentation, BTW, you can use them together.
255, 255, 255 BGR value is converted to (as can be checked by ffplay -i file -vf extractplanes=y) YCbCr values 234, 127, 127 (instead of good 235, 128, 128). So it is just an off-by-one somewhere. And no, it is not out-of-gamut. I suppose the bug is indeed somewhere in second line about special "unscaled" converter, whatever that means (you can print those with +print_info):
[swscaler @ 00000275e7aa00c0] bicubic scaler, from bgr24 to yuv420p using MMXEXT [swscaler @ 00000275e7aa00c0] using unscaled bgr24 -> yuv420p special converter
as there is no this second line with print_info+accurate_rnd.
What I am concerned with are JPEGs: they are also affected (?), need to check RGB one.
Also see duplicate #8056: bmp is affected too, since it uses BGR. (DPX can be affected too.)
I also clarified what is the bad one: bgr, not rgb.
P.S. -vf scale=out_range=pc is not affected.
P.P.S. If accurate_rnd is bit perfect, we just need to switch as default for only bgr formats. Simple as that.
comment:13 by , 4 months ago
Resolution: | → fixed |
---|---|
Status: | open → closed |
This is fixed now in 255, 255, 255 color...
But other colors are still broken.
comment:14 by , 4 months ago
Resolution: | fixed |
---|---|
Status: | closed → reopened |
Actually still an issue outside of color white, lol
comment:15 by , 4 months ago
Cc: | added |
---|
͏ "-sws_flags +accurate_rnd" alike should work, unless there was some recent regression?
͏ Or perhaps passing in this manner somehow failed certain cases..?
comment:16 by , 4 months ago
Keywords: | bmp removed |
---|---|
Status: | reopened → open |
Summary: | Unexpected Color Conversion (bgr->yuv vs rgb->yuv); first is bad, bmp affected → Abnormal colorspace conversion of BGR -> YUV comparing the RGB variant |
comment:17 by , 4 months ago
-sws_flags +accurate_rnd
sws_flags does not work at all, stop inserting it everywhere, it has no affect. -vf scale=flags=accurate_rnd is not the real fix for this, even if it works fine. The better fix is conversion to rgb24 first if you go from bgr to yuv420p. The reason being accurate_rnd has effect on rgb to yuv420p too.
Also bmp is very much affected, since it uses bgr24.
comment:18 by , 4 months ago
͏ "sws_flags" indeed worked, per:
͏ https://github.com/MasterInQuestion/attach/commit/e18e5cd26b9d27bacefa685ba70d0d41bc7d7cbc#diff-75a0130f1814ba26c23e3a6fc6ce5046620163061d392086bc55e2c13c145bf9L264
͏ Probably the "-sws_flags" was problematic..?
͏ https://trac.ffmpeg.org/ticket/10993
͏ .
͏ But to my experience it adequately worked.
͏ I know this is somewhat off-topic, but discussion right on the constrained wiki page...
͏ Likely troublesome.
͏ I believe the issue affects everything uses BGR: more than just BMP.
͏ I also tried to add a comment on:
͏ https://github.com/FFmpeg/FFmpeg/commit/3e064f52eb368a373ded6e3704fcf29f1db3ff12
͏ .
͏ But somehow failed with obscure error?
͏ I'm even unable to add comment on:
͏ https://github.com/FFmpeg/FFmpeg/commit/2f2f73dc3ae3fadbf040eb73f217213b1c99ff39
͏ ; that of my own...
͏ Seems to be sort of auth issue, as I tested in my own repository and succeeded.
͏ But per: https://api.github.com/repos/FFmpeg/FFmpeg/comments?per_page=100&page=4
͏ Commit comments should be permitted..?
͏ ----
https://github.com/FFmpeg/FFmpeg/pull/390
͏ "An owner of this repository has limited the ability to comment to users that are collaborators on this repository."
͏ GitHub issue... The UI for commit comment didn't properly reflect.
comment:19 by , 4 months ago
Accurate_rnd does not work, that other flag may work.
Commit comments should be permitted..?
People on gothub disabled it after this https://github.com/FFmpeg/FFmpeg/commit/8d0fea81c736280ff19283b05cd0ad935d1e35c5
comment:20 by , 4 months ago
͏ "accurate_rnd" passed in alike manner shall work? Per my observation in <colorspace>.
͏ Probably just failed this case..?
͏ Whatsoever would be another bug if you can confirm.
͏ This is the last commit comment indicated from the API:
͏ https://github.com/FFmpeg/FFmpeg/commit/7b8445f03d10faf7ed232e6201bf04ba73d980d7#commitcomment-123092236
͏ Rather interesting thing is:
͏ https://github.com/FFmpeg/FFmpeg/commit/eb0455d64690eed0068e5cb202f72ecdf899837c#commitcomment-122074348
͏ ; the auto-spam...
͏ Both succeed the mentioned event.
comment:21 by , 4 months ago
͏>"accurate_rnd" passed in alike manner shall work? Per my observation in <colorspace>. Probably just failed this case..?
Okay, apparently -sws_flags accurate_rnd only works if you use it after -i option. Good to know. Still it is more simple to use it inside vf scale.
Let's have some reference md5 here. I will convert the strange raw file ffmpeg -f rawvideo -video_size 256x256 -pix_fmt bgr24 -i testin.raw gfajfa.bmp and attach this file. This file has the same issue with
ffplay -i gfajfa.bmp -vf scale=out_color_matrix=bt709,format=yuv420p
Let's have some reference md5 here:
ffmpeg -i gfajfa.bmp -vf scale=out_color_matrix=bt709,format=yuv420p -f framemd5 -
says md5 34e64aa3a7268a6002381af53366ad09
Now convert bmp to png, this now has rgb24:
ffmpeg -i gfajfa.bmp anfca.png
and
ffmpeg -i anfca.png -vf scale=out_color_matrix=bt709,format=yuv420p -f framemd5 -
fef01c4f4721b3ab650088081ac6616d
yep, md5 is not the same, this is the bug. But if you add -sws_flags accurate_rnd after -i both md5 become
52c2faebb1c31fba70487f43887b46de.
Now we need to understand where is the problem is and which is closer to 52c2faebb1c31fba70487f43887b46de: 255, 255, 255 color problem is gone, so it is harder to tell where the issue is. Indeed, in older version from 2020 the md5 of png file to yuv420p frame is the same, but bmp to yuv420p differs in old and in new, so this issue is partially fixed. Let's compare the files between accurate_rnd and without (two files without) in hex editor.
INDEED, the accurate_rnd and ffmpeg -i anfca.png -vf scale=out_color_matrix=bt709,format=yuv420p filee.yuv are mostly the same (in the start of the file), yet ffmpeg -i gfajfa.bmp -vf scale=out_color_matrix=bt709,format=yuv420p file.yuv is not even close.
by , 4 months ago
Attachment: | gfajfa.bmp added |
---|
comment:22 by , 4 months ago
I don't think even white color is fixed; without accurate_rnd, it still produces yellowish color.
ffmpeg240821.exe -v error -i white.bmp -vf scale=out_color_matrix=bt709,format=yuv420p -color_primaries 1 -color_trc 1 -colorspace 1 -qp 0 "output/white.bmp__ffmpeg240821.exe.mp4" -y ffmpeg240821.exe -v error -i "output/white.bmp__ffmpeg240821.exe.mp4" "output/img_white.bmp__ffmpeg240821.exe.png" -y
(Note: I convert the image to a proper mp4 video with all the color-related tags first, and then grab the frame from it. Otherwise different versions of FFMPEG often have different implied color matrix/space/etc., and end up with different results.)
I can also confirm there is a hash difference bwtween older version (I use 4.2.1 to test) and current version when converting BGR to YUV (without accurate_rnd) as Balling observed. But perceptually the difference is so small they're visually the same (as in, not fixed at all.)
by , 4 months ago
Attachment: | white.bmp.zip added |
---|
comment:23 by , 4 months ago
I don't think even white color is fixed
It is not fixed with your file, indeed, but with a white bmp file of different size it is fixed. Hilarious.
ffplay.exe -i bgr.bmp -vf scale=out_color_matrix=bt709,format=yuv420p
and use a color picker. Same for your file.
by , 4 months ago
comment:24 by , 3 months ago
Keywords: | scale added |
---|
I was thinking about this issue, since dither cannot be removed in scale in this without BITDEPTH change transform (sws_dither none indeed has no effect, see Paul comment https://trac.ffmpeg.org/ticket/9407#comment:10), only in zscale (by default no dither) that is what causes this issue, I imagine. Off-by-one in colors must be dither. And again even the better result is still
"are mostly the same (in the start of the file), " in the end of the file it is different anyway, so that means it is dither.
output of testcase executable