Opened 12 months ago

Last modified 12 months ago

#10418 new defect

DirectShow, wrong pixel format

Reported by: Michael Koch Owned by:
Priority: normal Component: avdevice
Version: git-master Keywords: dshow
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description (last modified by Michael Koch)

There is a problem with a ASI178MM (monochrome) camera and its DirectShow driver. When I check the options with this command

ffmpeg -list_options 1 -f dshow -i video="ASI178MM Camera (ZWO Design)"

I get this output:

[dshow @ 00000203f5b315c0] pixel_format=rgb8 min s=3096x2080 fps=1.0842e-12 max s=3096x2080 fps=30
[dshow @ 00000203f5b315c0] pixel_format=rgb8 min s=1280x960 fps=1.0842e-12 max s=1280x960 fps=30
[dshow @ 00000203f5b315c0] pixel_format=rgb8 min s=1280x800 fps=1.0842e-12 max s=1280x800 fps=30
... (there are more sizes listed, all with the same pixel format rgb8)

When FFmpeg gets a frame from the camera, the detected pixel format is rgb8 (as defined in FFmpeg: 2 bits red, 3 bits green, 3 bits blue). Of course that's wrong, because it's a monochrome camera and the data is 8-bit gray.

My first idea was that it must be a bug in the DirectShow driver. The data is obviously correct 8-bit gray (I did verify that), but it's incorrectly labeled as "rgb8" instead of "gray".

When you look at the specifications of DirectShow

https://learn.microsoft.com/en-us/windows/win32/directshow/uncompressed-rgb-video-subtypes

you see that there exists no "gray" pixel format, and "RGB8" is a different pixel format described as "RGB, 8 bpp, palettized". It seems an uncompressed 8-bit gray frame must be encoded with "RGB8" pixel format, together with a palette which contains 256 levels of gray.

"RGB8" in DirectShow should become "pal8" or "gray" in FFmpeg, and not "rgb8".

By the way, in line 64 of dshow.c there is a comment that this pixel format is untested. So this might be the first test.

Attachments (1)

screenshot_graphstudionext.png (382.2 KB ) - added by Michael Koch 12 months ago.
Screenshot of GraphStudioNext with the properties of the video renderer.

Download all attachments as: .zip

Change History (20)

comment:1 by Michael Koch, 12 months ago

Description: modified (diff)

comment:2 by Balling, 12 months ago

gray

That is YCbCr, except it has Y plane only.

comment:3 by Michael Koch, 12 months ago

I suspect the problem is in line 70 of dshow.c, but I can't make a test because I can't compile FFmpeg under Windows.

"RGB8" is in the Directshow documentation defined as "RGB, 8 bpp, palettized".
"AV_PIX_FMT_RGB8" in FFmpeg is defined in line 86 of pixfmt.h as "packed RGB 3:3:2, 8bpp, (msb)2R 3G 3B(lsb)".
These are two different pixel formats.

Shouldn't line 70 in dshow.c be changed to AV_PIX_FMT_PAL8, and the palette must be loaded from the DirectShow driver?

The same problem is also in line 68 for RGB4, and in line 66 for RGB1 (here it should work, but the colors from the palette are lost and replaced by black and white).

Last edited 12 months ago by Michael Koch (previous) (diff)

comment:4 by Balling, 12 months ago

Keywords: dshow added

Is not this a dup of #6279?

comment:5 by Michael Koch, 12 months ago

I don't think it's a duplicate because #6279 is about RGB24.

comment:6 by Balling, 12 months ago

No, it is a dup, because avpriv_find_pix_fmt() does not find RGB formats.

comment:7 by Balling, 12 months ago

dshow.c, but I can't make a test because I can't compile FFmpeg under Windows.

Just use github actions pointing to your FFmpeg fork to compile. For that clone https://github.com/BtbN/FFmpeg-Builds enable actions and press build and publish a release. Do not forget to change the repo of FFmpeg.

comment:8 by Balling, 12 months ago

It seems an uncompressed 8-bit gray frame must be encoded with "RGB8" pixel format, together with a palette which contains 256 levels of gray.

Where is the code that will decode palette?? How does it look?

in reply to:  8 comment:9 by Michael Koch, 12 months ago

Replying to Balling:

It seems an uncompressed 8-bit gray frame must be encoded with "RGB8" pixel format, together with a palette which contains 256 levels of gray.

Where is the code that will decode palette?? How does it look?

"For palettized formats, the color of each pixel is specified as an index into a palette. The palette must be included in the format block, following the BITMAPINFOHEADER structure."

Source: https://learn.microsoft.com/en-us/windows/win32/directshow/uncompressed-rgb-video-subtypes

and some more details when you click on BITMAPINFOHEADER.

by Michael Koch, 12 months ago

Screenshot of GraphStudioNext with the properties of the video renderer.

comment:10 by Michael Koch, 12 months ago

With GraphStudioNext the output is correct 8-bit grayscale. A screenshot with the properties of the video renderer is attached. I think "Extradata" is the palette.

comment:11 by Balling, 12 months ago

So you need to parse

biClrUsed

Specifies the number of color indices in the color table that are actually used by the bitmap. See Remarks for more information.

biClrImportant

Specifies the number of color indices that are considered important for displaying the bitmap. If this value is zero, all colors are important.

in reply to:  7 ; comment:12 by Michael Koch, 12 months ago

Replying to Balling:

dshow.c, but I can't make a test because I can't compile FFmpeg under Windows.

Just use github actions pointing to your FFmpeg fork to compile. For that clone https://github.com/BtbN/FFmpeg-Builds enable actions and press build and publish a release. Do not forget to change the repo of FFmpeg.

Is this somewhere explained in more detail step by step? I don't understand anything of this github stuff.

in reply to:  12 comment:13 by Balling, 12 months ago

Replying to Michael Koch:

Replying to Balling:

dshow.c, but I can't make a test because I can't compile FFmpeg under Windows.

Just use github actions pointing to your FFmpeg fork to compile. For that clone https://github.com/BtbN/FFmpeg-Builds enable actions and press build and publish a release. Do not forget to change the repo of FFmpeg.

Is this somewhere explained in more detail step by step? I don't understand anything of this github stuff.

No, that is what I use.

comment:14 by Michael Koch, 12 months ago

A simple bugfix could be in line 70 of dshow.c to replace AV_PIX_FMT_RGB8 by AV_PIX_FMT_GRAY8. I know that it's a hack and the correct way is to use AV_PIX_FMT_PAL8 and read the palette.
But for this camera the hack should work because the palette does anyway contain only 256 shades of gray. Because nobody noticed this issue earlier, that probably means that there exist no color cameras that use the RGB8 pixel format with a non-grayscale palette.

comment:15 by Balling, 12 months ago

hack should work

So you did not test it?

With GraphStudioNext the output is correct 8-bit grayscale

Are you sure it does not read palette?

Also can you paste all 1040 bytes of the palette here?

Last edited 12 months ago by Balling (previous) (diff)

in reply to:  15 comment:16 by Michael Koch, 12 months ago

Replying to Balling:

hack should work

So you did not test it?

As said before, I can't compile FFmpeg under Windows.


With GraphStudioNext the output is correct 8-bit grayscale

Are you sure it does not read palette?

I'm quite sure that GraphStudioNext does read the palette.

Also can you paste all 1040 bytes of the palette here?

00 00 00 00 01 01 01 00 02 02 02 00 03 03 03 00 04 04 04 00 05 05 05 00 06 06 06 00 07 07 07 00 08 08 08 00 09 09 09 00 0A 0A 0A 00 0B 0B 0B 00 0C 0C 0C 00 0D 0D 0D 00 0E 0E 0E 00 0F 0F 0F 00 10 10 10 00 11 11 11 00 12 12 12 00 13 13 13 00 14 14 14 00 15 15 15 00 16 16 16 00 17 17 17 00 18 18 18 00 19 19 19 00 1A 1A 1A 00 1B 1B 1B 00 1C 1C 1C 00 1D 1D 1D 00 1E 1E 1E 00 1F 1F 1F 00 20 20 20 00 21 21 21 00 22 22 22 00 23 23 23 00 24 24 24 00 25 25 25 00 26 26 26 00 27 27 27 00 28 28 28 00 29 29 29 00 2A 2A 2A 00 2B 2B 2B 00 2C 2C 2C 00 2D 2D 2D 00 2E 2E 2E 00 2F 2F 2F 00 30 30 30 00 31 31 31 00 32 32 32 00 33 33 33 00 34 34 34 00 35 35 35 00 36 36 36 00 37 37 37 00 38 38 38 00 39 39 39 00 3A 3A 3A 00 3B 3B 3B 00 3C 3C 3C 00 3D 3D 3D 00 3E 3E 3E 00 3F 3F 3F 00 40 40 40 00 41 41 41 00 42 42 42 00 43 43 43 00 44 44 44 00 45 45 45 00 46 46 46 00 47 47 47 00 48 48 48 00 49 49 49 00 4A 4A 4A 00 4B 4B 4B 00 4C 4C 4C 00 4D 4D 4D 00 4E 4E 4E 00 4F 4F 4F 00 50 50 50 00 51 51 51 00 52 52 52 00 53 53 53 00 54 54 54 00 55 55 55 00 56 56 56 00 57 57 57 00 58 58 58 00 59 59 59 00 5A 5A 5A 00 5B 5B 5B 00 5C 5C 5C 00 5D 5D 5D 00 5E 5E 5E 00 5F 5F 5F 00 60 60 60 00 61 61 61 00 62 62 62 00 63 63 63 00 64 64 64 00 65 65 65 00 66 66 66 00 67 67 67 00 68 68 68 00 69 69 69 00 6A 6A 6A 00 6B 6B 6B 00 6C 6C 6C 00 6D 6D 6D 00 6E 6E 6E 00 6F 6F 6F 00 70 70 70 00 71 71 71 00 72 72 72 00 73 73 73 00 74 74 74 00 75 75 75 00 76 76 76 00 77 77 77 00 78 78 78 00 79 79 79 00 7A 7A 7A 00 7B 7B 7B 00 7C 7C 7C 00 7D 7D 7D 00 7E 7E 7E 00 7F 7F 7F 00 80 80 80 00 81 81 81 00 82 82 82 00 83 83 83 00 84 84 84 00 85 85 85 00 86 86 86 00 87 87 87 00 88 88 88 00 89 89 89 00 8A 8A 8A 00 8B 8B 8B 00 8C 8C 8C 00 8D 8D 8D 00 8E 8E 8E 00 8F 8F 8F 00 90 90 90 00 91 91 91 00 92 92 92 00 93 93 93 00 94 94 94 00 95 95 95 00 96 96 96 00 97 97 97 00 98 98 98 00 99 99 99 00 9A 9A 9A 00 9B 9B 9B 00 9C 9C 9C 00 9D 9D 9D 00 9E 9E 9E 00 9F 9F 9F 00 A0 A0 A0 00 A1 A1 A1 00 A2 A2 A2 00 A3 A3 A3 00 A4 A4 A4 00 A5 A5 A5 00 A6 A6 A6 00 A7 A7 A7 00 A8 A8 A8 00 A9 A9 A9 00 AA AA AA 00 AB AB AB 00 AC AC AC 00 AD AD AD 00 AE AE AE 00 AF AF AF 00 B0 B0 B0 00 B1 B1 B1 00 B2 B2 B2 00 B3 B3 B3 00 B4 B4 B4 00 B5 B5 B5 00 B6 B6 B6 00 B7 B7 B7 00 B8 B8 B8 00 B9 B9 B9 00 BA BA BA 00 BB BB BB 00 BC BC BC 00 BD BD BD 00 BE BE BE 00 BF BF BF 00 C0 C0 C0 00 C1 C1 C1 00 C2 C2 C2 00 C3 C3 C3 00 C4 C4 C4 00 C5 C5 C5 00 C6 C6 C6 00 C7 C7 C7 00 C8 C8 C8 00 C9 C9 C9 00 CA CA CA 00 CB CB CB 00 CC CC CC 00 CD CD CD 00 CE CE CE 00 CF CF CF 00 D0 D0 D0 00 D1 D1 D1 00 D2 D2 D2 00 D3 D3 D3 00 D4 D4 D4 00 D5 D5 D5 00 D6 D6 D6 00 D7 D7 D7 00 D8 D8 D8 00 D9 D9 D9 00 DA DA DA 00 DB DB DB 00 DC DC DC 00 DD DD DD 00 DE DE DE 00 DF DF DF 00 E0 E0 E0 00 E1 E1 E1 00 E2 E2 E2 00 E3 E3 E3 00 E4 E4 E4 00 E5 E5 E5 00 E6 E6 E6 00 E7 E7 E7 00 E8 E8 E8 00 E9 E9 E9 00 EA EA EA 00 EB EB EB 00 EC EC EC 00 ED ED ED 00 EE EE EE 00 EF EF EF 00 F0 F0 F0 00 F1 F1 F1 00 F2 F2 F2 00 F3 F3 F3 00 F4 F4 F4 00 F5 F5 F5 00 F6 F6 F6 00 F7 F7 F7 00 F8 F8 F8 00 F9 F9 F9 00 FA FA FA 00 FB FB FB 00 FC FC FC 00 FD FD FD 00 FE FE FE 00 FF FF FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

comment:17 by Balling, 12 months ago

Okay, so palette AF AF AF 00 is R, G, B, A. Hm...

in reply to:  17 ; comment:18 by Michael Koch, 12 months ago

Replying to Balling:

Okay, so palette AF AF AF 00 is R, G, B, A. Hm...

"A" makes no sense for a camera. I think the 4th byte is always 00.

in reply to:  18 comment:19 by Balling, 12 months ago

Replying to Michael Koch:

Replying to Balling:

Okay, so palette AF AF AF 00 is R, G, B, A. Hm...

"A" makes no sense for a camera. I think the 4th byte is always 00.

It is RGBQUAD, i.e. it has 4 bytes, where 4th is "Alpha or Don't Care".

Note: See TracTickets for help on using tickets.