codec_whitelist does not influence codec picking

[Rudolf Polzer]

Summary of the bug:

The codec_whitelist feature, as introduced in 5c8eb16769e581f828ce420373c558c190185cc1, appears to block codecs from decoding, but it does not prevent automatic codec selection from picking a blocked codec (and thus it does not make ffmpeg behave as if only the listed codecs were compiled in, which I presume was the intent - it is at least what the Chromium developers believe the intent was in ​https://source.chromium.org/chromium/chromium/src/+/02508a7461cde79277a7e5e55f1704d3fb6a5f7a).

Thus, when the feature is used and one wants to play/decode a file whose preferred codec is not on the whitelist, but a different codec is on the whitelist that can decode the file, decoding will fail unless explicitly selecting the decoder.

This currently breaks Opus decoding (and thus 100% of youtube.com) in Chromium on Alpine Linux (and probably any other system that compiles Chromium to link against a system-wide installed ffmpeg library, and configures that system-wide library to contain both the opus and libopus decoders): ​https://gitlab.alpinelinux.org/alpine/aports/-/issues/15585

Also, as the below log shows, even when explicitly setting the codec, there still appear to be issues during analysis of the file; these do not appear to break playback though.

Component set to avformat as I believe find_probe_decoder in libavformat/demux.c should take the whitelist into account and skip codecs not listed on it.

Priority set to important, as Chromium being able to play YouTube videos is pretty important.

Not sure if Type should really be defect or enhancement - it depends on what the intent of the codec whitelist feature was (and thus whether Chrome is abusing it). The documentation of the feature does not go into much detail.

How to reproduce:

(NOTE: this repro requires _any_ opus file, and a ffmpeg binary that has both the opus and libopus decoders enabled in it. For my tests I was using ​http://rm.cloudns.org/~xonotic/temp/b-opus.mp4)

% ffplay -codec_whitelist h264,libopus /tmp/b-opus.mp4
ffplay version N-113171-g85b8d59ec7 Copyright (c) 2003-2024 the FFmpeg developers
  built with gcc 13 (Debian 13.2.0-5)
  configuration: --prefix=/home/rpolzer/src/update-ffmpeg/out --cc=gcc --cxx=g++ --objcc= --dep-cc=gcc --disable-static --enable-shared --enable-nonfree --enable-gpl --enable-version3 --enable-libaom --enable-libvpx --enable-libwebp --enable-libx264 --enable-libfribidi --enable-libass --enable-libfontconfig --enable-libfreetype --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-libpulse --enable-vaapi --enable-vdpau --enable-demuxer=dash --enable-gnutls --enable-libxml2
  libavutil      58. 36.100 / 58. 36.100
  libavcodec     60. 36.100 / 60. 36.100
  libavformat    60. 20.100 / 60. 20.100
  libavdevice    60.  4.100 / 60.  4.100
  libavfilter     9. 16.100 /  9. 16.100
  libswscale      7.  6.100 /  7.  6.100
  libswresample   4. 13.100 /  4. 13.100
  libpostproc    57.  4.100 / 57.  4.100
[opus @ 0x7ff164007a80] Codec (opus) not on whitelist 'h264,libopus'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7ff164000c80] Failed to open codec in avformat_find_stream_info
[opus @ 0x7ff164007a80] Codec (opus) not on whitelist 'h264,libopus'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7ff164000c80] Could not find codec parameters for stream 1 (Audio: opus (Opus / 0x7375704F), 48000 Hz, 0 channels, 104 kb/s): unspecified number of channels
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/b-opus.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.16.100
  Duration: 00:00:25.48, start: 0.000000, bitrate: 884 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt470bg/unknown/unknown, progressive), 640x360, 768 kb/s, 60 fps, 60 tbr, 15360 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc60.31.102 libx264
  Stream #0:1[0x2](und): Audio: opus (Opus / 0x7375704F), 48000 Hz, 2 channels, 104 kb/s (default)
      Metadata:
        handler_name    : SoundHandler
        vendor_id       : [0][0][0][0]
[opus @ 0x7ff164022d80] Codec (opus) not on whitelist 'h264,libopus'
Failed to open file '/tmp/b-opus.mp4' or configure filtergraph

% ffplay -codec_whitelist h264,libopus -vn -codec:a libopus /tmp/b-opus.mp4 
ffplay version N-113171-g85b8d59ec7 Copyright (c) 2003-2024 the FFmpeg developers
  built with gcc 13 (Debian 13.2.0-5)
  configuration: --prefix=/home/rpolzer/src/update-ffmpeg/out --cc=gcc --cxx=g++ --objcc= --dep-cc=gcc --disable-static --enable-shared --enable-nonfree --enable-gpl --enable-version3 --enable-libaom --enable-libvpx --enable-libwebp --enable-libx264 --enable-libfribidi --enable-libass --enable-libfontconfig --enable-libfreetype --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-libpulse --enable-vaapi --enable-vdpau --enable-demuxer=dash --enable-gnutls --enable-libxml2
  libavutil      58. 36.100 / 58. 36.100
  libavcodec     60. 36.100 / 60. 36.100
  libavformat    60. 20.100 / 60. 20.100
  libavdevice    60.  4.100 / 60.  4.100
  libavfilter     9. 16.100 /  9. 16.100
  libswscale      7.  6.100 /  7.  6.100
  libswresample   4. 13.100 /  4. 13.100
  libpostproc    57.  4.100 / 57.  4.100
[opus @ 0x7fd82c011d40] Codec (opus) not on whitelist 'h264,libopus'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd82c000c80] Failed to open codec in avformat_find_stream_info
[opus @ 0x7fd82c011d40] Codec (opus) not on whitelist 'h264,libopus'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd82c000c80] Could not find codec parameters for stream 1 (Audio: opus (Opus / 0x7375704F), 48000 Hz, 0 channels, 104 kb/s): unspecified number of channels
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/b-opus.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.16.100
  Duration: 00:00:25.48, start: 0.000000, bitrate: 884 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt470bg/unknown/unknown, progressive), 640x360, 768 kb/s, 60 fps, 60 tbr, 15360 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc60.31.102 libx264
  Stream #0:1[0x2](und): Audio: opus (Opus / 0x7375704F), 48000 Hz, 2 channels, 104 kb/s (default)
      Metadata:
        handler_name    : SoundHandler
        vendor_id       : [0][0][0][0]
   0.58 M-A:  0.000 fd=   0 aq=   10KB vq=    0KB sq=    0B f=0/0