Opened 2 years ago
Closed 2 years ago
#10221 closed defect (fixed)
Null pointer when nvenc gets aborted
Reported by: | Daniel Stankewitz | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | avcodec |
Version: | git-master | Keywords: | |
Cc: | Daniel Stankewitz | Blocked By: | |
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description (last modified by )
Summary of the bug:
libavcodec crashes if nvenc gets aborted with i.e. "No capable devices found" or any other error.
How to reproduce:
With any NVIDIA GPU that is not AV1 capable (because that'll create the error "No capable devices found") with this example.
% ffmpeg.exe -v trace -f lavfi -i smptebars=size=1920x1080 -t 5 -c:v av1_nvenc -b:v 500k -c:a none -pix_fmt yuv420p test.mp4
or
const AVCodec* codec = avcodec_find_encoder_by_name("av1_nvenc"); AVCodecContext* codecContext = avcodec_alloc_context3(codec); codecContext->width = 320; codecContext->height = 240; codecContext->time_base = { 1, 25 }; codecContext->pix_fmt = AV_PIX_FMT_YUV420P; int res = avcodec_open2(codecContext, codec, NULL); avcodec_free_context(&codecContext); ffmpeg version: a02e45a1f3cc6f07c7437c1e225dea2683f843cf
In ff_nvenc_encode_init the initialization order is:
if ((ret = nvenc_load_libraries(avctx)) < 0) return ret; if ((ret = nvenc_setup_device(avctx)) < 0) return ret; if ((ret = nvenc_setup_encoder(avctx)) < 0) return ret; if ((ret = nvenc_setup_surfaces(avctx)) < 0) return ret;
... but in ff_nvenc_encode_close() it tries to work with fifo buffers that are still null because they'll get initialized in nvenc_setup_surfaces() later (after nvenc_setup_device()).
Change History (3)
comment:1 by , 2 years ago
Description: | modified (diff) |
---|
comment:2 by , 2 years ago
Status: | new → open |
---|
comment:3 by , 2 years ago
Resolution: | → fixed |
---|---|
Status: | open → closed |
Fixed in 21101d9eb1f26c1f1a58454919a9ef5993ae9900.
Note:
See TracTickets
for help on using tickets.
Should be fixed by http://ffmpeg.org/pipermail/ffmpeg-devel/2023-February/307035.html