GPU decoding of avif?

[s1ddok]

Is it possible to decode avif with something like NVDec?

[y-guyon]

AVIF is based on the video codec AV1. NVDEC supports AV1 Main Profile [1] so yes, in theory.
In practice libavif does not interface with NVDEC directly or indirectly for now. I am not aware of plans towards that goal in libavif.
There may exist other AVIF implementations than libavif that make use of NVDEC. I could not find any with a quick search.

[1] https://docs.nvidia.com/video-technologies/video-codec-sdk/12.2/nvdec-video-decoder-api-prog-guide/index.html#supported-codecs

[s1ddok]

is it somehow possible to get a pointer to AV1 data portion and feed to GPU decoder?

[y-guyon]

Not with libavif. The AV1 data location as identified by the iloc box is not exposed in the public API. Only the AV1 data sizes are available in the avifIOStats struct.

Other tools may help with that. For example ffmpeg -i file.avif -c:v copy av1.obu seems to extract the color AV1 track sample OBUs. That could be fed into the GPU decoder.

[s1ddok]

is it feasible to create a private fork of libavif that will expose it?

[y-guyon]

Yes.

Also if FFmpeg does not work for you, maybe take a look at mp4box.

[wantehchang]

Hi Andrey,

In libavif, the avifCodec struct defined in include/avif/internal.h is the interface libavif uses to talk to an AV1 encoder or decoder. There are several implementations of the avifCodec interface in the src/codec_*.c files. In your private fork of libavif, you can add a src/codec_nvdec.c file to use NVDec as an AV1 decoder. The src/codec_*.c files can serve as examples.

[wantehchang]

Andrey: Specifically, look at the aomCodecGetNextImage and dav1dCodecGetNextImage functions as examples. When the getNextImage function in the avifCodec interface is called, AV1 data is in the buffer sample->data.data of sample->data.size bytes.

[y-guyon]

Note that merging something such as #2381 but for the decoder side would allow for the use case without the need for a fork.

[s1ddok]

thank you! i will start looking at these, do you have your own plans for introduction gpu driven decoding?

[y-guyon]

There is no such plan in libavif that I am aware of as of today.