Remember tune=iq option across frames

[juliobbv-p]

Also add a ProgressiveTest to test two layer encoding with tune=iq.

[y-guyon]

Thank you for this PR.

[y-guyon]

I tried to deduce what would happen in the following two-layer scenarios, assuming libavifDefaultTuneMetric=AOM_TUNE_IQ:

no avifEncoderSetCodecSpecificOption() call (equivalent to avifEncoderSetCodecSpecificOption("tune", NULL))

1. ContainExplicitTuning()=0, useLibavifDefaultTune=1, useTuneIq=1, previousLayerUsedIq=1
2. ContainExplicitTuning()=0, useLibavifDefaultTune=1, useTuneIq=1

✅

avifEncoderSetCodecSpecificOption("tune", "ssim") before first frame, avifEncoderSetCodecSpecificOption("tune", "iq") after first frame

1. ContainExplicitTuning()=1, useLibavifDefaultTune=0, useTuneIq=0, previousLayerUsedIq=0
2. ContainExplicitTuning()=1, useLibavifDefaultTune=0, useTuneIq=1

✅

avifEncoderSetCodecSpecificOption("tune", "iq") before first frame, avifEncoderSetCodecSpecificOption("tune", "ssim") after first frame

1. ContainExplicitTuning()=1, useLibavifDefaultTune=0, useTuneIq=1, previousLayerUsedIq=1
2. ContainExplicitTuning()=1, useLibavifDefaultTune=0, useTuneIq=0

✅

avifEncoderSetCodecSpecificOption("tune", "iq") before first frame

1. ContainExplicitTuning()=1, useLibavifDefaultTune=0, useTuneIq=1, previousLayerUsedIq=1
2. ContainExplicitTuning()=0, useLibavifDefaultTune=1, useTuneIq=1

✅

avifEncoderSetCodecSpecificOption("tune", "ssim") before first frame

1. ContainExplicitTuning()=1, useLibavifDefaultTune=0, useTuneIq=0, previousLayerUsedIq=0
2. ContainExplicitTuning()=0, useLibavifDefaultTune=1, useTuneIq=1
   is this the expected behavior? ------------------------------------------^

[juliobbv-p]

avifEncoderSetCodecSpecificOption("tune", "ssim") before first frame
ContainExplicitTuning()=1, useLibavifDefaultTune=0, useTuneIq=0, previousLayerUsedIq=0
ContainExplicitTuning()=0, useLibavifDefaultTune=1, useTuneIq=1
is this the expected behavior? ------------------------------------------^

This is a good question. In this case, useTuneIq has to match the logic that determines libavifDefaultTuneMetric, and because for the second frame: useLibavifDefaultTune=1 and libavifDefaultTuneMetric=AOM_TUNE_IQ, then useTuneIq=1 is expected behavior here.

This does bring us another question: do we want to persist the tune IQ decision for future frames when the tuning mode is unset for a frame (and have that influence libavif's tune picking decision), not just remember whether the previous frame used tune IQ or not? That'd require a more complex implementation I think, because libavif would need to keep track of whether the caller has ever set the tuning mode for past frames (independently for color and alpha), and modify the tune picking decision to leverage that information. Doable, but I'm concerned the logic would be difficult to read and reason about.

@wantehchang and I also had the idea to mark the tuning mode as not-updatable -- libavif will enforce the same tuning mode from the first frame onto the rest. This makes tune-picking logic much more simple, but does come with a breaking API change through. IMO, I find it odd that the tuning mode can be updatable between frames in the first place 😄.

What do your thoughts on the situation?

[y-guyon]

Remembering if any tune was set in the past should be simple enough.

There is no way to "reset" the tune setting in aom through libavif right now, so if any tune setting was forwarded till aom once, there is no need for libavif to set anything by default ever after.
In other words, libavif should only set a tune by default on the first frame, if there is no explicit user tune. And libavif should remember the last default or explicit tune for any frame, so that it can adapt the quality in case of tune=iq.
That should be enough, unless I missed something.

[juliobbv-p]

Got you, I believe you're right on resetting the tune -- I don't see a way to do it either.

Just to confirm: there's no way in libavif to encode multi-frame layers with alpha, right?

[juliobbv-p]

From an email convo with @wantehchang:

Yes, libavif supports multi-frame encoding with alpha. Alpha is
commonly used in animated images (image sequences).

libavif creates separate AV1 encoders to encode the color and alpha
bitstreams. The AV1 encoders receive the "avifBool alpha" parameter so
that they know whether they are encoding alpha or color.

The AV1 encoders are created by the avifCodecCreate() call in
avifEncoderAddImageItems().

[juliobbv-p]

All right, I've reacted to PR feedback, implementing this behavior:

There is no way to "reset" the tune setting in aom through libavif right now, so if any tune setting was forwarded till aom once, there is no need for libavif to set anything by default ever after.
In other words, libavif should only set a tune by default on the first frame, if there is no explicit user tune. And libavif should remember the last default or explicit tune for any frame, so that it can adapt the quality in case of tune=iq.

I kept the previousFrameUsedTuneIq boolean variable from my first revision, but I can easily convert it to aom_tune_metric if that's more preferable. Both work well here.