-
Notifications
You must be signed in to change notification settings - Fork 281
Disable Sample Transform decoding by default #3018
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+91
−28
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have two comments on this.
avifDecoderstruct. Since sample transforms also produce color and alpha, sample transforms are not a new content type.In summary, please consider exposing this control as a new
avifBool enableExtendedBitDepthsfield in theavifDecoderstructThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the review.
Also
avifImageContentTypeFlagsis a bit mask rather than an enum, meaning the differentavifImageContentTypeFlagvalues are more orthogonal concepts that can each be toggled on or off than distinct types.Each
avifImageContentTypeFlagmaps to a separate image item in AVIFv1 (primary color item, aux alpha, derived gain map, derived sample transform).Maryla's comment "Tying sample transforms decoding to bit depth" was specifically about avifdec's
--depthflag. I am not sure this is what you meant.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am commenting on the change to the public header include/avif/avif.h.
If a 'sato' derived image item has one 8-bit AV1 image item and adds the constant 0 to the input image item, the 'sato' derived image item should have a 'pixi' box that indicates bit depth 8. Current clients of libavif can handle this output image. This is why we only need to disable by default the 'sato' derived image items whose 'pixi' box indicates a bit depth other than 8, 10, 12.
Since the outut image is still represented with the
avifImagestruct, it seems that the only field inavifImagethat may catch existing clients by surprise is thedepthfield. Are there other kinds of 'sato' derived image items that produce an outputavifImagethat current clients of libavif cannot handle?Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. The question is: should it still be applied?
With this specific example (noop 0 addition), it is obviously better NOT to enable Sample Transforms, as it would just make decoding slower for the same result.
I do not think so. But it does not mean it is beneficial to do so.
libavif only supports Sample Transform derived image items that are NOT the primary item, and in the same
altrgroup as the primary item. So there is always an alternative, and the primary item should be the default output of libavif in my opinion.Advanced users can enable Sample Transform decoding to extract more information when needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought the only concern was breaking the clients unprepared to handle new bit depths. I did not realize the cost incurred by applying sample transforms was also a concern.
We need to provide guidance on whether libavif clients should enable sample transforms. Besides the two concerns mentioned above, are there other concerns?
I think we should constrain the new bit depths that libavif supports and document what they are. My code analysis shows that libavif malfunctions if
decoder->data->meta->sampleTransformDepth > 16, so we need to at least disallow bit depths > 16.Note: You can ignore the rest of this comment.
Re:
AVIF_IMAGE_CONTENT_SAMPLE_TRANSFORMSvs.enableSampleTransforms: In your interpretation ofavifImageContentTypeFlagsbased on the image item type,AVIF_IMAGE_CONTENT_SAMPLE_TRANSFORMSmakes sense. I have a different interpretation ofavifImageContentTypeFlagsbased on the output destination:AVIF_IMAGE_CONTENT_COLOR_AND_ALPHA: the output goes todecoder->imageAVIF_IMAGE_CONTENT_GAIN_MAP: the output goes todecoder->image->gainMap->imageWhen sample transforms are enabled, the output still goes to
decoder->image. Therefore sample transforms, unlike gain maps, do not behave like a new image content type.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One more thing:
avifDecoderSampleTransformItemValidateProperties()performs the following check:This check is redundant because
avifParsePixelInformationProperty()already performs the same check.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wrote:
Your PR #3038 disallows bit depths > 16 in
avifParsePixelInformationProperty(). That is stricter and also works.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done in #3040
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not really, because it goes through:
libavif/src/read.c
Lines 6846 to 6855 in 0812f44
dstImage->depthis likely 8 or 12 because it contains the primary coded image item.decoder->data->meta->sampleTransformDepthis 16 (or higher). So the condition is true, andavifImageCreate()is called withsampleTransformDepthbits. ForsampleTransformDepth>16,avifImageCreate()returns null andavifDecoderApplySampleTransform()returns the wrongAVIF_RESULT_OUT_OF_MEMORYerror code.If I understand correctly, the only way to bypass this would be for the user to explicitly set
avifDecoder::image::depthto exactlysampleTransformDepthafter parsing/decoding the primary item but beforeavifDecoderApplySampleTransform()is called.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I consider this thread resolved.