Fix non-monotonic output timestamps from c2.mtk.hevc.decoder - #3352
Draft
2bitoperations wants to merge 2 commits into
Draft
Fix non-monotonic output timestamps from c2.mtk.hevc.decoder#33522bitoperations wants to merge 2 commits into
2bitoperations wants to merge 2 commits into
Conversation
On Google TV Streamer 4K (MT8696, c2.mtk.hevc.decoder), specific HEVC Main10 content causes the codec to return output buffers with non-monotonic presentationTimeUs, while the buffer release order itself remains correct. MediaCodecRenderer processes output buffers in raw release order and uses the codec-reported timestamp directly for the render/drop decision, causing MediaCodecVideoRenderer to treat these buffers as arriving too late and drop them. Measured on affected content: ~30% of decoded video frames dropped, sustained throughout playback. This re-derives each output buffer's presentationTimeUs from its release order and the stream's known frame duration, instead of trusting the codec-reported value. Buffer release order is untouched. Verified on-device on the affected hardware/content: dropped frames go to 0, playback confirmed visually smooth by direct observation.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
The per-frame duration was rounded to the nearest microsecond once, then multiplied by a growing frame index. Any fractional-microsecond remainder in the true frame duration (e.g. 41708.333...us for 24000/1001fps content) was silently dropped every frame and never recovered, so the error accumulated linearly with runtime: ~0.7ms over a 90-second clip (undetectable, and the only length tested before this fix), but ~58ms by the end of a 2-hour movie — enough to produce audible/visible A/V desync that gets worse the longer playback continues. Fix: keep the per-frame duration unrounded and compute each buffer's offset independently as round(frameIndex * unroundedDurationUs), rounding only the final result. Error no longer accumulates; it stays bounded to at most +/-0.5us for the life of the stream. Found via real-world testing in a third-party player (Plezy) on full- length movies, where the 90-second on-device test clips used to validate the original fix could never have revealed it.
Author
|
Pushed a fix for a second bug: audio/video would slowly drift out of Tested in two real players on the affected hardware (Jellyfin Android |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3347
Problem
On Google TV Streamer 4K (MediaTek MT8696,
c2.mtk.hevc.decoder),decoding specific HEVC Main10 content causes the codec to return
output buffers with non-monotonic
presentationTimeUs, while theorder in which those buffers are released remains correct — confirmed
by comparing each buffer's release-order rank against its
timestamp-sorted rank: the two never diverge by more than a few
positions, and the timestamp values themselves are individually
correct for the frame they belong to.
Reproduced independently of ExoPlayer, using a minimal test harness
built directly on
AMediaCodec/AMediaExtractor(no ExoPlayer, noJava
MediaCodec) against the same file: identical non-monotonictimestamp pattern. This confirms the defect is in the decoder/HAL
output, not in ExoPlayer's handling of it.
MediaCodecRendererprocesses output buffers in raw release orderbut uses the codec-reported
presentationTimeUsdirectly for therender/drop decision in
MediaCodecVideoRenderer. When a buffer'stimestamp is behind the previous one, it's treated as arriving too
late and dropped. Measured on affected content: ~30% of decoded video
frames dropped, sustained throughout playback, reproduced in the
Media3 demo app as well as Plex, Jellyfin, and other ExoPlayer-based
players on the same device/file.
Fix
Re-derive each output buffer's
presentationTimeUsfrom its releaseorder and the stream's known frame duration (from
Format.frameRate),instead of trusting the codec-reported value. Buffer release order is
left untouched — buffers are not held or reordered.
Testing
Verified on the affected device (Google TV Streamer 4K) against three
files that reliably reproduce the issue on unpatched
main,including a 90-second real-world clip: dropped-frame count goes from
~30% to 0, and the fix was visually confirmed on-device — direct
observation of smooth playback, not just log metrics. HOWEVER, with this approach, video and audio lose sync.
An earlier approach that buffered and released output buffers in
sorted-timestamp order was also tried and rejected: it also eliminated
logged drops, but produced visible playback artifacts on-device. That
result is why this PR does not reorder buffers.