Skip to content

Fix Unity media stream lifetime leaks and add latest-frame video coalescing#198

Open
xianshijing-lk wants to merge 4 commits intomainfrom
sxian/CLT-1/fix_audio_video_streams_leaks
Open

Fix Unity media stream lifetime leaks and add latest-frame video coalescing#198
xianshijing-lk wants to merge 4 commits intomainfrom
sxian/CLT-1/fix_audio_video_streams_leaks

Conversation

@xianshijing-lk
Copy link
Contributor

This PR tightens Unity-side media stream lifetime management and reduces unnecessary video frame retention under bursty delivery.

What changed

  • Fixed AudioStream cleanup so it fully releases owned resources on dispose:
    • unsubscribe from FfiClient.AudioStreamEventReceived
    • unsubscribe/destroy AudioProbe
    • dispose RingBuffer
    • dispose AudioResampler
    • dispose the owned FFI stream handle
  • Fixed per-callback audio frame wrapper leaks by disposing both:
    • the incoming AudioFrame
    • the remixed/resampled output AudioFrame
  • Made AudioResampler explicitly disposable and responsible for releasing its native resampler handle.
  • Fixed VideoStream cleanup so it:
    • unsubscribes from FfiClient.VideoStreamEventReceived
    • disposes current and pending native frame buffers
    • disposes the owned FFI stream handle
  • Reworked VideoStream buffering to latest-frame-wins:
    • separate pending vs currently consumed frame buffers
    • coalesce bursts to the newest pending frame
    • drop superseded pending frames immediately
    • drop incoming frames immediately when the stream is not actively playing
    • dispose the currently displayed native frame buffer on Stop()
  • Added comments documenting the producer/consumer split between _pendingBuffer and VideoBuffer.
  • Added EditMode regression tests covering media stream disposal and video coalescing invariants.

Why
Before this change, Unity media streams could retain stale event subscriptions and native resources across unsubscribe/disconnect/reconnect flows. AudioStream and VideoStream subscribed to long-lived FfiClient events but did not fully
tear down owned resources, and audio callback-scoped frame wrappers were not being disposed.

For video, Unity also only had a single active buffer slot, which was less explicit about producer/consumer separation and did not document/latest-frame coalescing behavior clearly. This PR makes that behavior explicit and ensures
older pending frames are dropped immediately rather than retained.

Behavioral impact

  • No public API changes.
  • Under video burst/backlog conditions, Unity now more explicitly keeps only the newest pending frame for upload.
  • If VideoStream is not actively playing, incoming video frames are dropped immediately on the Unity side instead of being retained.
  • Media stream disposal is stricter and releases native resources sooner.

Testing

  • Added EditMode regression tests for media stream cleanup and latest-frame coalescing structure.
  • I could not run a build in this shell because dotnet / msbuild are not available here.

Copy link
Contributor

@cloudwebrtc cloudwebrtc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants