rtsp: don't free the server handle while encoder threads still use it - #40
Draft
johnchia wants to merge 1 commit into
Draft
rtsp: don't free the server handle while encoder threads still use it#40johnchia wants to merge 1 commit into
johnchia wants to merge 1 commit into
Conversation
main.c calls rtsp_finish() at shutdown and then keeps going for another dozen lines before sdk_stop() finally stops the encoder threads. It never clears the rtspHandle global, and media.c passes that global straight into rtp_send_h26x() and rtp_send_mp3(), so every frame produced in that window dereferences a handle that rtsp_finish() has already freed. The guard that was supposed to catch this could not: it read the quit flag through h->pool->sharedp->gbl, but rtsp_finish() raises that flag and then calls threadpool_delete(h->pool), so the check consulted freed memory to decide whether the memory was freed. Once the allocation is reused -- and region_stop() and night_disable() run in between, so it is -- the check reads whatever now occupies that word, passes, and the sender goes on to lock a destroyed mutex and walk a destroyed connection list. Keep the shutdown flag in the handle itself, and stop freeing the handle and destroying its mutex in rtsp_finish(), so a late sender always finds a valid mutex and a flag telling it to turn around. The connections, pools and threadpool are still released as before. rtsp_create() runs once per process, so what remains is one small allocation reclaimed at exit; rtsp_create()'s own error path frees the handle explicitly, since nothing can reach it there yet. Seen on an Infinity6E (ssc30kq) as a SIGSEGV on every SIGTERM restart with two clients connected, faulting a few dozen bytes into the sender's connection-list walk on a junk pointer.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
rtsp_finish()frees the server handle, but the encoder threads keep using it for a while afterwards.main.ctears down in this order:rtspHandleis never cleared, andmedia.cpasses it straight intortp_send_h26x()/rtp_send_mp3(). So every frame encoded between :115 and :128 dereferences freed memory.The guard that should have caught this can't, because it lives in the memory it is checking:
rtsp_finish()raises that quit flag and then callsthreadpool_delete(h->pool)andFREE(h). The check reads freed memory to decide whether the memory has been freed.region_stop()andnight_disable()allocate in between, so the chunk gets reused, the check reads whatever now occupies that word, passes, and the sender goes on tortsp_lock(h)on a destroyed mutex andlist_map_inline(&h->con_list, ...)over a destroyed list.DASSERT(h, ...)doesn't help —his dangling, not NULL.The fix
Move the shutdown flag into the handle itself, and stop freeing the handle in
rtsp_finish():struct __rtsp_obj_tgains afinishedflag, guarded by the existing mutex.rtsp_finish()raises it first, before releasing anything else.h->finishedinstead of reaching throughh->pool.rtsp_finish()no longer callspthread_mutex_destroy()/FREE(h), so a late sender always finds a valid mutex and a flag telling it to turn around.Connections, buffer pools, MIME buffers and the threadpool are all still released exactly as before. What stays allocated is the handle struct itself.
rtsp_create()runs once per process, so that's a single small allocation reclaimed at exit.rtsp_create()'s own error path frees it explicitly, since nothing can reach it there yet.Alternative worth considering
Calling
sdk_stop()beforertsp_finish()inmain.cwould remove the hazardous window entirely and let the handle be freed normally. I didn't do that here because it changes shutdown ordering for every SoC and I can only test one — but if you'd prefer that shape, I'm happy to redo it.Testing
Found on an Infinity6E (ssc30kq) running under Frigate with two streams: a SIGTERM restart segfaulted essentially every time, the faulting address landing a few dozen bytes into the sender's connection-list walk on a junk pointer. The equivalent fix has been running clean on that hardware since.
Cross-compiled clean for
arm-openipc-linux-gnueabihfwith no new warnings. Only the ssc30kq path is hardware-tested; the change is SoC-independent but a second pair of eyes on another target would be welcome.Not addressed here
There's a narrower pre-existing race left in place: a sender that gets past the flag check can still be mid-send, holding transfer items, when
rtsp_finish()deletes the pools. Closing it needs a drain barrier on the sender count. I left it out deliberately since I can't exercise it on this stack — happy to follow up separately if you want it.