Observed below crash with Traffic Server 8.1.11
Stacktrace:
#0 in ProcessManager::signalManager (this=0x559e532ae840, msg_id=msg_id@entry=16, data_raw=data_raw@entry=0x559e53a81a90 "\003", data_len=135752) at ProcessManager.cc:268
#1 in RecMessageSend (msg=msg@entry=0x559e53a81a90) at RecProcess.cc:325
#2 in send_push_message () at P_RecCore.cc:123
#3 in sync_cont::sync (this=0x559e532f88e0) at RecProcess.cc:194
#4 in Continuation::handleEvent (data=0x559e53308ac0, event=2, this=<optimized out>) at I_Continuation.h:160
#5 EThread::process_event (this=this@entry=0x7f4b69e36010, e=e@entry=0x559e53308ac0, calling_code=2) at UnixEThread.cc:131
#6 in EThread::execute_regular (this=0x7f4b69e36010) at UnixEThread.cc:244
#7 in spawn_thread_internal (a=0x559e533028c0) at Thread.cc:85
#8 in start_thread () from /lib/x86_64/libc.so.6
#9 in clone3 () from /lib/x86_64/libc.so.6
Environment Details:
ATS Version: 8.1.11
Platform: Linux x86_64
Compiler: g++ 11.5.0
In ATS 8.1.11, ProcessManager::signalManager(int msg_id, const char *data_raw,int data_len) allocates a MgmtMessageHdr *mh, fills it, and enqueues it on mgmt_signal_queue. After that enqueue, the process-manager thread owns the message and may dequeue and free it in processSignalQueue.
However, the sender still reads mh->msg_id after publishing the pointer to the queue:
enqueue(mgmt_signal_queue, mh)
if (wakeup_fd != NO_FD && **mh->msg_id** != MGMT_SIGNAL_LIBRECORDS) ...
That is a race condition. If the process-manager thread runs between the enqueue and the sender's mh->msg_id read, it can dequeue the same mh, write it to traffic_manager, and call ats_free(mh). The sender then dereferences freed memory at the line reported by ACR (ProcessManager.cc:268).
Why it is rare:
- The race window is very small.
MGMT_SIGNAL_LIBRECORDS is common, but the crash only occurs if scheduling
I have simulated this crash with help of local c++ multi-threaded program, file attached (reproduce_msgq_uaf.cpp)
reproduce_msgq_uaf.cpp
Is there any known issue?
Observed below crash with Traffic Server 8.1.11
Stacktrace:
Environment Details:
ATS Version: 8.1.11
Platform: Linux x86_64
Compiler: g++ 11.5.0
In ATS 8.1.11,
ProcessManager::signalManager(int msg_id, const char *data_raw,int data_len)allocates aMgmtMessageHdr *mh, fills it, and enqueues it onmgmt_signal_queue. After that enqueue, the process-manager thread owns the message and may dequeue and free it inprocessSignalQueue.However, the sender still reads
mh->msg_idafter publishing the pointer to the queue:That is a race condition. If the process-manager thread runs between the enqueue and the sender's
mh->msg_idread, it can dequeue the samemh, write it to traffic_manager, and callats_free(mh). The sender then dereferences freed memory at the line reported by ACR (ProcessManager.cc:268).Why it is rare:
MGMT_SIGNAL_LIBRECORDSis common, but the crash only occurs if schedulingI have simulated this crash with help of local c++ multi-threaded program, file attached (reproduce_msgq_uaf.cpp)
reproduce_msgq_uaf.cpp
Is there any known issue?