v3.3.1 macOS arm64: SIGSEGV in _mi_heap_main via mi_thread_init — theap->tld is NULL on a freshly-initialized thread heap

[adamhunter]

Summary

Two frame-identical SIGSEGVs (EXC_BAD_ACCESS, KERN_INVALID_ADDRESS at 0x18) in mimalloc
v3.3.1 statically linked into Apache Arrow C++ 25.0.0 (libarrow.2500.dylib, MI_OVERRIDE=OFF,
MI_BUILD_SHARED=OFF), on macOS 26.5.1 arm64.

The crash is a NULL mi_theap_t::tld dereferenced by _mi_heap_main(), reached from the final
statistics increment in mi_thread_init(). It is intermittent and load-dependent: it fires early in
a process that creates many short-lived threads, and the same workload later completed 32,091
operations untouched.

Environment

mimalloc	v3.3.1 (as pinned by apache/arrow cpp/thirdparty/versions.txt)
build flags	MI_OVERRIDE=OFF, MI_BUILD_SHARED=OFF, MI_BUILD_OBJECT=OFF, MI_LOCAL_DYNAMIC_TLS=ON, MI_NO_OPT_ARCH=ON; static lib linked into a dylib
host	macOS 26.5.1 (25F80), arm64 (Apple Silicon)
consumer	CPython 3.12.13, pyarrow 25.0.0; a Python ETL writing Delta tables via delta-rs
threads at crash	93 and 104 respectively (two tokio runtimes, a polars pool, ~32 Python worker threads)

Only one mimalloc instance is present in the process (verified by binary inspection: the other
native extensions bundle jemalloc or use system malloc; CPython 3.12 vendors no mimalloc). This is
not a multiple-instance teardown problem.

Faulting stack (identical in both crashes)

libarrow.2500.dylib  mi_heap_main
libarrow.2500.dylib  mi_thread_init
libarrow.2500.dylib  _mi_malloc_generic
libarrow.2500.dylib  mi_theap_malloc_zero_aligned_at_overalloc
libarrow.2500.dylib  arrow::MimallocAllocator::AllocateAligned(...)
libarrow.2500.dylib  arrow::BaseMemoryPoolImpl<MimallocAllocator>::Allocate(...)
libarrow.2500.dylib  arrow::SchemaExporter::Finish(ArrowSchema*)
libarrow.2500.dylib  arrow::ExportSchema(...)
libarrow.2500.dylib  ExportedArrayStream<RecordBatchReader,false>::StaticGetSchema(...)
<rust ext>           (Arrow C Data Interface consumer)
python3.12           thread_run → pythread_wrapper → _pthread_start

The faulting thread is a plain Python worker thread, and mi_thread_init in the stack means
this was that thread's first mimalloc allocation.

Instruction-level analysis

Disassembling the faulting PC (_mi_heap_main):

_mi_heap_main:
  mrs  x8, TPIDRRO_EL0        ; thread control block
  and  x8, x8, #0xfffffffffffffff8
  ldr  x9, [x8, #0x360]       ; 0x360 = 108*8 → fixed TSD slot 108 (MI_TLS_MODEL_FIXED_SLOT_DEFAULT)
  adrp x8, <static fallback>
  add  x8, x8, #0x40
  cbz  x9, .Lfallback         ; slot was NON-NULL → fallback NOT taken
  ldr  x9, [x9]               ; x9 = theap->tld        (mi_theap_t offset 0)   → loaded 0x0
  ldr  x9, [x9, #0x18]        ; x9 = tld->subproc      (mi_tld_t offset 0x18)  → *** SIGSEGV ***
  ldr  x0, [x9, #0x568]

• Offsets match include/mimalloc/types.h at v3.3.1: mi_theap_s.tld at 0, mi_tld_s.subproc at 0x18.
• Crash register state corroborates: x9 = 0x0, fault address 0x18, vmRegionInfo:
  "0x18 is not in any region."
• The return address places the call at the last statement of mi_thread_init() —
  mi_heap_stat_increase(mi_heap_main(), threads, 1).

So: the thread's theap pointer in TSD slot 108 was non-NULL (passing the only guard in
_mi_heap_main) while its tld was NULL.

Hypotheses I ruled out

1. macOS heap corruption at thread exit (SIGTRAP) since v3.4.0/v3.4.1's fixed-TLS-slot change (126/127) -- bisected to f211b370 #1333 (macOS TLS slots 126/127 regression). v3.3.1 predates it; the reverted-to slot 108 is
   what the disassembly shows in use. Also a different signature (SIGTRAP at thread exit).
2. Two independently-linked mimalloc v3 instances (CPython 3.14 vendored + static in a Python extension) SIGSEGV in _mi_theap_collect_retired at process exit when Arrow's v2 copy is also loaded (macOS arm64) #1327 (two mimalloc instances). Only one mimalloc in the process; verified by inspecting
   every loaded native library.
3. Stale empty-sentinel inherited via a recycled thread control block. _mi_thread_done() sets
   the slots to &_mi_theap_empty rather than NULL, and mimalloc writes slot 108 directly (no
   pthread_key_create), so a recycled TCB inheriting the sentinel looked like an excellent
   explanation for "non-NULL theap, NULL tld". Falsified: reading the shipped binary,
   __mi_theap_empty (in __DATA_CONST,__const) has .tld = <non-NULL static>, .heap = 0x0,
   .refcount = 1. A stale sentinel would give a non-NULL tld.
4. Unsynchronized mi_heap_main_init() double-checked init. _mi_process_attach is a load-time
   initializer in this build (tail-calls _mi_auto_process_init, and _mi_process_init is
   once-guarded), so process/main-heap init completes single-threaded at dylib load.

Remaining hypothesis (offered, not proven)

In _mi_thread_init_theap_default():

mi_tld_t* tld = mi_tld_alloc();
theap = _mi_theap_create(mi_heap_main(), tld);

there is no NULL check between those two lines, whereas the other _mi_theap_create() call site (in
heap.c) does check its result. If mi_tld_alloc() can return NULL — or return a tld that is not
yet published — a theap with tld == NULL is installed into the TSD slot, and the very next
statement in mi_thread_init() dereferences it. That would match the fault exactly. The
meta-page allocator backing mi_tld_alloc() is relatively young in the 3.3.x line, which is why I
raise it rather than assert it.

Question for maintainers: is mi_tld_alloc() allowed to fail or to return an unpublished
tld here, and should _mi_heap_main()'s guard cover a non-NULL theap with a NULL tld?

Reproduction

No minimal reproducer — the crash is non-deterministic and appeared twice in ~12 minutes under a
workload that creates a new short-lived thread per buffered flush, each of which makes its first
mimalloc allocation while ~100 threads are live. Identical subsequent runs completed 32,091 and
1,789 operations without crashing. Shape needed to reproduce: many short-lived threads, each
touching mimalloc for the first time, in a process with a high live-thread count.

I have two full .ips crash reports and can attach redacted copies or answer specific questions against them.

Workaround in use

Selecting Arrow's system allocator (ARROW_DEFAULT_MEMORY_POOL=system) removes mimalloc from the
allocation path entirely; verified by driving the same ExportSchema chain and confirming the
per-backend byte counters move to the system pool.

[daanx]

Hmm, not sure for the root cause but recently we switched to use pthreads on macOS -- can you try with the latest release (tag v3.4.5) ?
Also, over the last 2 weeks I updated the initialization logic for threads and that may also fix this -- the latest dev3 branch has these changes.