fix: avoid deadlock when growing dynamic thread-local slots

[zackees]

Symptom

On current dev3 (f45596ab), a full-debug build hangs deterministically when one thread creates enough heaps to grow the dynamic thread-local slot array beyond its initial capacity.

This is reachable through public APIs only:

#include <mimalloc.h>

int main(void) {
  mi_heap_t* heaps[40];
  for (size_t i = 0; i < 40; i++) {
    heaps[i] = mi_heap_new();
    if (heaps[i] == NULL) return 1;
    if (mi_heap_malloc(heaps[i], 32) == NULL) return 2;
  }
  for (size_t i = 0; i < 40; i++) mi_heap_destroy(heaps[i]);
  return 0;
}

Configuration and result on stock upstream:

cmake -S . -B build \
  -DCMAKE_BUILD_TYPE=Debug \
  -DMI_DEBUG_FULL=ON \
  -DMI_OVERRIDE=OFF \
  -DMI_BUILD_SHARED=OFF \
  -DMI_BUILD_STATIC=ON
cmake --build build --target mimalloc-test-threadlocal-growth
timeout 10s build/mimalloc-test-threadlocal-growth

exit 124 (deterministic timeout)

Tested with GCC 14.2.1 on Linux. The same focused test also hangs with MinGW-w64 GCC 14.2.0.

Cause

The dynamic slot array was moved to meta allocation in ac61f0fb. Its growth path now forms a same-thread lock cycle:

mi_thread_locals_expand
  -> _mi_meta_rezalloc
     -> acquire subproc->theap_meta_lock
     -> mi_theap_rezalloc
        -> moving realloc calls mi_free(oldp)
           -> mi_stat_free recognizes a meta page
              -> acquire subproc->theap_meta_lock again

theap_meta_lock is not recursive, so the second acquisition never completes. An attached debugger stops in __lll_lock_wait.

The move is guaranteed once the slot array crosses a size class, so this is not timing-dependent. Applications that create enough independent heaps can make assertion/statistics-enabled allocator builds stop making progress.

Proposed fix

The slot array has a simpler lifetime rule than a general realloc needs:

1. allocate a zeroed replacement from the meta allocator while it holds its lock;
2. copy only the old logical header and slots;
3. publish the replacement to thread-local storage;
4. free the old meta block after the allocation lock has been released.

This removes the nested lock acquisition and explicitly guarantees that newly added slots are zero. _mi_meta_rezalloc has no other callers and can be removed.

I have a focused regression test that is RED by deterministic timeout before the change and exits successfully afterward. I will send the patch as a separate PR referencing this issue.

[zackees]

Daan (@daanx) For reference, this failure mode is already avoided on zackees/mimalloc-pprof's main branch:

• PR #137 (C-core commit) makes mi_thread_locals_expand grow the TLS slot array from the process main heap, so it does not hold theap_meta_lock while a moving realloc frees the old array.
• PR #148 (C-core commit) explicitly zeroes the newly added slot range after growth.

That fork is pinned before ac61f0fb, so this is prior art rather than a directly portable patch for current dev3; the current meta-backed path still needs the allocate/copy/publish/free ordering described in this report.

[daanx]

Thanks! I pushed a fix. I'd like to keep it a meta allocation for (later) improved security where metadata gets guard pages around it. (Coincidentally, the issue with rezalloc not zero'ing correctly was already fixed in the latest release (v3.4.5)). I also added the test case if that is ok with you.

[zackees]

Can you provide me the url to your version of the fix so I can keep my version of the repo more aligned with the official upstream dev3?

I've asked the AI to further harden and find look alike bugs and shake them. You might have one or two similar PRs tomorrow if more bugs shake out from the effort.

[daanx]

Ah, recently I have been quite active with improving v3 for first-class heaps etc, and there were many changes which is not so usual for mimalloc. I think soonish it'll stabilize again and it'll be easier to keep it aligned. My hope is that you would eventually not need a "patched" version of mimalloc but can just use the upstream as is. Maybe we can add particular hooks to make this easier?

For the "rezalloc" fix, there is not one commit to point to unfortunately as it interacted with the umalloc api's and I revised most of it.
(like #427df0b47)

For LLM audits, sometimes it can find many issues; If there are many, sometimes we post them as a list as one github issue , like in the ongoing issue #1271, so I can address them in due time. If it is security sensitive you can also send issues to daan at microsoft.com . Or if it is clearly a confirmed reproducible issue (as the one here for example), reporting as you do now (per issue) is fine as well.

[zackees]

Daan (@daanx) Follow-up: the fork-side regression/diagnostic work has now landed in zackees/mimalloc-pprof#169.

It adds the public-API TLS growth test across capacity boundaries and thread/subprocess teardown, plus an allocation-free MI_DEBUG_FULL lock-owner diagnostic whose same-thread reacquisition control fails fast instead of timing out. The CI positive controls also exercise wrong TLS ownership, skipped zeroing, growth-allocation failure, and every lock diagnostic. Release builds are mechanically checked to contain no diagnostic source, storage, symbols, hooks, instructions, or data for either profiler ON or OFF.

The fork is still pinned before the upstream meta-allocation regression, so this is a durable detector for the next selective overlay rather than a replacement for your upstream fix. Thanks for fixing the meta-backed path and adding the reproducer upstream.

[zackees]

Previous message was from my AI fyi.

About the possibility of unpatch version:

The question falls to this: does microsoft have the appetite for pprof support? Maybe they can be done with hooks... that's an interesting question. However there is one standard way to do pprof which both TCMalloc and JEMalloc uses. Malloc stats is a bit harder: it's faster since theres no back trace but it must know about the internals of the allocator so the more stable api is one that hooks less and fills in a stats struct. This means would be the most stable api since the internal ledger can change without affecting the malloc stats shape.