v3: arena growth wedges at the 16 GiB reserve cap (info_slices 521 >= 512); all later allocations fall back to per-allocation OS mmap and exhaust vm.max_map_count

[andrewgazelka]

Summary

On mimalloc v3 (observed on 3.3.2, code unchanged on current dev3), a long-running allocation-heavy process (a Nix-language evaluator with a ~60 GiB peak heap) eventually has every mi_arena_reserve fail. From that point each allocation falls back to a direct OS mmap, the process accumulates mappings until vm.max_map_count (1,048,576 on our hosts) is exhausted, and it aborts with ENOMEM ("memory allocation of N bytes failed"). The same workload on mimalloc v2.3.2 finishes with 41 mappings.

Mechanics

mi_arena_reserve (src/arena.c) doubles the reservation every 8 arenas and clamps it to MI_ARENA_MAX_SIZE:

if (arena_count >= 1 && arena_count <= 128) {
  // scale up the arena sizes exponentially every 8 entries
  const size_t multiplier = (size_t)1 << _mi_clamp(arena_count/8, 0, 16);
  ...
}
...
const size_t max_reserve = MI_ARENA_MAX_SIZE;   // 16 GiB
...
else if (arena_reserve > max_reserve) {
  arena_reserve = max_reserve;
}

But a full 16 GiB arena can never initialize: mi_arena_initialize computes info_slices = 521 for slice_count = MI_BITMAP_MAX_BIT_COUNT (262,144 slices of 64 KiB) and rejects it:

else if (info_slices >= MI_ARENA_MAX_CHUNK_OBJ_SLICES) {   // 512
  _mi_warning_message("cannot use OS memory since it is too large with respect to the maximum object size (size %zu MiB, meta-info slices %zu, maximum object slices are %zu)", ...);
  return NULL;
}

So once the workload has created enough arenas that the doubled reservation hits the 16 GiB clamp, the reserve size is pinned at a value the initializer always refuses. mi_arena_reserve keeps proposing 16 GiB, mi_arena_initialize keeps returning NULL, and the allocator never gets a new arena again. With MIMALLOC_VERBOSE=1 the log fills with:

mimalloc: warning: cannot use OS memory since it is too large with respect to the maximum object size (size 16384 MiB, meta-info slices 521, maximum object slices are 512)

The explicit-reserve path has the same problem: mi_reserve_os_memory_ex clamps slice_count to MI_BITMAP_MAX_BIT_COUNT, which is exactly the size the initializer rejects.

Impact

After the wedge, every allocation that misses existing arenas takes the OS fallback. For a heap that keeps growing, that is one mmap per allocation: our evaluator hit 1,048,578 mappings and aborted, with RSS far below the cgroup limit. The failure looks like an OOM but is really mapping-count exhaustion.

Suggested fix

Either make the clamp respect the initializer's constraint, e.g.

// largest size whose meta-info still fits under MI_ARENA_MAX_CHUNK_OBJ_SLICES
const size_t max_reserve = MI_ARENA_MAX_SIZE - mi_size_of_slices(MI_ARENA_MAX_CHUNK_OBJ_SLICES);

(or compute the exact largest slice_count with info_slices < MI_ARENA_MAX_CHUNK_OBJ_SLICES and clamp to that), or allow mi_arena_initialize to accept the full bitmap size by reserving the meta-info slices out of the object space. Happy to send a PR for the clamp variant if that is the preferred direction.

Environment

• mimalloc 3.3.2 (via libmimalloc-sys 0.1.49) and current dev3 source
• Linux x86_64 (musl-linked Rust binary), 1.5 TB host, default vm.max_map_count raised to 1,048,576

[daanx]

Yikes -- thanks for the report! This was unexpected but I see now it was introduced recently. I saw the PR but I pushed a simpler fix as I think the condition that the info slices take up at most a chunk is not really needed and we can go over that. I am now looking at trying to reduce the meta size further in other ways to again end up within a chunk. Thanks again.