setenv can trigger a use-after-free in mimalloc's early/lazy option init when environ is reallocated.
Hi! I'm hitting a mimalloc crash in an internal setup. It turns out a setenv call is not always safe with mimalloc. I tried to reproduce the issue and ended up with this.
In short: until recently, setenv on glibc calls realloc, which can cause environ to be moved to another memory location. While the old chunk is being freed, mimalloc may query an environment variable while initializing one of its options (mi_option_visit_abandoned). This leads to a read of already-freed memory.
This is fairly rare, since the option should be in an UNINIT state at that point to trigger the issue. I see two ways it can happen:
- mimalloc is statically linked and its constructor runs after some allocator calls made from other libs' constructors.
setenv is called somewhere early in the program, so the option is still uninitialized.
setenv is called before mimalloc's constructor is called (os_preloading is still true).
The branch is based on upstream/dev3, mostly because the issue I hit - and the first reproducer - is specific to dev3 (as it allows options to be lazily initialized). The improved reproducer could probably be made to work on the other versions with some modification, but I haven't tried it.
As far as I understand the C standard, there's no guarantee that getenv can be called from within a setenv call. Even though we're not necessarily calling getenv directly (MI_USE_ENVIRON != 0), I am not sure we can rely on that.
That said, there seems to be a fix on the glibc side: bminor/glibc@7a61e7f. I haven't tested it, but it's probably enough to say the issue doesn't occur with a recent enough glibc (fix stated specifically "getenv is fully reentrant and can be called from the malloc call in setenv", if a replacement malloc uses getenv during its initialization").
If supporting older glibc matters, it's probably worth special-casing the environ realloc. We could either try to handle it somehow, as in my proof-of-concept fix (deliberately-naive workaround, not an actually proposed fix) or, even better, restrict option-init attampts in this case.
The branch is structured as follows:
- reproducer for lazy init + setenv in main
- fix (could lead to second redundant options loop and not fixing the issue completely)
- reproducer for
setenv before main, showing the fix is not sufficient
- fix rollback
- proof-of-concept fix with manual
environ handling. Definitely not ready to be a fix proposal, I'd like to discuss the correct approach before going further.
commit: e504759126c1ae7c8f8d0dd975307e8c1ba40736
uname -a: Linux 6.6.87.2-microsoft-standard-WSL2 #1 SMP PREEMPT_DYNAMIC x86_64 (WSL2)
glibc: Ubuntu GLIBC 2.39-0ubuntu8.7
cc: gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0
linkage: static (libmimalloc-debug.a), MI_DEBUG_FULL=ON
Here's the backtrace, which shows the full chain: setenv reallocating environ through mimalloc, the resulting free reaching abandoned-page reclaim, and the lazy option init then faulting in getenv:
stacktrace:
```
* thread #1, name = 'mimalloc-test-r', stop reason = signal SIGSEGV: invalid address (fault address: 0x0)
* frame #0: 0x000055555557ac33 mimalloc-test-repro-setenv-lazy`_mi_strnicmp(s="mimalloc_visit_abandoned", t="", n=24) at libc.c:27:21
frame #1: 0x00005555555917ab mimalloc-test-repro-setenv-lazy`_mi_prim_getenv(name="mimalloc_visit_abandoned", result="\xfa\xbf8\xf6\xe1\xfb\U0000001cl\xea\U0000001fn\xce\xe9\xd4\xfa\x948", result_size=65) at prim.c:833:9
frame #2: 0x000055555557af69 mimalloc-test-repro-setenv-lazy`_mi_getenv(name="mimalloc_visit_abandoned", result="\xfa\xbf8\xf6\xe1\xfb\U0000001cl\xea\U0000001fn\xce\xe9\xd4\xfa\x948", result_size=65) at libc.c:100:10
frame #3: 0x000055555557d902 mimalloc-test-repro-setenv-lazy`mi_option_init(desc=0x00005555555b15e0) at options.c:639:16
frame #4: 0x000055555557c5c5 mimalloc-test-repro-setenv-lazy`mi_option_get(option=mi_option_visit_abandoned) at options.c:277:5
frame #5: 0x000055555557c845 mimalloc-test-repro-setenv-lazy`mi_option_is_enabled(option=mi_option_visit_abandoned) at options.c:324:11
frame #6: 0x000055555556bd94 mimalloc-test-repro-setenv-lazy`_mi_arenas_page_unabandon(page=0x0000020000000000, current_theapx=0x00005555555af280) at arena.c:1219:48
frame #7: 0x000055555555aebb mimalloc-test-repro-setenv-lazy`mi_abandoned_page_try_reclaim(page=0x0000020000000000, reclaim_on_free=0) at free.c:349:3
frame #8: 0x000055555555b0eb mimalloc-test-repro-setenv-lazy`mi_free_try_collect_mt(page=0x0000020000000000, mt_free=0x0000000000000000) at free.c:384:9
frame #9: 0x000055555555a183 mimalloc-test-repro-setenv-lazy`mi_free_block_mt(page=0x0000020000000000, block=0x0000020000004000, was_guarded=false) at free.c:85:5
frame #10: 0x000055555555a6c9 mimalloc-test-repro-setenv-lazy`mi_free [inlined] mi_free_ex(page=0x0000020000000000, usable=0x0000000000000000, p=0x0000020000004000) at free.c:199:5
frame #11: 0x000055555555a68e mimalloc-test-repro-setenv-lazy`mi_free(p=0x0000020000004000) at free.c:209:3
frame #12: 0x000055555556269a mimalloc-test-repro-setenv-lazy`_mi_theap_realloc_zero(theap=0x00005555555af280, p=0x0000020000004000, newsize=12256, zero=false, usable_pre=0x0000000000000000, usable_post=0x0000000000000000) at alloc.c:408:7
frame #13: 0x00005555555626e1 mimalloc-test-repro-setenv-lazy`mi_theap_realloc(theap=0x00005555555af280, p=0x0000020000004000, newsize=12256) at alloc.c:415:10
frame #14: 0x0000555555562898 mimalloc-test-repro-setenv-lazy`mi_realloc(p=0x0000020000004000, newsize=12256) at alloc.c:444:10
frame #15: 0x00007ffff7c4ab80 libc.so.6`__add_to_environ(name="besttest1", value="some giant value", combined=0x0000000000000000, replace=) at setenv.c:156:31
frame #16: 0x0000555555558687 mimalloc-test-repro-setenv-lazy`repro_preload_ctor at test-repro-setenv-lazy.c:137:5
frame #17: 0x00007ffff7c2a304 libc.so.6`__libc_start_main_impl [inlined] call_init(env=, argv=0x00007fffffffddd8, argc=1)at libc-start.c:145:3
frame #18: 0x00007ffff7c2a28b libc.so.6`__libc_start_main_impl(main=(mimalloc-test-repro-setenv-lazy`main at test-repro-setenv-lazy.c:142:16), argc=1, argv=0x00007fffffffddd8, init=, fini=, rtld_fini=, stack_end=0x00007fffffffddc8) atlibc-start.c:347:5
frame #19: 0x0000555555558445 mimalloc-test-repro-setenv-lazy`_start + 37
```
Hi! I'm hitting a mimalloc crash in an internal setup. It turns out a
setenvcall is not always safe with mimalloc. I tried to reproduce the issue and ended up with this.In short: until recently,
setenvon glibc callsrealloc, which can causeenvironto be moved to another memory location. While the old chunk is being freed, mimalloc may query an environment variable while initializing one of its options (mi_option_visit_abandoned). This leads to a read of already-freed memory.This is fairly rare, since the option should be in an UNINIT state at that point to trigger the issue. I see two ways it can happen:
setenvis called somewhere early in the program, so the option is still uninitialized.setenvis called before mimalloc's constructor is called (os_preloadingis still true).The branch is based on
upstream/dev3, mostly because the issue I hit - and the first reproducer - is specific to dev3 (as it allows options to be lazily initialized). The improved reproducer could probably be made to work on the other versions with some modification, but I haven't tried it.As far as I understand the C standard, there's no guarantee that
getenvcan be called from within asetenvcall. Even though we're not necessarily callinggetenvdirectly (MI_USE_ENVIRON != 0), I am not sure we can rely on that.That said, there seems to be a fix on the glibc side: bminor/glibc@7a61e7f. I haven't tested it, but it's probably enough to say the issue doesn't occur with a recent enough glibc (fix stated specifically "
getenvis fully reentrant and can be called from the malloc call insetenv", if a replacement malloc usesgetenvduring its initialization").If supporting older glibc matters, it's probably worth special-casing the
environrealloc. We could either try to handle it somehow, as in my proof-of-concept fix (deliberately-naive workaround, not an actually proposed fix) or, even better, restrict option-init attampts in this case.The branch is structured as follows:
setenvbeforemain, showing the fix is not sufficientenvironhandling. Definitely not ready to be a fix proposal, I'd like to discuss the correct approach before going further.Here's the backtrace, which shows the full chain:
setenvreallocatingenvironthrough mimalloc, the resultingfreereaching abandoned-page reclaim, and the lazy option init then faulting ingetenv:stacktrace:
``` * thread #1, name = 'mimalloc-test-r', stop reason = signal SIGSEGV: invalid address (fault address: 0x0) * frame #0: 0x000055555557ac33 mimalloc-test-repro-setenv-lazy`_mi_strnicmp(s="mimalloc_visit_abandoned", t="", n=24) at libc.c:27:21 frame #1: 0x00005555555917ab mimalloc-test-repro-setenv-lazy`_mi_prim_getenv(name="mimalloc_visit_abandoned", result="\xfa\xbf8\xf6\xe1\xfb\U0000001cl\xea\U0000001fn\xce\xe9\xd4\xfa\x948", result_size=65) at prim.c:833:9 frame #2: 0x000055555557af69 mimalloc-test-repro-setenv-lazy`_mi_getenv(name="mimalloc_visit_abandoned", result="\xfa\xbf8\xf6\xe1\xfb\U0000001cl\xea\U0000001fn\xce\xe9\xd4\xfa\x948", result_size=65) at libc.c:100:10 frame #3: 0x000055555557d902 mimalloc-test-repro-setenv-lazy`mi_option_init(desc=0x00005555555b15e0) at options.c:639:16 frame #4: 0x000055555557c5c5 mimalloc-test-repro-setenv-lazy`mi_option_get(option=mi_option_visit_abandoned) at options.c:277:5 frame #5: 0x000055555557c845 mimalloc-test-repro-setenv-lazy`mi_option_is_enabled(option=mi_option_visit_abandoned) at options.c:324:11 frame #6: 0x000055555556bd94 mimalloc-test-repro-setenv-lazy`_mi_arenas_page_unabandon(page=0x0000020000000000, current_theapx=0x00005555555af280) at arena.c:1219:48 frame #7: 0x000055555555aebb mimalloc-test-repro-setenv-lazy`mi_abandoned_page_try_reclaim(page=0x0000020000000000, reclaim_on_free=0) at free.c:349:3 frame #8: 0x000055555555b0eb mimalloc-test-repro-setenv-lazy`mi_free_try_collect_mt(page=0x0000020000000000, mt_free=0x0000000000000000) at free.c:384:9 frame #9: 0x000055555555a183 mimalloc-test-repro-setenv-lazy`mi_free_block_mt(page=0x0000020000000000, block=0x0000020000004000, was_guarded=false) at free.c:85:5 frame #10: 0x000055555555a6c9 mimalloc-test-repro-setenv-lazy`mi_free [inlined] mi_free_ex(page=0x0000020000000000, usable=0x0000000000000000, p=0x0000020000004000) at free.c:199:5 frame #11: 0x000055555555a68e mimalloc-test-repro-setenv-lazy`mi_free(p=0x0000020000004000) at free.c:209:3 frame #12: 0x000055555556269a mimalloc-test-repro-setenv-lazy`_mi_theap_realloc_zero(theap=0x00005555555af280, p=0x0000020000004000, newsize=12256, zero=false, usable_pre=0x0000000000000000, usable_post=0x0000000000000000) at alloc.c:408:7 frame #13: 0x00005555555626e1 mimalloc-test-repro-setenv-lazy`mi_theap_realloc(theap=0x00005555555af280, p=0x0000020000004000, newsize=12256) at alloc.c:415:10 frame #14: 0x0000555555562898 mimalloc-test-repro-setenv-lazy`mi_realloc(p=0x0000020000004000, newsize=12256) at alloc.c:444:10 frame #15: 0x00007ffff7c4ab80 libc.so.6`__add_to_environ(name="besttest1", value="some giant value", combined=0x0000000000000000, replace=) at setenv.c:156:31 frame #16: 0x0000555555558687 mimalloc-test-repro-setenv-lazy`repro_preload_ctor at test-repro-setenv-lazy.c:137:5 frame #17: 0x00007ffff7c2a304 libc.so.6`__libc_start_main_impl [inlined] call_init(env=, argv=0x00007fffffffddd8, argc=1)at libc-start.c:145:3 frame #18: 0x00007ffff7c2a28b libc.so.6`__libc_start_main_impl(main=(mimalloc-test-repro-setenv-lazy`main at test-repro-setenv-lazy.c:142:16), argc=1, argv=0x00007fffffffddd8, init=, fini=, rtld_fini=, stack_end=0x00007fffffffddc8) atlibc-start.c:347:5 frame #19: 0x0000555555558445 mimalloc-test-repro-setenv-lazy`_start + 37 ```