Skip to content

Commit 5562097

Browse files
committed
gh-149146: Add dealloc-depth counter fallback to trashcan trigger
Under memory pressure (for example RLIMIT_AS), Python's trashcan trigger in _Py_Dealloc never fires. The trigger relies on _Py_RecursionLimit_GetMargin, which compares the machine stack pointer against c_stack_soft_limit. When RLIMIT_AS prevents the kernel from growing the C stack, the kernel SIGSEGVs while the stack pointer is still megabytes above the soft limit (see sibling issue gh-150722 for an LLDB trace showing SP ~7.2 MB above c_stack_hard_limit at the SIGSEGV), so the trashcan never deposits and the recursive tuple_dealloc chain runs out the stack. Add a per-thread c_dealloc_depth counter as a fallback trigger, mirroring the historical _PyTrash_UNWIND_LEVEL=50 protection that was removed when the trashcan was consolidated into _Py_Dealloc. _Py_RecursionLimit_GetMargin remains the primary signal; the counter only kicks in when the stack-pointer signal cannot fire. _PyTrash_thread_destroy_chain itself bumps c_dealloc_depth for the duration of the drain so any _Py_Dealloc invoked while draining cannot recursively re-enter destroy_chain (which would rebuild the same unbounded recursion the trashcan exists to prevent). This mirrors the historical delete_nesting bookkeeping. The counter is per-tstate (no atomics needed in the free-threaded build) and is only touched for GC types, so non-GC types pay zero overhead. Add a regression test in test_gc that builds a 100,000-deep (b, None) tuple chain and asserts that ``del b`` cleans it up without a C-stack overflow.
1 parent 11f032f commit 5562097

5 files changed

Lines changed: 61 additions & 2 deletions

File tree

Include/cpython/pystate.h

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -177,6 +177,14 @@ struct _ts {
177177
*/
178178
PyObject *delete_later;
179179

180+
/* gh-149146: per-thread recursion counter for _Py_Dealloc. Acts as a
181+
* fallback trigger for the trashcan in scenarios where the
182+
* stack-pointer-based _Py_RecursionLimit_GetMargin cannot fire (most
183+
* notably when RLIMIT_AS prevents the kernel from growing the C stack,
184+
* so the kernel SIGSEGVs while the stack pointer is still well above
185+
* c_stack_soft_limit). */
186+
int c_dealloc_depth;
187+
180188
/* Tagged pointer to top-most critical section, or zero if there is no
181189
* active critical section. Critical sections are only used in
182190
* `--disable-gil` builds (i.e., when Py_GIL_DISABLED is defined to 1). In the

Lib/test/test_gc.py

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -468,6 +468,19 @@ def __del__(self):
468468
v = {1: v, 2: Ouch()}
469469
gc.disable()
470470

471+
def test_trashcan_nested_tuple_deep(self):
472+
# gh-149146: deallocating a deeply nested ``(b, None)``-style tuple
473+
# chain must not blow the C stack. The trashcan inside
474+
# ``_Py_Dealloc`` has two complementary triggers: the
475+
# stack-pointer-based ``_Py_RecursionLimit_GetMargin`` check and a
476+
# per-thread dealloc-depth counter. The counter ensures that the
477+
# trashcan still fires when the stack-pointer check cannot, e.g.
478+
# when ``RLIMIT_AS`` prevents the kernel from growing the C stack.
479+
b = None
480+
for _ in range(100_000):
481+
b = (b, None)
482+
del b # must not segfault
483+
471484
@threading_helper.requires_working_threading()
472485
def test_trashcan_threads(self):
473486
# Issue #13992: trashcan mechanism should be thread-safe
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Fix a crash in :c:func:`!_Py_Dealloc` when deallocating deeply nested
2+
container objects under memory pressure (for example after a
3+
:exc:`MemoryError`). The trashcan now also deposits objects based on a
4+
per-thread dealloc-depth counter, not only on the stack-pointer margin,
5+
so it still defers cleanup when ``RLIMIT_AS`` prevents the kernel from
6+
growing the C stack. Patch by Bhuvi.

Objects/object.c

Lines changed: 33 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3201,6 +3201,11 @@ _PyTrash_thread_deposit_object(PyThreadState *tstate, PyObject *op)
32013201
void
32023202
_PyTrash_thread_destroy_chain(PyThreadState *tstate)
32033203
{
3204+
/* gh-149146: bump c_dealloc_depth for the duration of the drain so
3205+
* that _Py_Dealloc invoked from the deallocators below does not see
3206+
* depth == 0 and re-enter destroy_chain recursively. Mirrors the
3207+
* historical _PyTrash_end / delete_nesting bookkeeping. */
3208+
tstate->c_dealloc_depth++;
32043209
while (tstate->delete_later) {
32053210
PyObject *op = tstate->delete_later;
32063211
destructor dealloc = Py_TYPE(op)->tp_dealloc;
@@ -3226,6 +3231,7 @@ _PyTrash_thread_destroy_chain(PyThreadState *tstate)
32263231
_PyObject_ASSERT(op, Py_REFCNT(op) == 0);
32273232
(*dealloc)(op);
32283233
}
3234+
tstate->c_dealloc_depth--;
32293235
}
32303236

32313237
void _Py_NO_RETURN
@@ -3286,6 +3292,19 @@ next" object in the chain to 0. This can easily lead to stack overflows.
32863292
To avoid that, if the C stack is nearing its limit, instead of calling
32873293
dealloc on the object, it is added to a queue to be freed later when the
32883294
stack is shallower */
3295+
3296+
/* gh-149146: Fallback trigger for the trashcan.
3297+
*
3298+
* The primary trigger above (margin < 2) compares the machine stack pointer
3299+
* with c_stack_soft_limit. Under RLIMIT_AS the kernel can refuse to grow
3300+
* the C stack and SIGSEGV while the stack pointer is still well above
3301+
* c_stack_soft_limit, so the primary trigger never fires. This counter
3302+
* deposits into the trashcan once we have recursed through _Py_Dealloc
3303+
* enough times to be sure no realistic dealloc chain would overflow the
3304+
* stack first. The value matches the historical _PyTrash_UNWIND_LEVEL
3305+
* (50) used before the trashcan was consolidated into _Py_Dealloc. */
3306+
#define _Py_DEALLOC_DEPTH_LIMIT 50
3307+
32893308
void
32903309
_Py_Dealloc(PyObject *op)
32913310
{
@@ -3294,10 +3313,14 @@ _Py_Dealloc(PyObject *op)
32943313
destructor dealloc = type->tp_dealloc;
32953314
PyThreadState *tstate = _PyThreadState_GET();
32963315
intptr_t margin = _Py_RecursionLimit_GetMargin(tstate);
3297-
if (margin < 2 && gc_flag) {
3316+
if (gc_flag && (margin < 2
3317+
|| tstate->c_dealloc_depth >= _Py_DEALLOC_DEPTH_LIMIT)) {
32983318
_PyTrash_thread_deposit_object(tstate, (PyObject *)op);
32993319
return;
33003320
}
3321+
if (gc_flag) {
3322+
tstate->c_dealloc_depth++;
3323+
}
33013324
#ifdef Py_DEBUG
33023325
#if !defined(Py_GIL_DISABLED) && !defined(Py_STACKREF_DEBUG)
33033326
/* This assertion doesn't hold for the free-threading build, as
@@ -3340,7 +3363,15 @@ _Py_Dealloc(PyObject *op)
33403363
Py_XDECREF(old_exc);
33413364
Py_DECREF(type);
33423365
#endif
3343-
if (tstate->delete_later && margin >= 4 && gc_flag) {
3366+
if (gc_flag) {
3367+
tstate->c_dealloc_depth--;
3368+
}
3369+
/* gh-149146: only drain at the very top of the dealloc chain.
3370+
* _PyTrash_thread_destroy_chain itself bumps c_dealloc_depth so any
3371+
* _Py_Dealloc invoked while draining cannot recursively re-enter the
3372+
* drain (which would otherwise rebuild the same unbounded recursion
3373+
* the trashcan exists to prevent). */
3374+
if (tstate->delete_later && gc_flag && tstate->c_dealloc_depth == 0) {
33443375
_PyTrash_thread_destroy_chain(tstate);
33453376
}
33463377
}

Python/pystate.c

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1628,6 +1628,7 @@ init_threadstate(_PyThreadStateImpl *_tstate,
16281628
_tstate->jit_tracer_state = NULL;
16291629
#endif
16301630
tstate->delete_later = NULL;
1631+
tstate->c_dealloc_depth = 0;
16311632

16321633
llist_init(&_tstate->mem_free_queue);
16331634
llist_init(&_tstate->asyncio_tasks_head);

0 commit comments

Comments
 (0)