Skip to content

Commit 70a3de4

Browse files
committed
gh-155151: Check the recursion limit in CALL_EX_PY and CALL_KW_BOUND_METHOD
The generic CALL_FUNCTION_EX and CALL_KW opcodes reach start_frame, which calls _Py_EnterRecursivePy() and raises RecursionError before the callee runs. Their specialized forms end in _PUSH_FRAME, which decrements py_recursion_remaining without checking it, so once a call site had been warmed up the callee was entered and returned normally where the unspecialized instruction raised. Warming a call site should not change whether the target function executes. CALL_KW_PY and CALL_BOUND_METHOD_EXACT_ARGS already guard their frame push with _CHECK_RECURSION_REMAINING; add it to these two as well. It deopts, so it goes where a deopt is already safe: after the existing flush in CALL_KW_BOUND_METHOD, following the pattern of CALL_BOUND_METHOD_EXACT_ARGS, and after _CHECK_IS_PY_CALLABLE_EX in CALL_EX_PY, which already exits at that point.
1 parent 4130af1 commit 70a3de4

6 files changed

Lines changed: 122 additions & 3 deletions

File tree

Include/internal/pycore_opcode_metadata.h

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Lib/test/test_opcache.py

Lines changed: 81 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
import types
77
import unittest
88
from test.support import (threading_helper, check_impl_detail,
9-
requires_specialization,
9+
infinite_recursion, requires_specialization,
1010
cpython_only, requires_jit_disabled, reset_code)
1111
from test.support.import_helper import import_module
1212

@@ -568,6 +568,86 @@ def test(default=None):
568568
with self.assertRaises(RecursionError):
569569
test()
570570

571+
# gh-155151: a specialized call opcode must not enter the callee where the
572+
# generic one raises RecursionError. Warming a call site must not change
573+
# whether the target runs. The probes below drive the recursion limit and
574+
# then make the specialized call from the deepest live frame; the target
575+
# bumps the counter with plain bytecode, so reaching it cannot itself be
576+
# stopped by a second recursion check.
577+
578+
@requires_jit_disabled
579+
@requires_specialization
580+
def test_recursion_check_for_call_ex_py(self):
581+
hits = 0
582+
deepest_frame_claimed = False
583+
584+
def target(*args):
585+
nonlocal hits
586+
hits += 1
587+
return 42
588+
589+
def probe(at_limit):
590+
nonlocal deepest_frame_claimed
591+
while not at_limit:
592+
try:
593+
return probe(False)
594+
except RecursionError:
595+
# Only the deepest live frame makes the call. Outer frames
596+
# re-raise, so the count is not taken after a frame has
597+
# unwound and freed recursion budget.
598+
if deepest_frame_claimed:
599+
raise
600+
deepest_frame_claimed = True
601+
at_limit = True
602+
args = ()
603+
return target(*args)
604+
605+
for _ in range(_testinternalcapi.SPECIALIZATION_THRESHOLD):
606+
probe(True)
607+
self.assert_specialized(probe, "CALL_EX_PY")
608+
609+
hits_before = hits
610+
with infinite_recursion(50):
611+
with self.assertRaises(RecursionError):
612+
probe(False)
613+
self.assertEqual(hits, hits_before)
614+
615+
@requires_jit_disabled
616+
@requires_specialization
617+
def test_recursion_check_for_call_kw_bound_method(self):
618+
hits = 0
619+
deepest_frame_claimed = False
620+
621+
class Receiver:
622+
def target(self, *, value):
623+
nonlocal hits
624+
hits += 1
625+
return value
626+
627+
bound_target = Receiver().target
628+
629+
def probe(at_limit):
630+
nonlocal deepest_frame_claimed
631+
while not at_limit:
632+
try:
633+
return probe(False)
634+
except RecursionError:
635+
if deepest_frame_claimed:
636+
raise
637+
deepest_frame_claimed = True
638+
at_limit = True
639+
return bound_target(value=43)
640+
641+
for _ in range(_testinternalcapi.SPECIALIZATION_THRESHOLD):
642+
probe(True)
643+
self.assert_specialized(probe, "CALL_KW_BOUND_METHOD")
644+
645+
hits_before = hits
646+
with infinite_recursion(50):
647+
with self.assertRaises(RecursionError):
648+
probe(False)
649+
self.assertEqual(hits, hits_before)
650+
571651
def test_dont_specialize_custom_vectorcall(self):
572652
def f():
573653
raise Exception("no way")
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Fix the specialized ``CALL_EX_PY`` and ``CALL_KW_BOUND_METHOD`` instructions
2+
entering the callee at the recursion limit, where the generic
3+
``CALL_FUNCTION_EX`` and ``CALL_KW`` raise :exc:`RecursionError`. Both now
4+
check the remaining recursion depth before pushing the new frame, so warming
5+
a call site no longer changes whether the target is executed.

Modules/_testinternalcapi/test_cases.c.h

Lines changed: 16 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Python/bytecodes.c

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5537,6 +5537,7 @@ dummy_func(
55375537
_CHECK_METHOD_VERSION_KW +
55385538
_EXPAND_METHOD_KW +
55395539
flush + // so that self is in the argument array
5540+
_CHECK_RECURSION_REMAINING +
55405541
_PY_FRAME_KW +
55415542
_SAVE_RETURN_OFFSET +
55425543
_PUSH_FRAME;
@@ -5757,6 +5758,7 @@ dummy_func(
57575758
_CHECK_PEP_523 +
57585759
_MAKE_CALLARGS_A_TUPLE +
57595760
_CHECK_IS_PY_CALLABLE_EX +
5761+
_CHECK_RECURSION_REMAINING +
57605762
_PY_FRAME_EX +
57615763
_SAVE_RETURN_OFFSET +
57625764
_PUSH_FRAME;

Python/generated_cases.c.h

Lines changed: 16 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)