[Bug] `prune_dependency_tree`'s behaviour conflicts with the latest improvements in `getfixtureclosure` (pytest 9.x)

[smarie]

Hi all,
I recently came across the changes of getfixtureclosure in #13789, that make the fixture closure better handle situations where a fixture is overridden. These changes are great and seem to work fine, thanks! They triggered an issue in smarie/python-pytest-cases#374 but a contributor provided an initial solution, and I upgraded it based on the latest changes in #13789.

I was about to release the fix but before releasing I created another variant of test, that unfortunately made the pytest engine fail - this is not related with pytest cases as it is a simple variant of one of the tests in pytest codebase.

Basically the test is identical to

pytest/testing/python/fixtures.py

Lines 5144 to 5180 in 1faee72

	def test_fixture_closure_with_overrides(pytester: Pytester) -> None:
	"""Test that an item's static fixture closure properly includes transitive
	dependencies through overridden fixtures (#13773)."""
	pytester.makeconftest(
	"""
	import pytest
	@pytest.fixture
	def db(): pass
	@pytest.fixture
	def app(db): pass
	"""
	)
	pytester.makepyfile(
	"""
	import pytest
	# Overrides conftest-level `app` and requests it.
	@pytest.fixture
	def app(app): pass
	class TestClass:
	# Overrides module-level `app` and requests it.
	@pytest.fixture
	def app(self, app): pass
	def test_something(self, request, app):
	# Both dynamic and static fixture closures should include 'db'.
	assert 'db' in request.fixturenames
	assert 'db' in request.node.fixturenames
	# No dynamic dependencies, should be equal.
	assert set(request.fixturenames) == set(request.node.fixturenames)
	"""
	)
	result = pytester.runpytest("-v")
	result.assert_outcomes(passed=1)

except that the test_something is parametrized.

def test_fixture_closure_with_overrides_and_intermediary(pytester: Pytester) -> None:
    """Test that an item's static fixture closure properly includes transitive
    dependencies through overridden fixtures (#13773).

    A more complicated case than test_fixture_closure_with_overrides, adds an
    intermediary so the override chain is not direct.
    """
    pytester.makeconftest(
        """
        import pytest

        @pytest.fixture
        def db(): pass

        @pytest.fixture
        def app(db): pass

        @pytest.fixture
        def intermediate(app): pass
        """
    )
    pytester.makepyfile(
        """
        import pytest

        # Overrides conftest-level `app` and requests it.
        @pytest.fixture
        def app(intermediate): pass

        class TestClass:
            # Overrides module-level `app` and requests it.
            @pytest.fixture
            def app(self, app): pass

            @pytest.mark.parametrize("a", [1])  # <--- this is the modification
            def test_something(self, request, app, a):
                # Both dynamic and static fixture closures should include 'db'.
                assert 'db' in request.fixturenames
                assert 'db' in request.node.fixturenames
                # No dynamic dependencies, should be equal.
                assert set(request.fixturenames) == set(request.node.fixturenames)
        """
    )
    result = pytester.runpytest("-v")
    result.assert_outcomes(passed=1)

The log is

============================= test session starts =============================
platform win32 -- Python 3.14.0b1, pytest-9.0.2, pluggy-1.6.0 -- C:\<...>\.nox\tests-3-14-env-pytest-latest\Scripts\python.EXE
cachedir: .pytest_cache
rootdir: C:\<...>\AppData\Local\Temp\pytest-of-USER\pytest-27\test_fixture_closure_with_overrides_and_intermediary0
...
collecting ... collected 1 item

test_fixture_closure_with_overrides_and_intermediary.py::TestClass::test_something[1] FAILED [100%]

================================== FAILURES ===================================
_________________________ TestClass.test_something[1] _________________________

self = <test_fixture_closure_with_overrides_and_intermediary.TestClass object at 0x0000018C98CB0190>
request = <FixtureRequest for <Function test_something[1]>>, app = None, a = 1

    @pytest.mark.parametrize("a", [1])
    def test_something(self, request, app, a):
        # Both dynamic and static fixture closures should include 'db'.
        assert 'db' in request.fixturenames
>       assert 'db' in request.node.fixturenames
E       AssertionError: assert 'db' in ['event_loop_policy', 'request', 'app', 'a']
E        +  where ['event_loop_policy', 'request', 'app', 'a'] = <Function test_something[1]>.fixturenames
E        +    where <Function test_something[1]> = <FixtureRequest for <Function test_something[1]>>.node

test_fixture_closure_with_overrides_and_intermediary.py:16: AssertionError
=========================== short test summary info ===========================
FAILED test_fixture_closure_with_overrides_and_intermediary.py::TestClass::test_something[1]
============================== 1 failed in 0.11s ==============================

The culprit is _genfunctions : when there is no parametrization, there is no attempt to prune the fixture closure. However when we add parametrization, it triggers fixtureinfo.prune_dependency_tree(), which in this example, will remove db from the closure as it is only requested by the original app fixture in the conftest, not by the app fixtures overriding it in the test file.

See

• pytest/src/_pytest/python.py

  Lines 478 to 487 in 1faee72

  	if not metafunc._calls:
  	yield Function.from_parent(self, name=name, fixtureinfo=fixtureinfo)
  	else:
  	metafunc._recompute_direct_params_indices()
  	# Direct parametrizations taking place in module/class-specific
  	# `metafunc.parametrize` calls may have shadowed some fixtures, so make sure
  	# we update what the function really needs a.k.a its fixture closure. Note that
  	# direct parametrizations using `@pytest.mark.parametrize` have already been considered
  	# into making the closure using `ignore_args` arg to `getfixtureclosure`.
  	fixtureinfo.prune_dependency_tree()

• pytest/src/_pytest/fixtures.py

  Lines 332 to 352 in 1faee72

  	def prune_dependency_tree(self) -> None:
  	"""Recompute names_closure from initialnames and name2fixturedefs.
  	Can only reduce names_closure, which means that the new closure will
  	always be a subset of the old one. The order is preserved.
  	This method is needed because direct parametrization may shadow some
  	of the fixtures that were included in the originally built dependency
  	tree. In this way the dependency tree can get pruned, and the closure
  	of argnames may get reduced.
  	"""
  	closure: set[str] = set()
  	working_set = set(self.initialnames)
  	while working_set:
  	argname = working_set.pop()
  	if argname not in closure and argname in self.names_closure:
  	closure.add(argname)
  	if argname in self.name2fixturedefs:
  	working_set.update(self.name2fixturedefs[argname][-1].argnames)
  	self.names_closure[:] = sorted(closure, key=self.names_closure.index)

This might be related with #11243