Using `@parametrize` on class or higher-scoped `@fixture`s makes scope ignored

[zhiltsov-max]

Hi, I think I've discovered a bug: if a class-scoped (or higher) @fixture is @parametrized and then requested via fixture_ref, the fixture is being called on each test function call, making the scope of the fixture is effectively ignored.

Reproducing example

Environment:
python 3.12,
pytest==9.0.3
pytest-cases==3.10.1

Details

"""
A class-scoped fixture that is *parametrized* is torn down and re-created for
every test in the class, as soon as it is used as an alternative of a fixture union
(i.e. referenced through ``fixture_ref`` in ``@parametrize``).

An otherwise identical fixture without ``@parametrize`` is correctly created once.

Run with::

    pytest -q test_class_scoped_parametrized_fixture_in_union.py

Expected: both fixtures are set up once per class.
Actual:   ``fxt_parametrized`` is set up once per test (3 times here).

Environment:
python 3.12,
pytest==9.0.3
pytest-cases==3.10.1

Also reproduces on pytest_cases==3.8.6, pytest==6.2.5, python 3.10
"""

import pytest
from pytest_cases import fixture, fixture_ref, parametrize

SETUP_CALLS = []


class Base:
    @fixture(scope="class")
    def fxt_plain(self):
        SETUP_CALLS.append("fxt_plain")
        return "fxt_plain"

    @fixture(scope="class")
    @parametrize("value", [1])
    def fxt_parametrized(self, value):
        SETUP_CALLS.append(f"fxt_parametrized[{value}]")
        return f"fxt_parametrized[{value}]"

    _cases = [fixture_ref("fxt_plain"), fixture_ref("fxt_parametrized")]


class TestUnion(Base):
    @parametrize("case", Base._cases)
    def test_1(self, case):
        pass

    @parametrize("case", Base._cases)
    def test_2(self, case):
        pass

    @parametrize("case", Base._cases)
    def test_3(self, case):
        pass


@pytest.fixture(scope="module", autouse=True)
def report_setup_counts():
    yield

    counts = {name: SETUP_CALLS.count(name) for name in sorted(set(SETUP_CALLS))}
    print(f"\nfixture setup calls: {counts}")

    assert counts == {
        "fxt_plain": 1,
        "fxt_parametrized[1]": 1,
    }, "class-scoped fixtures must be set up once per class"

Possible causes

Test nodes that do not select a given alternative have it deactivated via a dummy NOT_USED parameter (plugin.py#L1149-L1153). pytest keys the fixture cache on the parameter value (fixtures.py#L1119-L1120), so every switch between the real value and NOT_USED is a cache miss: pytest finalizes the cached value and re-runs setup. The declared scope no longer controls the fixture's lifetime.

Non-parametrized fixtures are unaffected.

This is the same defect as #120, which was fixed for non-parametrized session- and module-scoped fixtures in 2.1.0 by restricting the dummy-parameter hack to function-scoped fixtures (plugin.py#L1128). The branch that handles parametrized fixtures has no such scope guard. It also cannot simply reuse the same fix: an unparametrized fixture can have its dummy parameter omitted entirely, whereas a parametrized one must carry some value, and any value other than the real one is a different cache key.

Test nodes using the same union alternative (produced by @parametrize with fixture_ref) are not grouped either, so the alternatives interleave and the setup/teardown happens on nearly every test.

Known workarounds

Putting this in conftest.py allows pytest group tests how it's supposed to do:

@pytest.hookimpl(tryfirst=True)
def pytest_collection_modifyitems(items):
    for item in items:
        callspec = getattr(item, "callspec", None)
        if callspec is None:
            continue
        for argname, value in callspec.params.items():
            if value is NOT_USED:
                callspec.indices[argname] = -1 # put a fake index that cannot be confused with real ones

pytest-cases already overrides indices this way in plugin.py#L1167 for non-parametrized fixtures. The workaround is only a partial fix to the problem, because it only establishes grouping to reduce the number of fixture changes. If the order of tests is modified after this, or it's not possible to make such a grouping without interleaving, the fix stops working.

Other possible workarounds, less convenient, are:

• maintaining your own cache, making the function-scoped fixtures only requesting from it
• dropping parametrize