can coverage be made faster somehow?

[ThomasWaldmann]

Our tests take quite some time and quite a significant amount is due to coverage (see #9463).

[AmauMaill]

FYI:

An approximate of 733 tests in the testsuite:

grep -r "def test" src/borg/testsuite/. | wc -l
733

On my machine (linux laptop) top 200 of longest tests are in archiver/, see file for more details:

top_200.txt

Refactoring archiver tests fixtures to scope="module" could reduce execution time I guess. However, this requires to handle unique IDs for parallel tests. One concern is whether sharing fixtures between tests introduces a risk of test result falsification?

This needs more investigating to properly understand the ins and outs. Just my 2 cents, I'll keep looking into it.

[ThomasWaldmann]

@AmauMaill Thanks for looking into that.

What you did seems to rather fit into #9324 than #9470 though, the latter being specifically about the influence of coverage tracking and not about slower vs. faster tests in general.

[AmauMaill]

@ThomasWaldmann thanks for your reply,

It seems to me that patch = ["subprocess", "_exit"] in pyproject.toml causes coverage.py to inject COVERAGE_PROCESS_CONFIG into every subprocess environment. For remote_archiver tests, each cmd() call
spawns a fresh borg serve subprocess which inherits this variable, triggering full coverage tracing (Python startup + C-tracer init + .coverage.* write on exit). This generates small overhead per subprocess, for no gain since RepositoryServer is already # pragma: no cover and all other server-side paths are covered by local in-process tests.

I tried this fix: strip COVERAGE_PROCESS_CONFIG and COVERAGE_PROCESS_START from the subprocess environment in prepare_subprocess_env(system=False) (src/borg/helpers/process.py), which is the central point all borg-internal subprocess spawns flow through.

Test suite runs in 19 minutes with 0 failures, down ~35% on the previously slowest tests, and coverage report is unchanged at 79.2%.

Could this be a viable solution regarding coverage tracking?