Skip to content

Fix relative PRE_TEST discovery paths - #3175

Open
NgoQuocViet2001 wants to merge 1 commit into
catchorg:develfrom
NgoQuocViet2001:fix/3051-relative-pre-test-paths
Open

Fix relative PRE_TEST discovery paths#3175
NgoQuocViet2001 wants to merge 1 commit into
catchorg:develfrom
NgoQuocViet2001:fix/3051-relative-pre-test-paths

Conversation

@NgoQuocViet2001

@NgoQuocViet2001 NgoQuocViet2001 commented Jul 11, 2026

Copy link
Copy Markdown

Description

Add an opt-in USE_RELATIVE_PATHS mode to catch_discover_tests for DISCOVERY_MODE PRE_TEST.

When enabled, the generated CTest include resolves the test executable, discovery script, working directory, and discovery cache relative to its own relocated build-tree location. This keeps the existing behavior unchanged by default, supports single- and multi-config generators, and requires CMake 3.24 for the path generator expressions. The option and version requirement are documented.

The regression relocates the build tree before first discovery, uses a generator-expression WORKING_DIRECTORY, verifies the discovered names, runs the actual tests, and cleans the generated discovery cache.

GitHub Issues

Closes #3051

Testing

  • Single-config Ninja relocation before first discovery: 5/5 tests passed
  • Multi-config Ninja/Zig in a path containing spaces: default and relocated relative modes both discovered 5 tests; relocated CTest passed
  • CMake 3.23.3: default mode configured successfully; opt-in mode failed with the expected CMake 3.24 requirement
  • Python syntax and git diff --check passed

@codecov

codecov Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.16%. Comparing base (499442d) to head (899f969).

Additional details and impacted files
@@           Coverage Diff           @@
##            devel    #3175   +/-   ##
=======================================
  Coverage   91.16%   91.16%           
=======================================
  Files         206      206           
  Lines        9016     9016           
=======================================
  Hits         8219     8219           
  Misses        797      797           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@NgoQuocViet2001
NgoQuocViet2001 force-pushed the fix/3051-relative-pre-test-paths branch from 77c8a2f to c15c563 Compare August 11, 2026 09:50
@NgoQuocViet2001

Copy link
Copy Markdown
Author

Rebased onto current devel and resolved the upstream discovery-test changes while preserving the USE_RELATIVE_PATHS regression.

Focused Windows/MSVC validation passed for both the default and relocated relative modes: 22 test names matched the CTest listing, 22 matched TEST_LIST, and the relocated tests ran successfully. AppVeyor is also green across all 5 jobs.

@NgoQuocViet2001
NgoQuocViet2001 force-pushed the fix/3051-relative-pre-test-paths branch from c15c563 to 899f969 Compare August 24, 2026 04:30
@NgoQuocViet2001

Copy link
Copy Markdown
Author

Rebased onto current devel — the branch had gone conflicting again and now applies cleanly.

The conflict, and why it was not a mechanical resolve. devel has since added TEST_PREFIX " prefix " / TEST_SUFFIX " suffix " to the discovery test, with VerifyRegistration.py asserting TEST_LIST against CTEST_NAME_PREFIX + name + CTEST_NAME_SUFFIX. This PR adds a second registration — the relative-registration/ subdirectory used for the relocation case — which had no prefix or suffix. Taking either side alone breaks it:

  • keep only mine → the main case loses the prefix/suffix coverage devel just added;
  • keep only devel's → the relative case registers plain names while expected_names demands decorated ones, so TEST_LIST mismatches.

So the resolution puts the same TEST_PREFIX / TEST_SUFFIX on the relative registration, which also makes the relocation case exercise the same discovery options as the main one rather than a reduced variant. expected_names then stays unconditional.

Verified, not assumed. Built and ran the real script under CMake 3.28.3 / g++ 13.3 (the relative branch needs ≥ 3.24):

$ python3 tests/TestScripts/DiscoverTests/VerifyRegistration.py <src> <out>
22 tests matched in CTest listing
22 tests matched in TEST_LIST variable      <- normal build
22 tests matched in CTest listing
22 tests matched in TEST_LIST variable      <- relocated, USE_RELATIVE_PATHS

I also checked the resolution was necessary rather than incidental — dropping the two lines back off the relative registration reproduces the failure:

  Catch2 name ' prefix Some test suffix ' not in TEST_LIST
  Catch2 name ' prefix [ suffix ' not in TEST_LIST
  ... (10 more)

The USE_RELATIVE_PATHS regression this PR exists for is intact, and the final diff is 5 files / +188 / −42 against devel.

No maintainer has looked at this yet and I know the queue is long — no rush from me. Flagging only that it is current and green again, so it is not blocked on being stale. Happy to close it if the approach is not one you want.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

catch_discover_tests needs to support relative paths

1 participant