Ignore all `#[test]`-related code automatically

[Swatinem]

UPDATE3(taiki-e):

If you want to ignore all #[test]-related code, you can use module-level #[coverage(off)] attribute:

#![cfg_attr(coverage_nightly, feature(coverage_attribute))]

#[cfg(test)]
#[cfg_attr(coverage_nightly, coverage(off))]
mod tests {
    // ...
}

cargo-llvm-cov excludes code contained in the directory named tests and file named tests.rs/*_tests.rs/*-tests.rs from the report by default, so you can also use it instead of #[coverage(off)] attribute.

#[coverage(off)] attribute has been stabilized in rust-lang/rust#130766 (will be included in Rust 1.85). UPDATE: stabilization has been reverted in rust-lang/rust#134672.

---

UPDATE2(taiki-e):

See #123 (comment)

---

UPDATE(taiki-e):

If you want to ignore all #[test]-related code, consider using coverage-helper crate version 0.2+.

cargo-llvm-cov excludes code contained in the directory named tests from the report by default, so you can also use it instead of coverage-helper crate.

https://github.com/taiki-e/cargo-llvm-cov#exclude-function-from-coverage

---

AFAIK this is not currently possible (though I would love to be proven wrong), so just filing this as a nice-to-have tracking issue.

What I would like is that only library code counts towards coverage. Having all of the #[test] and #[cfg(test)] code (fn and whole utility modules) count towards coverage metrics right now skews those numbers up quite a bit.

Right now cargo-llvm-cov automatically ignores all the examples and tests code via -ignore-filename-regex but unit test functions are still counted (and possibly doctests once those are being fixed [tentative fix in #122]).

[taiki-e]

The idea that probably works here is a helper attribute that generates #[cfg_attr(coverage, no_coverage)].

#[cfg(coverage)]
use coverage_helper::test;

#[test]
fn test() {
}

generated:

#[cfg(coverage)]
use coverage_helper::test;

#[cfg_attr(coverage, no_coverage)]
#[::core::prelude::v1::test]
fn test() {
}

UPDATE: this idea is now implemented as https://github.com/taiki-e/coverage-helper.

[scouten-adobe]

@taiki-e I'm not an expert on attribute macros. Would you (or anyone) be kind enough to translate this into code that we could implement?

Adding the no_coverage attribute in front of every test seems unsustainable.

[taiki-e]

Above my approach is now implemented as https://github.com/taiki-e/coverage-helper.

[scouten-adobe]

I appreciate the coverage-helper crate, but since the #[no_coverage] attribute is not yet stabilized, I've decided not to use this approach in my project.

For those following along, if you want to exclude test-related code from coverage and base your CI builds on stable Rust versions, there's a more drastic approach that does seem to solve the problem. See adobe/xmp-toolkit-rs#66.