Tracking issue for new compiletest executor

[jieyouxu]

This is a tracking issue for migrating towards a new compiletest executor to replace the dependency on the
libtest executor.

This tracking issue is not meant for discussions or bug reports but is meant to track implementation progress, please open individual issues for bugs / discussions or zulip threads instead.

Summary

compiletest currently relies on the in-tree libtest (unstable programmatic API). This is problematic because:

1. This can cause unnecessary compiler rebuilds following the stage 0 std redesign (this is mitigated by bootstrap build.compiletest-use-stage0-libtest = true but that's more-or-less a hack / mitigation).
   • This workaround can additional cause friction for contributors modifying programmatic libtest API due to stage 0 vs in-tree libtest differences.
2. This is quite restrictive for compiletest because we can't easily tailor libtest behavior for rustc/rustdoc/ certain library test suites without regressing all the other libtest consumers.

Instead, we can roll our own minimal executor that replaces the libtest executor, which would allow us to drop the dependency on libtest once we migrate over to the new executor.

Known limitations

• The current new executor implementation still depends on an unstable library feature, namely #![feature(internal_output_capture)].

Known bugs

• Deadline-related bugs in new executor (compiletest: Use the new non-libtest executor by default #139998 (comment)). Fixed by compiletest: Fix deadline bugs in new executor #140031.

Implementation history

• compiletest: Add an experimental new executor to replace libtest #139660
• Call-for-testing on zulip: https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/Call.20for.20testing.3A.20New.20test.20executor.20for.20compiletest/with/512452105
• compiletest: Fix deadline bugs in new executor #140031
• compiletest: Use the new non-libtest executor by default #139998
• To be more safe (for post-merge test metrics double-checks), revert new executor to reland with a synthetic compiler/ tree change to inhibit download-rustc Revert compiletest new-executor, to re-land without download-rustc #140233
• Reland: compiletest: Re-land using the new non-libtest executor by default #140288
• Retire libtest executor: compiletest: Remove the libtest-based executor and its dependency #140392