tidy compiles very slowly

[matthiaskrgr]

1. modify some code of tidy
2. run ./x.py test
   =>
   for the first ~6 seconds, tidy is being built in parallel, but for the remaining ~1 minute and 9 seconds llvm is doing some sequential work.

I'm not sure what takes so long, there must be one epic cgu that it is processing or something?

[onur-ozkan]

I wanted to check the problem but couldn't reproduce it locally.

[matthiaskrgr]

   Compiling tidy v0.1.0 (/home/matthias/vcs/github/rust/src/tools/tidy)
time:   0.000; rss:   44MB ->   45MB (   +1MB)	parse_crate
time:   0.000; rss:   45MB ->   46MB (   +1MB)	setup_global_ctxt
time:   0.000; rss:   47MB ->   47MB (   +0MB)	crate_injection
time:   0.019; rss:   48MB ->   84MB (  +37MB)	expand_crate
time:   0.019; rss:   48MB ->   84MB (  +37MB)	macro_expand_crate
time:   0.000; rss:   84MB ->   84MB (   +0MB)	AST_validation
time:   0.000; rss:   85MB ->   85MB (   +0MB)	finalize_macro_resolutions
time:   0.004; rss:   85MB ->   88MB (   +4MB)	late_resolve_crate
time:   0.000; rss:   88MB ->   89MB (   +0MB)	resolve_check_unused
time:   0.000; rss:   89MB ->   89MB (   +0MB)	resolve_postprocess
time:   0.005; rss:   84MB ->   89MB (   +4MB)	resolve_crate
time:   0.001; rss:   96MB ->   96MB (   +0MB)	drop_ast
time:   0.011; rss:   89MB ->   96MB (   +7MB)	looking_for_derive_registrar
time:   0.013; rss:   89MB ->   97MB (   +9MB)	misc_checking_1
time:   0.023; rss:   97MB ->  129MB (  +32MB)	coherence_checking
time:   0.139; rss:   97MB ->  158MB (  +61MB)	type_check_crate
time:   0.143; rss:  158MB ->  201MB (  +43MB)	MIR_borrow_checking
time:   0.026; rss:  201MB ->  201MB (   +0MB)	MIR_effect_checking
time:   0.011; rss:  202MB ->  202MB (   +1MB)	module_lints
time:   0.011; rss:  202MB ->  202MB (   +1MB)	lint_checking
time:   0.004; rss:  202MB ->  202MB (   +0MB)	privacy_checking_modules
time:   0.016; rss:  201MB ->  202MB (   +1MB)	misc_checking_3
time:   0.091; rss:  202MB ->  214MB (  +12MB)	generate_crate_metadata
time:   0.084; rss:  214MB ->  220MB (   +5MB)	monomorphization_collector_graph_walk
time:   0.021; rss:  220MB ->  225MB (   +5MB)	partition_and_assert_distinct_symbols
time:   0.972; rss:  225MB ->  409MB ( +184MB)	codegen_to_LLVM_IR
time:   1.080; rss:  214MB ->  409MB ( +194MB)	codegen_crate
time:   0.016; rss:  409MB ->  355MB (  -54MB)	free_global_ctxt
time:  52.535; rss:  249MB ->  532MB ( +283MB)	LLVM_passes
time:   0.011; rss:  523MB ->  216MB ( -307MB)	join_worker_thread
time:  51.617; rss:  355MB ->  216MB ( -139MB)	finish_ongoing_codegen
time:   0.270; rss:  216MB ->  217MB (   +1MB)	link_rlib
time:   0.074; rss:  217MB ->  217MB (   +0MB)	link_binary_remove_temps
time:   0.344; rss:  216MB ->  217MB (   +1MB)	link_binary
time:   0.344; rss:  216MB ->  216MB (   +0MB)	link_crate
time:  51.961; rss:  355MB ->  216MB ( -139MB)	link
time:  53.517; rss:   32MB ->  186MB ( +154MB)	total
time:   0.000; rss:   44MB ->   45MB (   +1MB)	parse_crate
time:   0.000; rss:   46MB ->   46MB (   +1MB)	setup_global_ctxt
time:   0.000; rss:   47MB ->   48MB (   +0MB)	crate_injection
time:   0.009; rss:   48MB ->   76MB (  +28MB)	expand_crate
time:   0.000; rss:   76MB ->   76MB (   +0MB)	check_unused_macros
time:   0.009; rss:   48MB ->   76MB (  +28MB)	macro_expand_crate
time:   0.000; rss:   76MB ->   77MB (   +0MB)	AST_validation
time:   0.000; rss:   77MB ->   77MB (   +0MB)	finalize_imports
time:   0.002; rss:   77MB ->   79MB (   +2MB)	late_resolve_crate
time:   0.000; rss:   79MB ->   79MB (   +0MB)	resolve_check_unused
time:   0.000; rss:   79MB ->   80MB (   +0MB)	resolve_postprocess
time:   0.002; rss:   77MB ->   80MB (   +3MB)	resolve_crate
time:   0.000; rss:   80MB ->   80MB (   +0MB)	complete_gated_feature_checking
time:   0.001; rss:   80MB ->   82MB (   +2MB)	looking_for_entry_point
time:   0.002; rss:   80MB ->   83MB (   +3MB)	misc_checking_1
time:   0.001; rss:   83MB ->   87MB (   +4MB)	coherence_checking
time:   0.030; rss:   83MB ->  111MB (  +29MB)	type_check_crate
time:   0.023; rss:  111MB ->  123MB (  +12MB)	MIR_borrow_checking
time:   0.004; rss:  123MB ->  124MB (   +1MB)	MIR_effect_checking
time:   0.002; rss:  124MB ->  126MB (   +2MB)	module_lints
time:   0.003; rss:  124MB ->  126MB (   +2MB)	lint_checking
time:   0.003; rss:  124MB ->  126MB (   +2MB)	misc_checking_3
time:   0.061; rss:  126MB ->  153MB (  +27MB)	monomorphization_collector_graph_walk
time:   0.010; rss:  153MB ->  155MB (   +2MB)	partition_and_assert_distinct_symbols
time:   0.000; rss:  156MB ->  159MB (   +3MB)	write_allocator_module
time:   0.437; rss:  165MB ->  288MB ( +122MB)	codegen_to_LLVM_IR
time:   0.513; rss:  126MB ->  288MB ( +161MB)	codegen_crate
time:   0.012; rss:  288MB ->  238MB (  -50MB)	free_global_ctxt
time:   1.769; rss:  189MB ->  273MB (  +85MB)	LLVM_passes
time:   0.001; rss:  228MB ->  225MB (   -2MB)	join_worker_thread
time:   1.367; rss:  238MB ->  226MB (  -12MB)	finish_ongoing_codegen
time:   0.879; rss:  226MB ->  226MB (   +0MB)	run_linker
time:   0.884; rss:  226MB ->  226MB (   +1MB)	link_binary
time:   0.884; rss:  226MB ->  226MB (   +1MB)	link_crate
time:   2.251; rss:  238MB ->  226MB (  -12MB)	link
time:   2.859; rss:   32MB ->  152MB ( +120MB)	total
    Finished `release` profile [optimized + debuginfo] target(s) in 56.66s

[matthiaskrgr]

🤔

[matthiaskrgr]

aha lol!
it boils down to issues.txt, an array of ~4K static &strs which is being inlined into the code it seems.
When I cut the file down to just a handful of items, compilation time reduces to 7 seconds 🙃

rust/src/tools/tidy/src/ui_tests.rs

Line 108 in 3d5528c

let issues_list =

cc @workingjubilee I guess?

[onur-ozkan]

Can you check if #123339 reduces this slowness?

[matthiaskrgr]

It does indeed.
Build completed successfully in 0:00:06

[workingjubilee]

Ah, but if people give the tests meaningful names, it will reduce the build time. 😌