Mark top-level Makefile .NOTPARALLEL to fix make -j build race#79
Open
darrylabbate wants to merge 1 commit into
Open
Mark top-level Makefile .NOTPARALLEL to fix make -j build race#79darrylabbate wants to merge 1 commit into
darrylabbate wants to merge 1 commit into
Conversation
The IMB-* targets each recurse into src_cpp/ and compile a shared set of
intermediate objects (imb.o, benchmark_suites_collection.o, args_parser.o,
scope.o, libyaml-cpp.a) to identical paths. Building two or more suites
concurrently under 'make -j' lets multiple sub-makes write those same
object files simultaneously, clobbering them and intermittently breaking
the link, e.g.:
benchmark_suites_collection.h: undefined reference to
`BenchmarkSuitesCollection::pnames'
imb.o: file not recognized: File truncated
The suite recipes invoke a plain 'make' (not $(MAKE) and not prefixed
with '+'), so they are not recognized as recursive. With GNU Make < 4.4's
pipe-based jobserver such sub-makes were cut off from the job pool and ran
serially, which masked the race. GNU Make >= 4.4's FIFO jobserver connects
non-recursive sub-makes to the parallel pool, so the suite builds now run
concurrently and the race surfaces routinely (observed on Ubuntu 26.04 /
RHEL 10, which ship make 4.4).
Add .NOTPARALLEL so the top-level suite builds are serialized regardless of
-j. Parallelism within each suite sub-make is unaffected, and the suites
build in seconds, so the wall-clock impact is negligible.
Signed-off-by: Darryl Abbate <drl@amazon.com>
charlesstoll
approved these changes
Jun 23, 2026
Author
|
Refactoring s.t. the Makefile emits per-suite objects is probably the better long-term solution (allows parallelizing), but declaring |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The IMB-* targets each recurse into src_cpp/ and compile a shared set of intermediate objects (imb.o, benchmark_suites_collection.o, args_parser.o, scope.o, libyaml-cpp.a) to identical paths. Building two or more suites concurrently under 'make -j' lets multiple sub-makes write those same object files simultaneously, clobbering them and intermittently breaking the link, e.g.:
The suite recipes invoke a plain 'make' (not $(MAKE) and not prefixed with '+'), so they are not recognized as recursive. With GNU Make < 4.4's pipe-based jobserver such sub-makes were cut off from the job pool and ran serially, which masked the race. GNU Make >= 4.4's FIFO jobserver connects non-recursive sub-makes to the parallel pool, so the suite builds now run concurrently and the race surfaces routinely (observed on Ubuntu 26.04 / RHEL 10, which ship make 4.4).
Add .NOTPARALLEL so the top-level suite builds are serialized regardless of -j. Parallelism within each suite sub-make is unaffected, and the suites build in seconds, so the wall-clock impact is negligible.