systemtests: archive and compare iterations.log for implicit coupling #755
Open
AdityaGupta716 wants to merge 3 commits intoprecice:developfrom
Open
systemtests: archive and compare iterations.log for implicit coupling #755AdityaGupta716 wants to merge 3 commits intoprecice:developfrom
AdityaGupta716 wants to merge 3 commits intoprecice:developfrom
Conversation
Author
|
hi @MakisH plz review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Archives
precice-*-iterations.logfiles from case directories intoiterations-logs/on every run, and compares them by SHA-256 hash againsta reference sidecar (
.iterations-hashes.json) to catch implicit-couplingregressions.
What this does
_collect_iterations_logs()finds allprecice-*-iterations.logfiles across each case directory__archive_iterations_logs()copies them intoiterations-logs/, prefixing with the case name when multiple cases are present to avoid filename collisions__compare_iterations_hashes()reads the reference sidecar and fails fast on any hash mismatch, missing log, or unexpected extra logrun()so regressions are caught early with a clear messagerun_for_reference_results()writes the.iterations-hashes.jsonsidecar automatically alongside the reference tar.gz at reference generation timeBackward compatibility
If no
.iterations-hashes.jsonsidecar exists for a test, the hash check is skipped and the test runs as before. Existing tests are unaffected until reference results are regenerated.Closes #440