added validation_config.json WorldDevelopmentIndicators#1996
added validation_config.json WorldDevelopmentIndicators#1996niveditasing wants to merge 34 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a new validation configuration file, validation_config.json, for the World Bank WDI dataset. It defines rules for checking deleted record counts and performing golden file comparisons. The review feedback highlights path inconsistencies for golden files and a missing directory prefix for input files, which could prevent the validation tool from locating the necessary data.
|
@gemini-code-assist review |
There was a problem hiding this comment.
Code Review
This pull request introduces import validation for the World Bank WDI dataset by adding a validation configuration, updating the manifest, and providing golden data files. Feedback includes a missing file reference for 'golden_WorldBank.csv', a path mismatch for the summary report, and a suggestion to move golden files to a local directory for better self-containment. Additionally, the documentation header in the README should be generalized to describe the process rather than the PR, and the threshold for the deleted records percentage may need adjustment to reflect the intended strictness.
|
LGTM |
91e5e72 to
2b1a9b8
Compare
Implemented golden checks and made a small increase to the threshold.
Tested results : https://storage.mtls.cloud.google.com/datcom-import-test/scripts/world_bank/wdi/WorldDevelopmentIndicators/2026_05_21T05_12_39_308498_07_00/input0/validation/validation_output.csv