S1201 schema fix and data refresh

[balit-raibot]

This PR includes changes:

• fixing download script, processing script to refresh data without getting into casting issue, library issues.
• fixed paths from utils such as generate_col_map.py

[gemini-code-assist]

Code Review

This pull request updates the US Census ACS 5-year subject tables scripts and API utilities. Key changes include adding the aiolimiter dependency, clearing overwrite_dcids in the S1201 specification, dynamically importing common column mapping functions to avoid circular imports, ensuring output directories are created, and converting relative imports to absolute imports across the API utilities. Feedback on these changes suggests adding defensive guard checks for dynamic module loading, simplifying directory creation with exist_ok=True, and appending module_dir_ to sys.path to prevent potential ModuleNotFoundError issues when importing the modified API utility modules from other directories.

[balit-raibot]

@gemini-code-assists review

[gemini-code-assist]

Code Review

This pull request updates US Census data processing scripts, particularly for the ACS 5-year subject tables (S1201) and API utilities. Key changes include updating statistical variable names to specify age ranges (e.g., adding "15OrMoreYears"), ignoring the "SUMMARY INDICATOR" column in the S1201 specification, casting dataframes to object types before replacing annotations, and refactoring relative imports. The reviewer feedback focuses on improving import robustness by using absolute imports relative to the repository root across multiple files instead of dynamically loading modules, modifying sys.path, or using sibling-level imports. Additionally, the reviewer recommends using a with statement for safe file handling when writing the column map JSON in process.py.

[balit-raibot]

@gemini-code-assist review

[gemini-code-assist]

Code Review

This pull request refactors US Census import scripts and configurations, primarily transitioning relative imports to absolute package imports across api_utils and s1201 modules. It also updates the s1201 specification, test data, and MCF files to use 15OrMoreYears statistical variables instead of overwriting them, adds aiolimiter to requirements, and improves file-writing safety in process.py. Review feedback recommends avoiding brittle relative path traversals (e.g., ../../../../..) when appending to sys.path by dynamically searching for repository markers like .git or WORKSPACE. Additionally, it suggests using absolute imports in process.py to prevent potential namespace collisions.

[balit-raibot]

@gemini-code-assist review

[gemini-code-assist]

Code Review

This pull request updates the US Census ACS 5-year subject tables (specifically S1201) and API utilities. Key changes include migrating relative imports to absolute imports, implementing a dynamic project root discovery pattern for Python path resolution, utilizing context managers for file operations, and updating several Statistical Variable DCIDs to explicitly include the "15OrMoreYears" age constraint in both test data and MCF files. Additionally, a pandas type-casting fix was introduced in the API downloader. Feedback suggests applying the robust project root discovery pattern to generate_col_map.py to replace a brittle relative path calculation.

[balit-raibot]

@gemini-code-assist review

[gemini-code-assist]

Code Review

This pull request updates several US Census scripts to use absolute imports instead of relative imports, adds aiolimiter to the requirements, and refactors directory creation and file writing in process.py. It also updates the statistical variable names in the S1201 test data and MCF files to explicitly include the age range (e.g., 15OrMoreYears). The feedback highlights a redundant os.makedirs call in process.py and recommends replacing wildcard imports with explicit imports in the test files to align with PEP 8 guidelines.

[balit-raibot]

@gemini-code-assist review

[gemini-code-assist]

Code Review

This pull request updates several files in the scripts/us_census directory to improve module import reliability, standardizes column naming conventions in S1201_spec.json and S1201_cleaned.csv, and includes minor code improvements in process.py and census_api_data_downloader.py. The review feedback identified potential runtime errors related to directory existence and missing columns, providing actionable suggestions to improve robustness.

[vish-cs]

Are we still doing it? Don't see it in the files changed

removed SVs override section in the spec.json to avoid schema issues such as those found in this:

[balit-raibot]

Are we still doing it? Don't see it in the files changed

removed SVs override section in the spec.json to avoid schema issues such as those found in this:

I had undid those changes after Ajai mentioned the potential conflict with NL stat vars, and thus the earlier changes have been removed from this PR.

The PR description has been modified