Skip to content

data_sources.md

Latest commit

 

History

History
41 lines (31 loc) · 2.06 KB

data_sources.md

File metadata and controls

41 lines (31 loc) · 2.06 KB

Data Sources — USDA Rural Opportunity Navigator

Version: 0.1
Last Updated: 2025-10-11 13:28

Source Name URL Description File Type Est. Raw Size Fields Used License / Notes
USDA Rural Development Data Gateway https://data.rd.usda.gov/ Rural dev funding programs CSV / API ~150 MB state, county, program_name, funding_type, intent_category Public Domain
USDA FSA Farm Loan Dataset https://data.nal.usda.gov/dataset/farm-loan-programs Loans by county CSV ~100 MB state, county, program_name, industry Public Domain
Grants.gov Open Data API https://www.grants.gov/services/open-data.html Federal grant metadata JSON / API ~50 MB title, agency, deadline Public API Key Req.
U.S. Census ACS 5-Year https://www.census.gov/data.html Median income by county CSV ~200 MB state, county, median_income CC-BY
USDA ERS Rural-Urban Continuum Codes https://www.ers.usda.gov/data-products/rural-urban-continuum-codes/ Rural/urban classification CSV ~20 MB county_fips, rucc_code Public Domain
BEA Industry Output Summary https://apps.bea.gov/regional/downloadzip.cfm Industry output by state CSV ~50 MB state, industry, output_value Public Domain

Total Raw Size: ≈ 570 MB Target Cleaned Size: ≤ 0.4 GB

Aggregation / Filtering Rules

  1. Keep latest 5 years (2019–2023).
  2. Aggregate to county (FIPS).
  3. ≤ 30 columns per file.
  4. Store as .parquet (Snappy).

Update Procedure

  1. Download raw files → /data/raw/ (local only).
  2. Record download timestamp & hash here.
  3. Run scripts/etl_usda.py --update.
  4. Validate & export → /data-samples/.
  5. Push docs & samples (≤ 50 MB each).

Revision History

Version Timestamp (CT) Author Summary
0.1 2025-10-11 13:45 CT Victor Olatunji Initial data source registry