Skip to content

SHA-256 digest capture and verification for all downloads (Provenance Phase 1) #1296

Description

@LalatenduMohanty

Summary

fromager downloads sdists and wheels without computing or verifying any cryptographic hash. A compromised upstream can replace artifact bytes between runs and fromager will accept them silently. This is Phase 1 of #1238.

Current state

  • download_url() streams bytes to disk with no hashlib computation
  • #sha256= URL fragments from PyPI indexes are carried in URL strings but never verified
  • pypi_simple.DistributionPackage.digests is available but never read
  • Candidate, DependencyNode, and build-order entries have no digest field

Proposed changes

  1. Compute SHA-256 during download — add streaming hash in download_url(), the single chokepoint for all HTTP downloads; return the digest alongside the file path
  2. Verify URL fragment hashes — extract #sha256= from download URLs and hard-fail on mismatch with computed digest
  3. Read index digests — use dp.digests from pypi_simple as a cross-check when creating Candidate objects
  4. Store digests in data structures — add optional digest field to Candidate, DependencyNode/DependencyNodeDict, and build-order entries (backward-compatible via NotRequired/defaults)
  5. Write provenance-index.json — aggregate per-package digests during finalize(), alongside build-order.json
  6. Re-download verification — when a digest is already recorded, verify new downloads produce identical bytes (trust-on-first-use)

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions