chore: consolidate npm package state

[epipav]

Note

Medium Risk
Schema migration plus behavior changes to which packages are enriched (critical-only vs universe-wide 30d), affecting worker throughput and data coverage after deploy.

Overview
Consolidates npm worker watermarks by merging npm_package_universe_state into npm_package_state (new 30d columns, row copy, drop old table) and retargeting all 30d DAL/worker code to the unified table with renamed selectors (getNpmPurlsDueForLatest30d / getNpmPurlsDueForLast30dHistory).

Re-scopes enrichment work: metadata ingest, daily-download backfill, and _changes refresh now only consider is_critical npm packages; 30d latest-window and history backfill stay on all npm packages (history backfill drops the previous critical-only filter). Unscanned metadata selection switches from “no state row” to metadata_last_run_at IS NULL.

^{Reviewed by Cursor Bugbot for commit 0db19e4. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

• feat: add user authentication (CM-123)
• feat: add user authentication (IN-123)

Projects:

• CM: Community Data Platform
• IN: Insights

Please add a Jira issue key to your PR title.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit bf26e78. Configure here.}

[Copilot]

Pull request overview

This PR consolidates npm package “universe” state into the shared npm_package_state table, and narrows metadata + daily downloads enrichment to packages.is_critical = TRUE.

Changes:

• Move last-30d (breadth/depth) watermarks from npm_package_universe_state into npm_package_state (code + migration).
• Restrict metadata ingest candidate selection and change-feed selection to is_critical npm packages.
• Restrict daily downloads backfill candidate selection to is_critical npm packages.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
services/libs/data-access-layer/src/packages/npmPackageState.ts	Adds is_critical filtering for metadata enrichment; current “unscanned” detection needs adjustment after state consolidation.
services/libs/data-access-layer/src/packages/downloadsLast30d.ts	Switches last-30d state reads/writes to npm_package_state and renames exported selectors accordingly.
services/libs/data-access-layer/src/packages/downloadsDaily.ts	Restricts daily-downloads backfill selection to packages.is_critical = TRUE.
services/apps/packages_worker/src/npm/activities.ts	Updates imports/usages to the renamed last-30d selector functions.
backend/src/osspckgs/migrations/V1781020800__merge_30d_into_npm_package_state.sql	Adds last-30d columns + indexes to npm_package_state, migrates data from npm_package_universe_state, then drops the old table.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[mbani01]

LGTM