Skip to content

feat: operator script for bulk pipeline re-runs across score set cohorts #796

Description

@bencap

Summary

#758 requires re-running mapping and annotation pipelines over every previously-mapped. score set. The only entry point today handles one score set per invocation, so executing #758 means hand-driving hundreds of runs with no cohort selection, no throttling, and no record of what failed. Having this mapping and annotation pipeline run on every human target score set is essential for variant search and detail pages to work as intended on all variants in MaveDB.

We should add a dedicated operator script that selects a cohort, orders it to exploit the 24-hour ClinGen cache, bounds the work per run, and reports per-score-set outcomes.

Problem

  • run_pipeline.py in mavedb/scripts requires --score-set-urn and exits when it is absent. It is strictly one score set per invocation.
  • No cohort selection. An operator has to assemble and hand-maintain the URN list for each phase of Rerun Mapping Jobs #758.
  • No limit. Re-running the full published corpus in one pass risks saturating both our workers and ClinGen's Allele Registry.
  • No ordering strategy, which wastes our cache. ClinGen allele lookups. Score sets assaying the same target gene overlap heavily in variant space, so running them consecutively turns repeat lookups across sets into cache hits. Arbitrary URN order scatters those lookups across multiple lookups.
  • No outcome summary. A partial failure mid-cohort leaves nothing resumable.

Proposed behavior

A new script alongside the existing ones, e.g. mavedb/scripts/run_score_set_pipelines.py:

  • Cohort selection: --collection-urn (covers the MaveMD phase), --published-only, an organism/taxonomy filter, and filter that allows exclusion of recently run score sets. --score-set-urn stays available and repeatable for ad-hoc reruns.
  • --limit N: process at most N score sets, so the full data set can be worked incrementally.
  • Loose gene grouping: order the resolved cohort by a normalized target-gene key so all score sets sharing a gene run consecutively inside one cache window.
  • --dry-run: print the resolved, ordered cohort with each entry's grouping key and exit without enqueuing anything.
  • Bounded concurrency, plus a per-score-set outcome summary (URN, pipeline, status, error) on stdout.
  • Skip work already current for the requested pipeline, so re-invocation is a no-op over completed score sets.

Acceptance criteria

  • --dry-run prints an ordered cohort with grouping keys and enqueues nothing.
  • --limit N processes exactly N score sets and leaves the rest untouched.
  • Score sets sharing a normalized target-gene name are adjacent in execution order.
  • Gene normalization collapses differing case and surrounding whitespace to one key.
  • A run that fails mid-cohort exits non-zero and still emits a summary distinguishing succeeded from failed score sets.
  • The emitted failure summary is accepted as input for a retry invocation.
  • Re-invoking after a fully successful run enqueues nothing.
  • An unknown pipeline name fails fast listing the available names, matching how run_pipeline behaves today.

Implementation notes

  • Reuse PIPELINE_DEFINITIONS validation, the --updater-id fallback chain (score set modifier, then creator), and --extra-param passthrough from run_pipeline so the two scripts behave consistently.
  • Keep cohort ordering a pure function over (score set, target gene names) so it can be exercised directly against synthetic input while the script is being built.
  • Enables Rerun Mapping Jobs #758. Motivated by the audit tracking added in Traceability and Auditing for Variant-level Job Results #627 and having a fully mapped + annotated data set for the MaveMD paper.

Metadata

Metadata

Assignees

No one assigned

    Labels

    app: backendTask implementation touches the backendtype: enhancementEnhancement to an existing feature

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions