bismark/align: intra-instance -p threading (replace --multicore fork model for Bowtie 2 + HISAT2)

[FelixKrueger]

What & why

bismark/align currently auto-computes --multicore ccore (ccore = task.cpus / 3 directional / / 5 non-directional). For Bowtie 2, --multicore N forks N file-level workers, and each worker runs the 2 (directional/pbat) or 4 (non-directional) concurrent per-strand Bowtie 2 instances — so the job holds N × (2 or 4) copies of the converted index in RAM and re-decodes the input per fork. It also under-subscribes the reservation (the /3,/5 divisor leaves cores idle).

This replaces that with intra-instance threading: -p = task.cpus / n_instances, where n_instances = 2 (directional/pbat), 4 (non-directional), 1 (--combined_index; 2 for --combined_index_parallel --non_directional). The job then holds only 2/4 index copies total and uses all task.cpus.

• Bowtie 2 output is byte-identical — -p N --reorder is thread-invariant.
• HISAT2 shares the same 2/4-instance model and honours -p too, so it gets the divisor. HISAT2 is not strictly thread-invariant, but the effect is negligible (see evidence) and HISAT2 was already task.cpus-dependent under --multicore.
• minimap2/rammap are unchanged — bismark hard-codes minimap2 -t 2 and auto-scales rammap, so the module can't route threads to them; they keep the existing --multicore auto-compute verbatim. (Unifying their threading is tracked separately, upstream in Bismark.)
• Escape hatch preserved: an explicit --multicore/--parallel/-p in ext.args short-circuits the auto-compute. -p is emitted only when task.cpus / n_instances >= 2, else omitted (faithful single-thread-per-instance).

Evidence (benchmark)

GRCh38, 2 M paired-end reads, bismark 3.0.0/3.1.0 (resource model is byte-identical across the two), process-tree peak-RSS sampler, records-only 3-way byte-identity. Median of ≥3 reps; OLD = --multicore ccore, NEW = -p (cpus/n):

mode	cpus	wall OLD→NEW	peak RSS OLD→NEW
directional	12	217.8→122.8 s (−43.6 %)	29.1→9.8 GB (−66 %)
directional	24	126.2→68.0 s (−46.1 %)	55.2→10.1 GB (−82 %)
non-directional	12	390.0→242.9 s (−37.7 %)	29.2→16.3 GB (−44 %)
non-directional	24	215.5→128.4 s (−40.4 %)	55.4→16.6 GB (−70 %)

Bowtie 2 records byte-identical OLD vs NEW vs single-core oracle (both file-order and order-normalised md5), directional + non-directional. HISAT2 (separate spot-check): no RAM change (it never forked), wall −8 to −41 %, output re-baseline negligible (~2–4 records differ per 3.245 M, 99.9999 % identical).

Testing

nf-test 9/9 pass on native amd64 with the bismark 3.1.0 container:

• The 5 existing tests are unchanged — at the module test default cpus=2, task.cpus / n < 2 so no -p is emitted for any path → commands byte-identical → no snapshots regenerated.
• 4 new tests assert the emitted -p via the alignment report's options echo (-p N --reorder), at cpus=8 (dedicated config): Bowtie 2 directional (-p 4), Bowtie 2 non-directional (-p 2), HISAT2 directional (-p 4), and --combined_index (-p 8, n=1).

Notes for reviewers

• No snapshot changes — deliberate (see above). The new -p behaviour is verified by report-echo assertions, not snapshots, because the alignment BAM is thread-invariant (Bowtie 2) or near-invariant (HISAT2), so a snapshot wouldn't distinguish it.
• Consumers: default Bowtie 2 alignments stay byte-identical. HISAT2 consumers running at high cpus will re-baseline by a handful of reads (thread-count-driven; already cpus-dependent). MultiQC parses metric counts (not the settings-echo line), so its output is unaffected for Bowtie 2.
• Pre-existing, out of scope: the minimap2/rammap legacy branch guards --multicore but not its alias --parallel; --minimap2 --parallel N at high cpu would crash on duplicate args. This predates this PR (the old single --multicore block had the same gap) and is left untouched here to keep minimap2/rammap byte-identical; it's folded into the upstream Bismark aligner-threading unification work.

Checklist

• nf-core modules lint bismark/align (CI)
• nf-test test green in CI (native amd64)
• modules.json unaffected (no version bump)

[FelixKrueger]

Thanks @mashehu! Two notes on these:

1. Long name — good call. The catch is that the module invokes bismark (not bowtie2 directly), and bismark's CLI currently defines -p as short-only — no --threads alias, unlike bowtie2 itself — so bismark --threads N would fail to parse against the shipped 3.1.0 container. I've added a --threads alias to bismark (aligner: route -p to minimap2/rammap -t (thread-invariant, default -t 2) FelixKrueger/Bismark#1075); once a bismark release carrying it lands and this module bumps to that container, we can switch the flag here. Keeping -p for now so the module works against the current bismark:3.1.0.

2. Test assertion — I've kept -p 2 --reorder rather than --threads 2: the assertion checks the alignment report, which echoes the underlying bowtie2 command. bismark always constructs that with -p internally regardless of how threads are passed to bismark, so --threads never appears in the report — the assertion has to stay -p.