Label change for process VCF_TO_CSV

[Stautis]

Description of the bug

Ran into this issue where pipeline was killed in the eighth of eight nf-NFCORE_VARIANTBENCHMARKING_VARIANTBENCHMARKING_COMPARE_BENCHMARK_RESULTS_VCF_TO_CSV jobs with exit status 137 indicating out of memory error. Pipeline completed successfully for me after labelling the VCF_TO_CSV process with process_low rather than process_single, felt it was worth raising the issue here for your consideration.

Command used and terminal output

Command used: 

nextflow run /scratch/vbm/fork/variantbenchmarking -resume -profile docker -process.executor=slurm -process.queue=high \
                --input ${SAMPLESHEET} \
                --outdir ${OUTDIR} \
                --analysis germline \
                --variant_type snv \
                --truth_id HG002 \
                --truth_vcf /data/input/MGM/154/output/variantbenchmarking/resources/HG002_GRCh38_1_22_v4.2.1_benchmark.vcf.gz \
                --regions_bed /data/input/MGM/154/output/variantbenchmarking/resources/HG002_GRCh38_1_22_v4.2.1_benchmark_noinconsistent.bed \
                --method happy,rtgtools \
                --genome GRCh38 \
                --fasta ${REF} \
                --fai ${REF_FAI}


Terminal output:

ERROR ~ Error executing process > 'NFCORE_VARIANTBENCHMARKING:VARIANTBENCHMARKING:COMPARE_BENCHMARK_RESULTS:VCF_TO_CSV (rtgtools)'

Caused by:
  Process `NFCORE_VARIANTBENCHMARKING:VARIANTBENCHMARKING:COMPARE_BENCHMARK_RESULTS:VCF_TO_CSV (rtgtools)` terminated with an error exit status (137)


Command executed:

  vcf_to_csv.py \
      rtgtools.TP_comp.vcf \
      rtgtools.TP_comp.csv

Command exit status:
  137

Command output:
  (empty)

Command error:
  .command.sh: line 4:    49 Killed                  vcf_to_csv.py rtgtools.TP_comp.vcf rtgtools.TP_comp.csv

Work dir:
  /scratch/vbm/7481/work/51/4158dceb960f42f02c1a68d8007448

Container:
  community.wave.seqera.io/library/pip_pandas:40d2e76c16c136f0

Relevant files

No response

System information

Nextflow version: 25.10.3
Hardware: HPC
Executor: slurm
Container engine: Docker
OS: Ubuntu
Version: 8b21c01

[kubranarci]

Hello @Stautis, interesting, I have done tests with germline HG002 snv variants and never get this issue. Probably your input set is somehow too big?

[Stautis]

That is true, the rtgtools.TP_comp.vcf file provided as input to the failing job is 6.6GB. I wanted to do a three-way comparison between deepvariant callsets produced for HG002 sequenced respectively with ONT and PacBio (original .vcf.gz inputs are 412MB and 398MB), and a clair3 callset for the ONT sequenced HG002 data (original .vcf.gz is 238MB). Not sure if the rtgtools.TP_comp.vcf seems unusually large based on this? or if it is just an issue of too large input data?

[kubranarci]

It seems like huge yes, would you mind if I check your file (probably only a subset: kbrnrc@gmail.com)? My tests were mostly on short read sequencing, maybe the collapsion is somehow due to long-read sequencing.