Include all processes in inspect command

[bentsherman]

I found a way to improve the inspect command without using the new script parser -- I just look at all process definitions across all included scripts, regardless of whether they are ever called.

• All processes are included, even if a process is defined but never called by the workflow. Static analysis would be required to extract the process calls from the workflow.

• The process "base name" is used instead of the fully qualified name, and aliases are not included. Static analysis would be required to get the fully qualified name. But I figured the base name is fine because you shouldn't need to specify different containers for different instances of the same process.

I went ahead with this approach because nf-core needs this functionality and I don't know how long it will take to incorporate the new script parser.

Here's the output for rnaseq:

process { withName: 'BBMAP_BBSPLIT' { container = 'quay.io/biocontainers/bbmap:39.10--h92535d8_0' } }
process { withName: 'BEDTOOLS_GENOMECOV' { container = 'quay.io/nf-core/bedtools_coreutils:a623c13f66d5262b' } }
process { withName: 'BRACKEN_BRACKEN' { container = 'quay.io/biocontainers/bracken:2.9--py38h2494328_0' } }
process { withName: 'CAT_FASTQ' { container = 'quay.io/nf-core/coreutils:9.5--ae99c88a9b28c264' } }
process { withName: 'CUSTOM_CATADDITIONALFASTA' { container = 'quay.io/biocontainers/python:3.12' } }
process { withName: 'CUSTOM_GETCHROMSIZES' { container = 'quay.io/biocontainers/samtools:1.21--h50ea8bc_0' } }
process { withName: 'CUSTOM_TX2GENE' { container = 'quay.io/biocontainers/python:3.10.4' } }
process { withName: 'DESEQ2_QC' { container = 'quay.io/biocontainers/mulled-v2-8849acf39a43cdd6c839a369a74c0adc823e2f91:ab110436faf952a33575c64dd74615a84011450b-0' } }
process { withName: 'DUPRADAR' { container = 'quay.io/biocontainers/bioconductor-dupradar:1.32.0--r43hdfd78af_0' } }
process { withName: 'FASTP' { container = 'quay.io/biocontainers/fastp:0.23.4--h5f740d0_0' } }
process { withName: 'FASTQC' { container = 'quay.io/biocontainers/fastqc:0.12.1--hdfd78af_0' } }
process { withName: 'FQ_SUBSAMPLE' { container = 'quay.io/biocontainers/fq:0.12.0--h9ee0642_0' } }
process { withName: 'GFFREAD' { container = 'quay.io/biocontainers/gffread:0.12.7--hdcf5f25_4' } }
process { withName: 'GTF2BED' { container = 'quay.io/biocontainers/perl:5.26.2' } }
process { withName: 'GTF_FILTER' { container = 'quay.io/biocontainers/python:3.9--1' } }
process { withName: 'GUNZIP' { container = 'quay.io/nf-core/ubuntu:22.04' } }
process { withName: 'HISAT2_ALIGN' { container = 'quay.io/biocontainers/mulled-v2-a97e90b3b802d1da3d6958e0867610c718cb5eb1:2cdf6bf1e92acbeb9b2834b1c58754167173a410-0' } }
process { withName: 'HISAT2_BUILD' { container = 'quay.io/biocontainers/hisat2:2.2.1--h1b792b2_3' } }
process { withName: 'HISAT2_EXTRACTSPLICESITES' { container = 'quay.io/biocontainers/hisat2:2.2.1--h1b792b2_3' } }
process { withName: 'KALLISTO_INDEX' { container = 'quay.io/biocontainers/kallisto:0.51.1--heb0cbe2_0' } }
process { withName: 'KALLISTO_QUANT' { container = 'quay.io/biocontainers/kallisto:0.51.1--heb0cbe2_0' } }
process { withName: 'KRAKEN2_KRAKEN2' { container = 'quay.io/biocontainers/mulled-v2-8706a1dd73c6cc426e12dd4dd33a5e917b3989ae:c8cbdc8ff4101e6745f8ede6eb5261ef98bdaff4-0' } }
process { withName: 'MULTIQC' { container = 'quay.io/biocontainers/multiqc:1.25.1--pyhdfd78af_0' } }
process { withName: 'MULTIQC_CUSTOM_BIOTYPE' { container = 'quay.io/biocontainers/python:3.9--1' } }
process { withName: 'PICARD_MARKDUPLICATES' { container = 'quay.io/biocontainers/picard:3.1.1--hdfd78af_0' } }
process { withName: 'PREPROCESS_TRANSCRIPTS_FASTA_GENCODE' { container = 'quay.io/nf-core/ubuntu:20.04' } }
process { withName: 'PRESEQ_LCEXTRAP' { container = 'quay.io/biocontainers/preseq:3.2.0--hdcf5f25_6' } }
process { withName: 'QUALIMAP_RNASEQ' { container = 'quay.io/biocontainers/qualimap:2.3--hdfd78af_0' } }
process { withName: 'RSEM_CALCULATEEXPRESSION' { container = 'quay.io/biocontainers/mulled-v2-cf0123ef83b3c38c13e3b0696a3f285d3f20f15b:64aad4a4e144878400649e71f42105311be7ed87-0' } }
process { withName: 'RSEM_MERGE_COUNTS' { container = 'quay.io/nf-core/ubuntu:20.04' } }
process { withName: 'RSEM_PREPAREREFERENCE' { container = 'quay.io/biocontainers/mulled-v2-cf0123ef83b3c38c13e3b0696a3f285d3f20f15b:64aad4a4e144878400649e71f42105311be7ed87-0' } }
process { withName: 'RSEQC_BAMSTAT' { container = 'quay.io/biocontainers/rseqc:5.0.3--py39hf95cd2a_0' } }
process { withName: 'RSEQC_INFEREXPERIMENT' { container = 'quay.io/biocontainers/rseqc:5.0.3--py39hf95cd2a_0' } }
process { withName: 'RSEQC_INNERDISTANCE' { container = 'quay.io/biocontainers/rseqc:5.0.3--py39hf95cd2a_0' } }
process { withName: 'RSEQC_JUNCTIONANNOTATION' { container = 'quay.io/biocontainers/rseqc:5.0.3--py39hf95cd2a_0' } }
process { withName: 'RSEQC_JUNCTIONSATURATION' { container = 'quay.io/biocontainers/rseqc:5.0.3--py39hf95cd2a_0' } }
process { withName: 'RSEQC_READDISTRIBUTION' { container = 'quay.io/biocontainers/rseqc:5.0.3--py39hf95cd2a_0' } }
process { withName: 'RSEQC_READDUPLICATION' { container = 'quay.io/biocontainers/rseqc:5.0.3--py39hf95cd2a_0' } }
process { withName: 'RSEQC_TIN' { container = 'quay.io/biocontainers/rseqc:5.0.3--py39hf95cd2a_0' } }
process { withName: 'SALMON_INDEX' { container = 'quay.io/biocontainers/salmon:1.10.3--h6dccd9a_2' } }
process { withName: 'SALMON_QUANT' { container = 'quay.io/biocontainers/salmon:1.10.3--h6dccd9a_2' } }
process { withName: 'SAMTOOLS_FLAGSTAT' { container = 'quay.io/biocontainers/samtools:1.21--h50ea8bc_0' } }
process { withName: 'SAMTOOLS_IDXSTATS' { container = 'quay.io/biocontainers/samtools:1.21--h50ea8bc_0' } }
process { withName: 'SAMTOOLS_INDEX' { container = 'quay.io/biocontainers/samtools:1.21--h50ea8bc_0' } }
process { withName: 'SAMTOOLS_SORT' { container = 'quay.io/biocontainers/samtools:1.21--h50ea8bc_0' } }
process { withName: 'SAMTOOLS_STATS' { container = 'quay.io/biocontainers/samtools:1.21--h50ea8bc_0' } }
process { withName: 'SORTMERNA' { container = 'quay.io/nf-core/sortmerna:4.3.7--6502243397c065ba' } }
process { withName: 'STAR_ALIGN' { container = 'quay.io/nf-core/star_samtools_htslib_gawk:10c6e8c834460019' } }
process { withName: 'STAR_ALIGN_IGENOMES' { container = 'quay.io/biocontainers/mulled-v2-1fa26d1ce03c295fe2fdcf85831a92fbcbd7e8c2:59cdd445419f14abac76b31dd0d71217994cbcc9-0' } }
process { withName: 'STAR_GENOMEGENERATE' { container = 'quay.io/nf-core/star_samtools_htslib_gawk:10c6e8c834460019' } }
process { withName: 'STAR_GENOMEGENERATE_IGENOMES' { container = 'quay.io/biocontainers/mulled-v2-1fa26d1ce03c295fe2fdcf85831a92fbcbd7e8c2:59cdd445419f14abac76b31dd0d71217994cbcc9-0' } }
process { withName: 'STRINGTIE_STRINGTIE' { container = 'quay.io/biocontainers/stringtie:2.2.3--h43eeafb_0' } }
process { withName: 'SUBREAD_FEATURECOUNTS' { container = 'quay.io/biocontainers/subread:2.0.6--he4a0461_2' } }
process { withName: 'SUMMARIZEDEXPERIMENT_SUMMARIZEDEXPERIMENT' { container = 'quay.io/biocontainers/bioconductor-summarizedexperiment:1.32.0--r43hdfd78af_0' } }
process { withName: 'TRIMGALORE' { container = 'quay.io/biocontainers/trim-galore:0.6.10--hdfd78af_1' } }
process { withName: 'TXIMETA_TXIMPORT' { container = 'quay.io/biocontainers/bioconductor-tximeta:1.20.1--r43hdfd78af_0' } }
process { withName: 'UCSC_BEDCLIP' { container = 'quay.io/biocontainers/ucsc-bedclip:377--h0b8a92a_2' } }
process { withName: 'UCSC_BEDGRAPHTOBIGWIG' { container = 'quay.io/biocontainers/ucsc-bedgraphtobigwig:469--h9b8f530_0' } }
process { withName: 'UMITOOLS_DEDUP' { container = 'quay.io/biocontainers/umi_tools:1.1.5--py39hf95cd2a_0' } }
process { withName: 'UMITOOLS_EXTRACT' { container = 'quay.io/biocontainers/umi_tools:1.1.5--py39hf95cd2a_0' } }
process { withName: 'UMITOOLS_PREPAREFORRSEM' { container = 'quay.io/biocontainers/umi_tools:1.1.5--py39hf95cd2a_0' } }
process { withName: 'UNTAR' { container = 'quay.io/nf-core/ubuntu:22.04' } }

[netlify]

✅ Deploy Preview for nextflow-docs-staging canceled.

Name	Link
🔨 Latest commit	4b31d1b
🔍 Latest deploy log	https://app.netlify.com/sites/nextflow-docs-staging/deploys/675ab5116b53640008e52b0d

[ewels]

Yesss! 🙏🏻

[pditommaso]

Looks good! made a couple of minor comments

[pditommaso]

Ok, thinks we are there. Pushed a few changes

• fixed a compilation error
• reverted ContainerInspectMode changes that were not needed and keep aligned with current implementation (we may want to backport this to 24.10.x
• keep method chaining in parseScript

[pditommaso]

Vamos!

[bentsherman]

This will still run the entry workflow if concretize is enabled, is that intended?

[pditommaso]

Good point, initially I was thinking the run was needed by the contretize to await for the container completion, but checking it now it implemented here.

therefore the "breaking" change is needed the ContainerInspectMode class

[pditommaso]

Solved by https://github.com/nextflow-io/nextflow/pull/5605/files