Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

modules config update - preprocessing refactor #485

Merged
merged 33 commits into from
Feb 20, 2022
Merged

modules config update - preprocessing refactor #485

Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
f291ad0
nf-core modules update -a
maxulysse Feb 15, 2022
587a48c
add back params.publish_dir_mode + use ext.when
maxulysse Feb 15, 2022
74ad51f
feat: simplify prepare_genome with ext.when
maxulysse Feb 16, 2022
7fa9d52
fix: publish_dir_more does not inherit
maxulysse Feb 16, 2022
0acbc6a
fix: typos
maxulysse Feb 16, 2022
1898618
feat: code polishing
maxulysse Feb 16, 2022
577e131
feat: code polishing
maxulysse Feb 16, 2022
d11f6a7
feat: code polishing
maxulysse Feb 16, 2022
a51f985
feat: code polishing + refactor fastqc_trimgalore + refactor (skip_qc…
maxulysse Feb 16, 2022
e8808ea
feat: refactor gatk4_mapping/main.nf
maxulysse Feb 16, 2022
dd3a301
feat: move split_fastq in its own subworkflow
maxulysse Feb 16, 2022
2644527
fix: add when directive to local modules
maxulysse Feb 17, 2022
3957f87
feat: refactor markduplicates.nf
maxulysse Feb 17, 2022
649fb51
feat: add comments
maxulysse Feb 18, 2022
2041071
feat: add comments
maxulysse Feb 18, 2022
cdb8254
feat: add comments
maxulysse Feb 18, 2022
af59565
Update nextflow.config
maxulysse Feb 18, 2022
1e97f42
feat: better comments
maxulysse Feb 18, 2022
323fa54
fix: comments polishing
maxulysse Feb 18, 2022
50f6fa3
feat: uncouple intervals from prepare_genome + comments
maxulysse Feb 18, 2022
21a0f74
feat: refactor prepare_recalibration
maxulysse Feb 18, 2022
dc385a5
feat: refactor recalibrate.nf
maxulysse Feb 18, 2022
44f17d5
Update conf/modules.config
maxulysse Feb 18, 2022
367d2be
fix: split_fastq tests
maxulysse Feb 18, 2022
57f766e
Merge branch 'dev_skip' of github.com:maxulysse/nf-core_sarek into de…
maxulysse Feb 18, 2022
89720c5
fix: more tests
maxulysse Feb 18, 2022
774f35a
Update subworkflows/local/split_fastq.nf
maxulysse Feb 18, 2022
30b9cc8
feat: code polishing
maxulysse Feb 19, 2022
c0d36e7
fix: missing channel
maxulysse Feb 19, 2022
31a4aa3
fix: --skip_tools markduplicates
maxulysse Feb 19, 2022
f360d4e
fix: bwa issues with multiple samples
maxulysse Feb 20, 2022
2f80245
fix: targeted tests
maxulysse Feb 20, 2022
096f97d
code polishing
maxulysse Feb 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
456 changes: 263 additions & 193 deletions conf/modules.config
View file
Open in desktop

Large diffs are not rendered by default.

104 changes: 52 additions & 52 deletions modules.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,106 +4,106 @@
"repos": {
"nf-core/modules": {
"bcftools/stats": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"bwa/index": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"bwa/mem": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"bwamem2/index": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"bwamem2/mem": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"cat/fastq": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"cnvkit/batch": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"custom/dumpsoftwareversions": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"deepvariant": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"ensemblvep": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"fastqc": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"fgbio/callmolecularconsensusreads": {
"git_sha": "9d0cad583b9a71a6509b754fdf589cbfbed08961"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"fgbio/fastqtobam": {
"git_sha": "e3285528aca2733ff2d544cb5e5fcc34599226f3"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"fgbio/groupreadsbyumi": {
"git_sha": "9d0cad583b9a71a6509b754fdf589cbfbed08961"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"freebayes": {
"git_sha": "9d0cad583b9a71a6509b754fdf589cbfbed08961"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/applybqsr": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/applyvqsr": {
"git_sha": "598d7abdb2a8df1aa3471c48d9186a9e3465983f"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/baserecalibrator": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/calculatecontamination": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/createsequencedictionary": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/estimatelibrarycomplexity": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/filtermutectcalls": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/gatherbqsrreports": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/genomicsdbimport": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "967fb22dedc2c8855f00e64c3d7b5814c85242a6"
},
"gatk4/genotypegvcfs": {
"git_sha": "598d7abdb2a8df1aa3471c48d9186a9e3465983f"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/getpileupsummaries": {
"git_sha": "f5d5926516d2319c1af83fb4b33834cc4461ce62"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/haplotypecaller": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/learnreadorientationmodel": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/markduplicates": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/mutect2": {
"git_sha": "f5d5926516d2319c1af83fb4b33834cc4461ce62"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"gatk4/variantrecalibrator": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"manta/germline": {
"git_sha": "9d0cad583b9a71a6509b754fdf589cbfbed08961"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"manta/somatic": {
"git_sha": "9d0cad583b9a71a6509b754fdf589cbfbed08961"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"manta/tumoronly": {
"git_sha": "9d0cad583b9a71a6509b754fdf589cbfbed08961"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"msisensorpro/msi_somatic": {
"git_sha": "c8ebd0de36c649a14fc92f2f73cbd9f691a8ce0a"
Expand All @@ -112,58 +112,58 @@
"git_sha": "c8ebd0de36c649a14fc92f2f73cbd9f691a8ce0a"
},
"multiqc": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"samblaster": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"samtools/bam2fq": {
"git_sha": "e751e5040af57e1b4e06ed4e0f3efe6de25c1683"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"samtools/faidx": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"samtools/index": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"samtools/merge": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"samtools/mpileup": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"samtools/stats": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"samtools/view": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"seqkit/split2": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"snpeff": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"strelka/germline": {
"git_sha": "f5d5926516d2319c1af83fb4b33834cc4461ce62"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"strelka/somatic": {
"git_sha": "f5d5926516d2319c1af83fb4b33834cc4461ce62"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"tabix/bgziptabix": {
"git_sha": "e22966ce74340cb671576143e5fdbbd71670cffa"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"tabix/tabix": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"tiddit/sv": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"trimgalore": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
},
"vcftools": {
"git_sha": "e20e57f90b6787ac9a010a980cf6ea98bd990046"
"git_sha": "e745e167c1020928ef20ea1397b6b4d230681b4d"
}
}
}
Expand Down
3 changes: 3 additions & 0 deletions modules/local/bgzip.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ process BGZIP {
tuple val(meta), path("*.vcf"), emit: vcf
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${meta.id}"
Expand Down
3 changes: 3 additions & 0 deletions modules/local/build_intervals/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ process BUILD_INTERVALS {
output:
path "*.bed", emit: bed

when:
task.ext.when == null || task.ext.when

script:
"""
awk -v FS='\t' -v OFS='\t' '{ print \$1, \"0\", \$2 }' ${fasta_fai} > ${fasta_fai.baseName}.bed
Expand Down
5 changes: 4 additions & 1 deletion modules/local/concat_vcf/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,16 @@ process CONCAT_VCF {
tuple val(meta), path("${prefix}.vcf.gz.tbi"), emit: tbi
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
prefix = task.ext.prefix ?: "${meta.id}"
def target_options = target_bed ? "-t ${target_bed}" : ""

"""
concatenateVCFs.sh -i ${fasta_fai} -c ${task.cpus} -o ${prefix}.vcf ${target_options} $args

cat <<-END_VERSIONS > versions.yml
"${task.process}":
bcftools: \$(bcftools --version 2>&1 | head -n1 | sed 's/^.*bcftools //; s/ .*\$//')
Expand Down
3 changes: 3 additions & 0 deletions modules/local/create_intervals_bed/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ process CREATE_INTERVALS_BED {
path ("*.bed"), emit: bed
//TODO version number missing

when:
task.ext.when == null || task.ext.when

script:
// If intervals file is in BED format,
// Fifth column is interpreted to contain runtime estimates
Expand Down
4 changes: 3 additions & 1 deletion modules/local/deeptools/bamcoverage.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,12 @@ process DEEPTOOLS_BAMCOVERAGE {
tuple val(meta), path("*.bigWig"), emit: bigwig
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${meta.id}"

"""
bamCoverage \
--bam $bam \
Expand Down
3 changes: 3 additions & 0 deletions modules/local/gatk4/applybqsrspark/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ process GATK4_APPLYBQSR_SPARK {
tuple val(meta), path("*.cram"), emit: cram
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
def avail_mem = 3
Expand Down
3 changes: 3 additions & 0 deletions modules/local/gatk4/baserecalibratorspark/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ process GATK4_BASERECALIBRATOR_SPARK {
tuple val(meta), path("*.table"), emit: table
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
def avail_mem = 3
Expand Down
3 changes: 3 additions & 0 deletions modules/local/gatk4/gatherpileupsummaries.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ process GATK4_GATHERPILEUPSUMMARIES {
tuple val(meta), path("*.table"), emit: table
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${meta.id}"
Expand Down
3 changes: 3 additions & 0 deletions modules/local/gatk4/intervallisttobed.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ process GATK4_INTERVALLISTTOBED {
tuple val(meta), path("*.bed"), emit: bed
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${meta.id}"
Expand Down
12 changes: 7 additions & 5 deletions modules/local/gatk4/markduplicatesspark/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,13 @@ process GATK4_MARKDUPLICATES_SPARK {
path fasta
path fasta_fai
path dict
val format //either "bam" or "cram"

output:
tuple val(meta), path("*.${format}"), emit: output
path "versions.yml" , emit: versions
tuple val(meta), path("${prefix}"), emit: output
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
Expand All @@ -26,13 +28,13 @@ process GATK4_MARKDUPLICATES_SPARK {
} else {
avail_mem = task.memory.giga
}
def prefix = task.ext.suffix ? "${meta.id}${task.ext.suffix}" : "${meta.id}"
prefix = task.ext.suffix ? "${meta.id}${task.ext.suffix}" : "${meta.id}"
maxulysse marked this conversation as resolved.
Show resolved Hide resolved
def bams = bam.collect(){ x -> "-I ".concat(x.toString()) }.join(" ")
"""
gatk \
MarkDuplicatesSpark \
$bams \
-O ${prefix}.${format} \
-O ${prefix} \
--reference ${fasta} \
--tmp-dir . \
--spark-master local[${task.cpus}] \\
Expand Down
3 changes: 3 additions & 0 deletions modules/local/gatk4/mergemutectstats.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ process GATK4_MERGEMUTECTSTATS {
tuple val(meta), path("*.stats"), emit: stats
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
prefix = task.ext.prefix ?: "${meta.id}"
Expand Down
3 changes: 3 additions & 0 deletions modules/local/index_target_bed/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ process INDEX_TARGET_BED {
output:
tuple path("*.gz"), path("*.gz.tbi"), emit: gz_tbi

when:
task.ext.when == null || task.ext.when

script:
"""
bgzip --threads ${task.cpus} -c ${target_bed} > ${target_bed}.gz
Expand Down
3 changes: 3 additions & 0 deletions modules/local/msisensorpro/msi/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ process MSISENSORPRO_MSI {
tuple val(meta), path("${prefix}_somatic.list") , emit: output_somatic
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
prefix = task.ext.suffix ? "${meta.id}${task.ext.suffix}" : "${meta.id}"
Expand Down
3 changes: 3 additions & 0 deletions modules/local/msisensorpro/scan/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ process MSISENSORPRO_SCAN {
tuple val(meta), path("*.list"), emit: list
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
def prefix = task.ext.suffix ? "${meta.id}${task.ext.suffix}" : "${meta.id}"
Expand Down
Loading