Mutect2 multi sample subworkflow #1013

berguner · 2023-05-05T09:44:39Z

~~Create a subworkflow~~ run mutect2 in joint mode for patients with multiple tumor samples as described in #916 .
~~- Created 2 subworkflows BAM_VARIANT_CALLING_TUMOR_ONLY_MUTECT2_MS and BAM_VARIANT_CALLING_SOMATIC_MUTECT2_MS.~~

Added the ~~--mutect2_multi_sample~~ --joint_mutect pipeline parameter to switch on multi sample mode.
Added test data for multi sample mode
~~- VCFs are saved under variant_calling/mutect2_ms in multi sample mode.~~

PR checklist

This comment contains a description of changes (with reason).
If you've fixed a bug or added code that should be tested, add tests!
Make sure your code lints (nf-core lint).
Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
CHANGELOG.md is updated.
README.md is updated (including new tool citations and authors/contributors).

… stats

…ables

- Add filtermutectcalls to mutect2_ms subwf

- Separated test profile of mutect2_ms from other tools

maxulysse · 2023-05-23T13:12:43Z

Hi @berguner sorry for the delay.
I'm afraid we have some merge conflicts to solve there.
You seem to have used slightly out of date code on you PR.
Are you ok into trying to solve these conflicts, or do you need any help with that?

berguner · 2023-05-24T08:24:56Z

Hi @maxulysse , I will try to solve them and get back to you soon.

nextflow.config

maxulysse · 2023-05-31T14:07:16Z

you have tiny issues with end of lines in some files, but this is looking super good.

Co-authored-by: Maxime U Garcia <maxime.garcia@seqera.io>

github-actions · 2023-05-31T14:08:40Z

`nf-core lint` overall result: Failed ❌

Posted for pipeline commit 2c320b3

+| ✅ 147 tests passed       |+
#| ❔   8 tests were ignored |#
-| ❌   6 tests failed       |-

❌ Test failures:

nextflow_config - Config variable not found: params.validationShowHiddenParams
nextflow_config - Config variable not found: params.validationSchemaIgnoreParams
files_unchanged - .github/CONTRIBUTING.md does not match the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml does not match the template
schema_params - Param schema_ignore_params from nextflow config not found in nextflow_schema.json
multiqc_config - 'assets/multiqc_config.yml' does not contain a matching 'report_comment'.

❔ Tests ignored:

files_exist - File is ignored: conf/modules.config
files_exist - File is ignored: conf/test.config
files_exist - File is ignored: conf/test_full.config
files_unchanged - File ignored due to lint config: assets/nf-core-sarek_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-sarek_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-sarek_logo_dark.png
files_unchanged - File ignored due to lint config: lib/NfcoreTemplate.groovy
template_strings - template_strings

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-sarek_logo_light.png
files_exist - File found: docs/images/nf-core-sarek_logo_light.png
files_exist - File found: docs/images/nf-core-sarek_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: lib/WorkflowSarek.groovy
files_exist - File found: modules.json
files_exist - File found: pyproject.toml
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: docs/images/nf-core-sarek_logo.png
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: 3.3dev
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
files_unchanged - pyproject.toml matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
readme - README Zenodo placeholder was replaced with DOI.
pipeline_todos - No TODO strings found
pipeline_name_conventions - Name adheres to nf-core convention
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: awsfulltest_germline.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: pytest-workflow_release.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: pytest-workflow.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'

Run details

nf-core/tools version 2.9
Run at 2023-07-04 18:19:02

berguner · 2023-05-31T14:10:28Z

I just run nf-core lint --fix files_unchanged, I don't know how else I can sort out the linting issues. Should I run prettier separately?

maxulysse · 2023-05-31T14:15:15Z

errors are Wrong line endings or no final newline, so it's not prettier, but similar, let me check what can be done.
Maybe a @nf-core-bot fix linting pretty please will do the trick

maxulysse · 2023-05-31T14:25:00Z

ok, so some fixes are done, but not all.
Do you mind if I push commit in your branch?

Co-authored-by: Maxime U Garcia <maxime.garcia@seqera.io>

subworkflows/local/bam_variant_calling_somatic_all/main.nf

subworkflows/local/bam_variant_calling_tumor_only_all/main.nf

CHANGELOG.md

nextflow.config

Co-authored-by: Maxime U Garcia <maxime.garcia@seqera.io>

maxulysse · 2023-06-30T12:12:32Z

Looks amazing.
We're now missing just some documentation about it

FriederikeHanssen

Fantastic work! I think you also found two inefficient computations we do, that can be fixed with your code

conf/modules/mutect2.config

conf/modules/prepare_genome.config

subworkflows/local/bam_variant_calling_somatic_mutect2/main.nf

Co-authored-by: Friederike Hanssen <Friederike.hanssen@qbic.uni-tuebingen.de>

subworkflows/local/bam_variant_calling_somatic_mutect2/main.nf

Co-authored-by: Maxime U Garcia <maxime.garcia@seqera.io>

maxulysse

AMAZING

berguner added 2 commits April 27, 2023 12:02

New subworkflow: bam_variant_calling_somatic_mutect2_ms

627b931

mutect2_ms for patients with matched normal producing merged vcfs and…

eb2f731

… stats

berguner changed the title ~~Mutect2 multi sample subworkflow #916~~ Mutect2 multi sample subworkflow May 5, 2023

berguner added 10 commits May 8, 2023 17:02

mutect2_ms: producing read orientation bias stats and contamination t…

47caddf

…ables

- Update gatk4/filtermutectcalls module

e6a0464

- Add filtermutectcalls to mutect2_ms subwf

- Completed mutect2_ms subwf for matched normal

5ed1e76

- Separated test profile of mutect2_ms from other tools

create bam_variant_calling_tumor_only_mutect2_ms subwf

eaed8c6

mix tumor only mutect2_ms_vcf with annotation channel

cb62b7f

Only run gatherpileupsummaries in mutect2_ms when num_intervals >1

9de143b

Changed mutec2_multi_sample into a mode instead of being a separate tool

8863b03

remove _MS suffix from process names

d02aa2e

linting

36569df

remove .contains('mutect2_multi_sample') checks

61f2a9b

berguner self-assigned this May 17, 2023

update CHANGELOG.md README.md

6ca5c14

berguner marked this pull request as ready for review May 17, 2023 09:42

berguner requested a review from maxulysse as a code owner May 17, 2023 09:42

berguner added 3 commits May 30, 2023 16:42

tiny change

8436b45

Fix merge conflicts with dev

cd3c8aa

mutect2_ms takes intervals as a separate input

5e11403

maxulysse reviewed May 31, 2023

View reviewed changes

nextflow.config Outdated Show resolved Hide resolved

Update nextflow.config

5a0f9b7

Co-authored-by: Maxime U Garcia <maxime.garcia@seqera.io>

[automated] Fix linting with Prettier

26d77b7

berguner and others added 3 commits June 30, 2023 10:06

Update subworkflows/local/bam_variant_calling_somatic_all/main.nf

f02b3f6

Co-authored-by: Maxime U Garcia <maxime.garcia@seqera.io>

add back mutect2 if blocks

137215a

EditorConfig

c266d43

maxulysse reviewed Jun 30, 2023

View reviewed changes

subworkflows/local/bam_variant_calling_somatic_all/main.nf Outdated Show resolved Hide resolved

maxulysse reviewed Jun 30, 2023

View reviewed changes

subworkflows/local/bam_variant_calling_tumor_only_all/main.nf Outdated Show resolved Hide resolved

berguner and others added 2 commits June 30, 2023 11:02

rename mutect2_multi_sample parameter to joint_mutect2

34b2dfb

Merge branch 'dev' into mutect2_multi_sample

7de02b9

maxulysse reviewed Jun 30, 2023

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

maxulysse reviewed Jun 30, 2023

View reviewed changes

nextflow.config Outdated Show resolved Hide resolved

berguner and others added 2 commits June 30, 2023 11:18

Update CHANGELOG.md

218eba3

Co-authored-by: Maxime U Garcia <maxime.garcia@seqera.io>

cascade params.joint_mutect2 as an input

04427d2

berguner added 2 commits June 30, 2023 14:17

fix mutect2.config

d4b4232

joint_mutect2 documentation

7e28e9e

FriederikeHanssen requested changes Jul 1, 2023

View reviewed changes

berguner and others added 6 commits July 3, 2023 10:29

Update conf/modules/mutect2.config

e7931e4

Co-authored-by: Friederike Hanssen <Friederike.hanssen@qbic.uni-tuebingen.de>

Update conf/modules/mutect2.config

ecdade4

Co-authored-by: Friederike Hanssen <Friederike.hanssen@qbic.uni-tuebingen.de>

Update conf/modules/mutect2.config

cf392c8

Co-authored-by: Friederike Hanssen <Friederike.hanssen@qbic.uni-tuebingen.de>

Update conf/modules/mutect2.config

20c3ce4

Co-authored-by: Friederike Hanssen <Friederike.hanssen@qbic.uni-tuebingen.de>

Update conf/modules/mutect2.config

9ea55e8

Co-authored-by: Friederike Hanssen <Friederike.hanssen@qbic.uni-tuebingen.de>

Update subworkflows/local/bam_variant_calling_somatic_mutect2/main.nf

3343cdb

Co-authored-by: Friederike Hanssen <Friederike.hanssen@qbic.uni-tuebingen.de>

berguner mentioned this pull request Jul 3, 2023

Pass on the number of tumor samples per patient #1141

Open

berguner added 2 commits July 3, 2023 11:22

fix configs

6327c14

remove redundancy for mutect2 pileups for tumor/normal pairs

b3fec41

maxulysse reviewed Jul 4, 2023

View reviewed changes

subworkflows/local/bam_variant_calling_somatic_mutect2/main.nf Outdated Show resolved Hide resolved

Update subworkflows/local/bam_variant_calling_somatic_mutect2/main.nf

2c320b3

Co-authored-by: Maxime U Garcia <maxime.garcia@seqera.io>

FriederikeHanssen approved these changes Jul 5, 2023

View reviewed changes

maxulysse approved these changes Jul 5, 2023

View reviewed changes

FriederikeHanssen merged commit 1b37753 into nf-core:dev Jul 5, 2023

maxulysse deleted the mutect2_multi_sample branch July 5, 2023 11:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mutect2 multi sample subworkflow #1013

Mutect2 multi sample subworkflow #1013

berguner commented May 5, 2023 •

edited

Loading

maxulysse commented May 23, 2023

berguner commented May 24, 2023

maxulysse commented May 31, 2023

github-actions bot commented May 31, 2023 •

edited

Loading

❌ Test failures:

❔ Tests ignored:

✅ Tests passed:

Run details

berguner commented May 31, 2023

maxulysse commented May 31, 2023

maxulysse commented May 31, 2023

maxulysse commented Jun 30, 2023

FriederikeHanssen left a comment

maxulysse left a comment

Mutect2 multi sample subworkflow #1013

Mutect2 multi sample subworkflow #1013

Conversation

berguner commented May 5, 2023 • edited Loading

PR checklist

maxulysse commented May 23, 2023

berguner commented May 24, 2023

maxulysse commented May 31, 2023

github-actions bot commented May 31, 2023 • edited Loading

nf-core lint overall result: Failed ❌

❌ Test failures:

❔ Tests ignored:

✅ Tests passed:

Run details

berguner commented May 31, 2023

maxulysse commented May 31, 2023

maxulysse commented May 31, 2023

maxulysse commented Jun 30, 2023

FriederikeHanssen left a comment

Choose a reason for hiding this comment

maxulysse left a comment

Choose a reason for hiding this comment

berguner commented May 5, 2023 •

edited

Loading

github-actions bot commented May 31, 2023 •

edited

Loading

`nf-core lint` overall result: Failed ❌