Dnascope module and subworkflow in Sarek #1193

asp8200 · 2023-08-28T10:46:52Z

Implementation of DNAscope. #405

PR checklist

workflows/sarek.nf

…e_pcr_indel_model

github-actions · 2023-08-30T08:41:20Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit d6f610c

+| ✅ 147 tests passed       |+
#| ❔   8 tests were ignored |#
!| ❗   1 tests had warnings |!

❗ Test warnings:

pipeline_todos - TODO string in WorkflowSarek.groovy: Optionally add in-text citation tools to this list.

❔ Tests ignored:

files_exist - File is ignored: .github/workflows/awsfulltest.yml
files_exist - File is ignored: conf/modules.config
files_unchanged - File ignored due to lint config: assets/nf-core-sarek_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-sarek_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-sarek_logo_dark.png
files_unchanged - File ignored due to lint config: lib/NfcoreTemplate.groovy
actions_ci - actions_ci
template_strings - template_strings

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-sarek_logo_light.png
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-sarek_logo_light.png
files_exist - File found: docs/images/nf-core-sarek_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: lib/WorkflowSarek.groovy
files_exist - File found: modules.json
files_exist - File found: pyproject.toml
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: docs/images/nf-core-sarek_logo.png
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.validationShowHiddenParams
nextflow_config - Config variable found: params.validationSchemaIgnoreParams
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: 3.4dev
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
files_unchanged - pyproject.toml matches the template
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
readme - README Zenodo placeholder was replaced with DOI.
pipeline_name_conventions - Name adheres to nf-core convention
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: release-announcments.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
multiqc_config - 'assets/multiqc_config.yml' contains a matching 'report_comment'.
multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'

Run details

nf-core/tools version 2.10
Run at 2023-10-04 16:14:54

nextflow.config

…1210

…t-valling with Dnascope

…alling with Sentieon Dnascope

…ant-calling

…plotyper_joint_germline

…nd genotype_tbi. (Needed for output csv-file.)

…rmline.yml

… docs.

maxulysse · 2023-09-29T14:18:55Z

Can you check the merge conflicts?

docs/output.md

… Sentieon/haplotyper/joint_germline

maxulysse · 2023-10-03T09:11:19Z

LGTM

maxulysse · 2023-10-04T15:44:10Z

Can you sync and update CHANGELOG

FriederikeHanssen · 2023-10-10T19:48:31Z

conf/modules/sentieon_dnascope_joint_germline.config

+
+process {
+
+    // TO-DO: duplicate!!


do we need an issue for this todo?

FriederikeHanssen · 2023-10-10T19:49:46Z

nextflow.config

+    joint_mutect2                     = false // if true, enables patient-wise multi-sample somatic variant calling
+    only_paired_variant_calling       = false // if true, skips germline variant calling for normal-paired sample
+    sentieon_dnascope_emit_mode       = "variant" // default value for Sentieon dnascope
+    sentieon_dnascope_model           = "https://s3.amazonaws.com/sentieon-release/other/SentieonDNAscopeModel1.1.model"


was this the s3 thing you guys mentioned or is this something new?

Yeah, I believe Maxime is working on getting that file into igenomes.

FriederikeHanssen · 2023-10-10T19:57:25Z

Great work, sorry I only got to review it now. Do you have a chance to add it to the subway map/overview image?

asp8200 · 2023-10-11T06:11:48Z

Great work, sorry I only got to review it now. Do you have a chance to add it to the subway map/overview image?

Oh, I totally forgot about that, Rike. I'm afraid I don't know how to use those graphics programs for updating the diagrams. Perhaps you or @maxulysse could update the diagrams?

FriederikeHanssen · 2023-10-11T06:23:22Z

Great work, sorry I only got to review it now. Do you have a chance to add it to the subway map/overview image?

Oh, I totally forgot about that, Rike. I'm afraid I don't know how to use those graphics programs for updating the diagrams. Perhaps you or @maxulysse could update the diagrams?

Sure. It's an alternative to haplotyper, right? When would you choose which?

asp8200 · 2023-10-11T06:30:20Z

Yes, it is an alternative to Sentieon's haplotyper, but where as Sentieon's haplotyper was designed to give the same results at GATK's haplotypercaller - but faster, Sentieon's dnascope improves both "speed and accuracy, and it can perform structural variant calling in addition to calling SNPs and small indels."

FriederikeHanssen · 2023-10-11T07:17:32Z

ok so in the plot should move it inbetween SNps/indels and svs then

asp8200 · 2023-10-11T07:26:28Z

SNps/indels and svs

? You mean listing "Sentieon DNAscope" (or just "DNAscope") with it own bullet between these two bullets:

I think that would be fine.

FriederikeHanssen · 2023-10-11T07:29:39Z

I was thinking more of the sbway map. Anyways I'll make a proposal in the coming days and send it your way

WIP: Implementing Dnascope module and subworkflow in Sarek.

72bb50b

asp8200 requested review from maxulysse and ramprasadn August 28, 2023 10:47

maxulysse reviewed Aug 28, 2023

View reviewed changes

workflows/sarek.nf Outdated Show resolved Hide resolved

asp8200 added 3 commits August 28, 2023 12:29

WIP: Changing sentieon_dnascope_model from string to file-channel

c81aeb5

align

4dcdc9c

Changing Sarek-option sentieon_dnascope_pcr_based to sentieon_dnascop…

cec591b

…e_pcr_indel_model

asp8200 added 2 commits August 30, 2023 14:44

Adding meta to some input-channels for Dnascope module

b2fbd53

Updated dnascope

c12ce59

maxulysse reviewed Sep 6, 2023

View reviewed changes

nextflow.config Outdated Show resolved Hide resolved

asp8200 added 19 commits September 6, 2023 14:51

Adding option dnascope_filter for skip_tools. Will be removed later #…

47ec913

…1210

Add variant-filter for DNASCOPE

79bf941

Trying to order stuff alphabetically

5aa33fb

calling BAM_JOINT_CALLING_GERMLINE_SENTIEON for joint-germline varian…

d8a06bd

…t-valling with Dnascope

Skipping Sentieon VarCal and ApplyVarCal for joint-germline variant-c…

56f65d6

…alling with Sentieon Dnascope

Merge branch 'dev' into sentieon_dnascope

7da3dec

Adding sentieon Dnascope to some error msgs and warnings

01562e7

resolving conflicts from merging dev

f30048d

params.joint_germline was unintentionally out-commented

0f4080c

Continued implementing Sentieon Dnascope

cc72786

Fixing name of pytest. Replacing problematic ampersand character

5cf6520

Bug fix: Sent tbi to vcf-channel and vcf to tbi-channel

ccdf4e5

Adding a bit of error-handling for sentieon-based joint-germline vari…

d74ec55

…ant-calling

Separate configs for sentieon_dnascope_joint_germline and sentieon_ha…

a367e37

…plotyper_joint_germline

Adding tests for dnascope (excel joint-germline)

f55c22e

Adding meta tags patient and variantcaller to channels genotype_vcf a…

a10ad13

…nd genotype_tbi. (Needed for output csv-file.)

joint germline with dnascope

6c78764

test_sentieon_joint_germline.yml -> test_sentieon_haplotyper_joint_ge…

c48f68c

…rmline.yml

Adding test for joint-germline with dnascope

af59cd3

asp8200 added 4 commits September 20, 2023 11:55

Updating changelog with PR for Sentieon DnaScope

235d784

Adding test of Sentieon VQSR not running for Sentieon DnaScope

5cb89c1

prettier

45e961a

Adding info concerning DNAscope. Also other minor improvements to the…

c1c7c6e

… docs.

asp8200 changed the title ~~DRAFT: Dnascope module and subworkflow in Sarek.~~ Dnascope module and subworkflow in Sarek. Sep 20, 2023

asp8200 marked this pull request as ready for review September 20, 2023 13:49

asp8200 requested review from FriederikeHanssen and maxulysse September 20, 2023 13:50

asp8200 changed the title ~~Dnascope module and subworkflow in Sarek.~~ Dnascope module and subworkflow in Sarek Sep 20, 2023

Removing redundant test

cfeb762

Resolving merge conflict and renaming some sentieon subsections

9579af4

maxulysse reviewed Oct 2, 2023

View reviewed changes

docs/output.md Show resolved Hide resolved

Trying to sort out section and subsection for Sentieon/haplotyper and…

31fc80a

… Sentieon/haplotyper/joint_germline

asp8200 added 2 commits October 4, 2023 16:03

Merge branch 'dev' into sentieon_dnascope

46f0788

Moving PR1193 to dev-section since the PR was not merged for Sarek 3.3.2

d6f610c

maxulysse approved these changes Oct 4, 2023

View reviewed changes

asp8200 merged commit df6df02 into dev Oct 4, 2023
58 checks passed

asp8200 deleted the sentieon_dnascope branch October 4, 2023 17:14

FriederikeHanssen reviewed Oct 10, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dnascope module and subworkflow in Sarek #1193

Dnascope module and subworkflow in Sarek #1193

asp8200 commented Aug 28, 2023 •

edited

Loading

github-actions bot commented Aug 30, 2023 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

maxulysse commented Sep 29, 2023

maxulysse commented Oct 3, 2023

maxulysse commented Oct 4, 2023

FriederikeHanssen Oct 10, 2023

FriederikeHanssen Oct 10, 2023

asp8200 Oct 11, 2023

FriederikeHanssen commented Oct 10, 2023

asp8200 commented Oct 11, 2023

FriederikeHanssen commented Oct 11, 2023

asp8200 commented Oct 11, 2023

FriederikeHanssen commented Oct 11, 2023

asp8200 commented Oct 11, 2023 •

edited

Loading

FriederikeHanssen commented Oct 11, 2023

Dnascope module and subworkflow in Sarek #1193

Dnascope module and subworkflow in Sarek #1193

Conversation

asp8200 commented Aug 28, 2023 • edited Loading

PR checklist

github-actions bot commented Aug 30, 2023 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

maxulysse commented Sep 29, 2023

maxulysse commented Oct 3, 2023

maxulysse commented Oct 4, 2023

FriederikeHanssen Oct 10, 2023

Choose a reason for hiding this comment

FriederikeHanssen Oct 10, 2023

Choose a reason for hiding this comment

asp8200 Oct 11, 2023

Choose a reason for hiding this comment

FriederikeHanssen commented Oct 10, 2023

asp8200 commented Oct 11, 2023

FriederikeHanssen commented Oct 11, 2023

asp8200 commented Oct 11, 2023

FriederikeHanssen commented Oct 11, 2023

asp8200 commented Oct 11, 2023 • edited Loading

FriederikeHanssen commented Oct 11, 2023

asp8200 commented Aug 28, 2023 •

edited

Loading

github-actions bot commented Aug 30, 2023 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️

asp8200 commented Oct 11, 2023 •

edited

Loading