feat: balsamic UMI workflow solution #932

ashwini06 · 2022-05-17T12:26:20Z

This PR:

Fixes: #896

Adds new analysis_workflow option to balsamic config case to run different workflows (balsamic, balsamic-umi)
If --analysis-workflow=balsamic run only balsamic workflow, else if --analysis-workflow=balsamic-umi runs both balsamic+UMI workflow

Test CommandLine

# create config json for balsamic only analysis

balsamic config case  --analysis-workflow balsamic --case-id TN_panel_balsamic --analysis-dir run_tests/ --balsamic-cache /home/proj/stage/cancer/balsamic_cache/ -t tests/test_data/fastq/S1_R_1.fastq.gz -n tests/test_data/fastq/S2_R_1.fastq.gz -p tests/test_data/references/panel/panel.bed

balsamic run analysis -s run_tests/ TN_panel_balsamic/TN_panel_balsamic.json

# create config json for balsamic+UMI  analysis

balsamic config case  --analysis-workflow balsamic-umi --case-id TN_panel_balsamic_umi --analysis-dir run_tests/ --balsamic-cache /home/proj/stage/cancer/balsamic_cache/ -t tests/test_data/fastq/S1_R_1.fastq.gz -n tests/test_data/fastq/S2_R_1.fastq.gz -p tests/test_data/references/panel/panel.bed 

balsamic run analysis -s run_tests/TN_panel_balsamic_umi/TN_panel_balsamic_umi.json

Review and tests:

Tests pass
Code review
New code is executed and covered by tests, and test approve

…Genomics/BALSAMIC into feat/workflow_solution

codecov · 2022-05-17T12:47:47Z

Codecov Report

Merging #932 (5e758cc) into develop (97e83cf) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff            @@
##           develop     #932   +/-   ##
========================================
  Coverage    99.25%   99.25%           
========================================
  Files           29       29           
  Lines         1749     1756    +7     
========================================
+ Hits          1736     1743    +7     
  Misses          13       13

Flag	Coverage Δ
unittests	`99.25% <100.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
BALSAMIC/commands/config/pon.py	`95.12% <ø> (ø)`
BALSAMIC/commands/config/qc.py	`100.00% <ø> (ø)`
BALSAMIC/commands/config/case.py	`96.42% <100.00%> (ø)`
BALSAMIC/constants/common.py	`100.00% <100.00%> (ø)`
BALSAMIC/utils/models.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 97e83cf...5e758cc. Read the comment docs.

ashwini06 · 2022-05-17T12:49:36Z

balsamic dry-runs

For Tonly_panel (Balsamic_only)

For Tonly_panel (Balsamic + UMI)

For TN_panel (Balsamic only)

For TN_panel (Balsamic+UMI)

khurrammaqbool

It looks good. I have a minor suggestion related code readability.

khurrammaqbool · 2022-05-19T12:31:59Z

BALSAMIC/commands/config/case.py

@@ -136,6 +129,19 @@
        "will be <outdir>/genome_version"
    ),
 )
+@click.option(
+    "-w",
+    "--analysis-workflow",


"analysis workflow" here and subsequently may cause confusion with "workflow solution". I suggest calling it umi-option with balsamic-umi-on and balsamic-umi-off as options and all subsequent terms updated accordingly, which could be a better alternative to adding "workflow" as keyword and may make it unique and descriptive.

Thanks Khurram! I agree and thought about this existing constantworkflow_solution

BALSAMIC/BALSAMIC/constants/common.py

Line 40 in 904d183

WORKFLOW_SOLUTION = ["BALSAMIC", "Sentieon", "DRAGEN", "Sentieon_umi"]

But this option (--workflow-solution) is not used in config case CLI. It is just a part of constant used within balsamic.smk

Your suggestion to use balsamic-umi-on and balsamic-umi-off instead of analysis_workflow might be too specific. I was thinking of expanding this analysis_workflow option for running other analysis snakemake workflows (for eg: balsamic_PON) in the future.

So the thought process for running balsamic from production based on different analysis_workflow(balsamic/balsamic_umi/balsamic_pon) is :

cg workflow balsamic start <case_id> ## executes balsamic config case --case-id xx --analysis-dir xx --balsamic-cache xx -t xx -p xx --analysis-workflow balsamic balsamic run analysis -s xx.json cg worfklow balsamic-umi start <case_id> balsamic config case --case-id xx --analysis-dir xx --balsamic-cache xx -t xx -p xx --analysis-workflow balsamic_umi balsamic run analysis -s xx.json cg workflow balsami-pon start <case_id> # Executes and generate PON reference file in specific location on hasta balsamic config pon --case-id xx --analysis-dir xx --balsamic-cache xx --fastq_dir xx -p xx --analysis-workflow balsamic_pon balsamic run analysis -s xx.json

I agree with --analysis_workflow and will also use it for balsamic-qc. I think it makes cg commands clearer. Indeed analysis_workflow is close to workflow_solution but workflow_solution could be more specific, no? Because only the variant callers are influenced and not the rest of the pipeline. What about something like calling_solution?

workflow_solution is just a constant name used within the balsamic.smk. If you think it could lead to confusion, we can change the name of it to something else. Like calling_solution or variant_calling_solution.

I don't think workflow_solution or analysis_workflow needs to be changed, but if you want to change one of those to make them more different from each other, I vote to change workflow_solution to variant_calling_solution or something like it and keep analysis_workflow

BALSAMIC/utils/models.py

BALSAMIC/workflows/balsamic.smk

rannick · 2022-05-20T14:29:41Z

BALSAMIC/commands/config/case.py

@@ -136,6 +129,19 @@
        "will be <outdir>/genome_version"
    ),
 )
+@click.option(
+    "-w",
+    "--analysis-workflow",


I don't think workflow_solution or analysis_workflow needs to be changed, but if you want to change one of those to make them more different from each other, I vote to change workflow_solution to variant_calling_solution or something like it and keep analysis_workflow

Co-authored-by: Annick Renevey <47788523+rannick@users.noreply.github.com>

sonarqubecloud · 2022-05-23T13:14:06Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

ivadym

👍

ashwini06 added 7 commits May 17, 2022 14:08

add option for analysis workflow to config case

fa11160

add analysis workflow to constants

2b7c71d

add analysis workflows to models

4045108

add conditions to balsamic workflow

0cdbd87

modify multiqc condition for umi

f87b1ab

modify condition for umi files

de92bcf

add default analysis workflow to qc_config

bb26795

ashwini06 requested review from hassanfa, khurrammaqbool and ivadym as code owners May 17, 2022 12:26

ashwini06 self-assigned this May 17, 2022

Merge branch 'master' into feat/workflow_solution

814f288

ashwini06 changed the base branch from master to develop May 17, 2022 12:26

ashwini06 marked this pull request as draft May 17, 2022 12:28

ashwini06 changed the title ~~Feat: balsamic UMI workflow solution~~ feat: balsamic UMI workflow solution May 17, 2022

ashwini06 added 3 commits May 17, 2022 14:35

modify test models

1db1242

add analysis workflow for pon to work fine

f784892

Merge branch 'feat/workflow_solution' of https://github.com/Clinical-…

085b502

…Genomics/BALSAMIC into feat/workflow_solution

ashwini06 added 4 commits May 17, 2022 15:58

change balsamic_umi to balsamic-umi

8e2a6c1

black format pon config file

ed4b448

Merge branch 'develop' into feat/workflow_solution

273f08f

update Changelog

6f9d32e

ashwini06 marked this pull request as ready for review May 19, 2022 09:38

khurrammaqbool requested changes May 19, 2022

View reviewed changes

ashwini06 requested a review from rannick May 20, 2022 14:00

rannick approved these changes May 20, 2022

View reviewed changes

ashwini06 changed the base branch from develop to master May 20, 2022 14:48

ashwini06 changed the base branch from master to develop May 20, 2022 14:49

ashwini06 and others added 2 commits May 20, 2022 17:24

fix space in models

d91beb3

Co-authored-by: Annick Renevey <47788523+rannick@users.noreply.github.com>

remove extra line

1bac55f

Co-authored-by: Annick Renevey <47788523+rannick@users.noreply.github.com>

This was referenced May 20, 2022

feat: balsamic UMI workflow solution-duplicate PR #934

Closed

feat: balsamic UMI workflow solution-duplicate PR1 #935

Closed

feat: Option to support new analysis workflows #936

Closed

feat: analysis workflow option #937

Closed

Merge branch 'develop' into feat/workflow_solution

5e758cc

ivadym approved these changes May 23, 2022

View reviewed changes

ashwini06 merged commit 8d8ec17 into develop May 23, 2022

ashwini06 deleted the feat/workflow_solution branch May 23, 2022 15:04

ashwini06 mentioned this pull request Jun 1, 2022

Decouple UMI workflow from BALSAMIC #896

Closed

ashwini06 mentioned this pull request Jun 13, 2022

Plan for BALSAMIC v10.0 #954

Closed

17 tasks

ashwini06 mentioned this pull request Jun 21, 2022

feat: New Release BALSAMIC v10.0.0 #964

Merged

3 tasks

ivadym mentioned this pull request Aug 5, 2022

adding of umi analysis type for BALSAMIC cases Clinical-Genomics/cg#1419

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: balsamic UMI workflow solution #932

feat: balsamic UMI workflow solution #932

ashwini06 commented May 17, 2022 •

edited

Loading

codecov bot commented May 17, 2022 •

edited

Loading

ashwini06 commented May 17, 2022 •

edited

Loading

khurrammaqbool left a comment

khurrammaqbool May 19, 2022

ashwini06 May 19, 2022

rannick May 20, 2022

ashwini06 May 20, 2022

rannick May 20, 2022 •

edited

Loading

rannick May 20, 2022 •

edited

Loading

sonarqubecloud bot commented May 23, 2022

ivadym left a comment

feat: balsamic UMI workflow solution #932

feat: balsamic UMI workflow solution #932

Conversation

ashwini06 commented May 17, 2022 • edited Loading

This PR:

Review and tests:

codecov bot commented May 17, 2022 • edited Loading

Codecov Report

ashwini06 commented May 17, 2022 • edited Loading

khurrammaqbool left a comment

Choose a reason for hiding this comment

khurrammaqbool May 19, 2022

Choose a reason for hiding this comment

ashwini06 May 19, 2022

Choose a reason for hiding this comment

rannick May 20, 2022

Choose a reason for hiding this comment

ashwini06 May 20, 2022

Choose a reason for hiding this comment

rannick May 20, 2022 • edited Loading

Choose a reason for hiding this comment

rannick May 20, 2022 • edited Loading

Choose a reason for hiding this comment

sonarqubecloud bot commented May 23, 2022

ivadym left a comment

Choose a reason for hiding this comment

ashwini06 commented May 17, 2022 •

edited

Loading

codecov bot commented May 17, 2022 •

edited

Loading

ashwini06 commented May 17, 2022 •

edited

Loading

rannick May 20, 2022 •

edited

Loading

rannick May 20, 2022 •

edited

Loading