-
Notifications
You must be signed in to change notification settings - Fork 71
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sharded AnnotateVcf workflow #566
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this. Unless I'm missing something, I think things can be simplified a bit though - see comments below.
...lates/terra_workspaces/cohort_mode/workflow_configurations/AnnotateVcf.SingleBatch.json.tmpl
Outdated
Show resolved
Hide resolved
inputs/templates/terra_workspaces/cohort_mode/workflow_configurations/AnnotateVcf.json.tmpl
Outdated
Show resolved
Hide resolved
wdl/AnnotateVcf.wdl
Outdated
String gatk_docker | ||
|
||
File? NONE_FILE_ | ||
|
||
RuntimeAttr? runtime_attr_svannotate | ||
RuntimeAttr? runtime_attr_concat_vcfs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unused
wdl/AnnotateVcf.wdl
Outdated
RuntimeAttr? runtime_attr_svannotate | ||
RuntimeAttr? runtime_attr_concat_vcfs | ||
RuntimeAttr? runtime_attr_prune_vcf | ||
RuntimeAttr? runtime_attr_shard_vcf |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unused
wdl/ShardedAnnotateVcf.wdl
Outdated
RuntimeAttr? runtime_attr_fix_ends_rescale_GQ | ||
RuntimeAttr? runtime_attr_concat_sharded_cluster | ||
RuntimeAttr? runtime_attr_preconcat_sharded_cluster | ||
RuntimeAttr? runtime_attr_hail_merge_sharded_cluster | ||
RuntimeAttr? runtime_attr_fix_header_sharded_cluster |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All unused
wdl/TasksMakeCohortVcf.wdl
Outdated
@@ -990,6 +990,7 @@ task ScatterVcf { | |||
|
|||
command <<< | |||
set -euo pipefail | |||
~{if !defined(vcf_index) then "tabix ~{vcf}" else ""} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is problematic if the vcf is unsorted. Can you make an explicit input like Boolean generate_index_if_unavailable
that defaults to false but is also required to be true here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removing this line and requiring index in AnnotateVcf as discussed
wdl/PruneAndAddVafs.wdl
Outdated
@@ -4,151 +4,108 @@ | |||
version 1.0 | |||
|
|||
import "TasksMakeCohortVcf.wdl" as MiniTasks | |||
import "ChromosomeAlleleFrequencies.wdl" as calcAF | |||
import "Utils.wdl" as util | |||
|
|||
# Prune off samples in annotated VCF, add VAF annotation | |||
workflow PruneAndAddVafs { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let delete this workflow and just move the tasks into ShardedAnnotateVcf.wdl
Addressed review comments and switched to a single input VCF as discussed to simplify the WDL structure. If the user wants to run a subset of contigs they should provide a subsetted contig_list, and they can submit multiple workflows for multiple subsets of contigs. Re-tested: womtool validation & successful run of AnnotateVcf.wdl on the reference panel. |
Updates
This PR incorporates @xuefzhao's changes to AnnotateVcf to shard the workflow for additional scaling capabilities and add the PAR BED file for AF annotation, as well as additional changes to streamline and clean up the AnnotateVcf workflow. Changes include:
Testing