bcbio priorities

[naumenko-sa]

Long TODO lists don't get done.
We are trying to keep an issue open if a real effort is happening to resolve it.
The below priority list gives a sense of where bcbio is heading in the next two releases.

Issues from the lower circles are re-opened and pushed to circle one,
when there is bandwidth to work on them.

Contributions are welcome to rescue an issue from any circle!

Closed issues and implemented features are described in the release notes.

Priority circle 1 ~ ~ release 1.3.0 ~ Q2 2022

• update transcriptome gtf from ensembl 94 to ensembl 104 - TSL1-3

Priority circle 2

• WGBS: + qualimap bamqc
• somalier for detecting mislabelled samples
• more tests of bcbio-se.R
• output a JSON file describing output files in the final for downstream uploader systems (solvebio)
• test and implement output of BigWig files in RNA-seq and ATAC-seq project (motivation: not everyone can store all bam files)
• add low level (barcodes, %rRNA) QC to single cell pipeline
• explore implementation of ichorCNA
• make consistent use of conda/mamba for bcbio_nextgen_install.py and bcbio_nextgen.py upgrade
• make an option "'--very-sensitive" instead of "--very-sensitive-local' in chip-seq. see Update softclipping for chipseq #3476
• --funsegmentation GATK4 in purecn.R for exomes
• CWL Debugging a CWL job on GCP #2904 feature request: cwl - detect default bcbio_system.yaml #2715 Features that are available in CWL #2872 bcbio CWL generator fails with: ValueError: Did not find variable config__algorithm__fusion_caller in #3471
• better gatkcnv support - cnvcalling with PON, gene panels seq and WES gatk-cnv failed in a T/N project - zero median coverage #3227
• Variant calling in RNA-seq - validation Variant calling on RNA-seq data using GATK4 #3161
• Multiqc documentation improvement ; related: duplicate metrics from bamsormadup #3075
• Latest STAR release 2.7.0d requires genome index rebuild Latest STAR release 2.7.0d requires genome index rebuild #2693 (discuss)
• Dfkz filter local/bcbio/bin/dkfzbiasfilter.py TypeError: unsupported operand type(s) for +: 'NoneType' and 'str' #3202
• Mosdepth index files missing mosdepth index files missing? #3124
• Validation of ALT contig calls hg38 validation - proper use of BWA alt index - missed variants in 641 genes? #3069
• multiqc for T/N
• cfDNA calling: test and integrate ichor RFE: handle circulating tumor DNA (ctDNA) copy number analysis #2254
• SV validation
• SV annotation Resources for SVs of normal populations in order to prioritize tumor-only variant calls #963
• Integrate Gnomad SV data and methods
• DNA/RNA variant calling
• T/N RNA-seq somatic variant calling with RNA-seq data  #2521 RNA-seq variant calling on cancer samples #3023
• update germline & somatic validation validation 2020 #3038 Evaluate updating to GATK 4.1.3.0 #2921 use data from https://www.biorxiv.org/content/10.1101/2021.02.27.433136v1.full.pdf
• PON support for mutect2 Somatic variant analysis questions #3254

Priority circle 3

new functionality

• Add passing in known fusions Add passing in known fusions #3030
• GATK4 GermlineCNVCaller GATK4 GermlineCNVCaller #2245
• AWS batch Feature request: run bcbio on awsbatch #3207 aws batch  #3022
• make deep variant work in bcbio make deep variant work in bcbio #3187 Deepvariant multi sample best practices  #3012
• Duplicates % from bamsormadup in QC
• Unaligned bam/cram (GI)
• Support for projects where samples are added to cohort Improving bcftools stats performance on large variant calling VCF files #3036 Running analysis from GVCF level with bcbio to get updated results for expanding population of samples #2336
• Small RNA-seq up and running
• Switch to Salmon selective alignment method Switch to Salmon selective alignment method #3166
• Mosiac chromosomal alterations Mosiac chromosomal alterations #3113
• miRNA pipeline miRNA pipeline #3103
• validate and improve use of Octopus Enhancement proposal: Octopus #1591
• document and improve use of GRIDSS for genomic rearrangements Feature request: gridss SV caller #1829
• Integrate vcf2maf Feature request: integrate vcf2maf #2293
• call repeat expansions RFC: repeat expansions #2797
• interchromosomal translocations with delly delly output from bcbio does not include translocations #2793
• RFE: svprioritize for multiple lists RFE: svprioritize for multiple lists #2325
• Including piRNA quantification in smallRNAseq pipeline Including piRNA quantification in smallRNAseq pipeline #2439
• vcfanno and gemini for RNA-seq variants vcfanno and gemini don't work in RNA-seq analysis? #2511
• Single-cell pipeline drops FASTA spike-ins if not defined in GTF file Single-cell pipeline drops FASTA spike-ins if not defined in GTF file #2387
• Support for Smoove/Lumpy population SV calling FeatureRequest: Support for Smoove/Lumpy population SV calling #2652
• Option to add -I -I flag in Arriba Option to add -I -I flag in Arriba #3018
• Are there any Cut & Run analysis specific parameters implemented in bcbio? Are there any Cut & Run analysis specific parameters implemented in bcbio? #3037
• mapDamage on bcbio mapDamage on bcbio #2867
• alignment-only RNA-seq pipeline Run Mapping alone for RNASeq data using bcbio #3239 (discuss)
• log running time of individual tools Add more analysis in the completed run #3253
• support rsem to re-run older projects RSEM not being executed #3276
• consider strelka --rna for RNA-seq variant calling Providing additional options (--rna) to strelka2 #3319
• productionalize purity/ploidy analysis PURPLE: purity & ploidy estimator support in bcbio #2532
• LoFreq New tools for CNV calling and low-frequency somatic calling #3322

Priority circle 4

Technical debt

The issues below don't break production, however, it is good to have them solved at some point.
Contributions welcome - pick an issue, ask to re-open the issue if you are working on it.
Some of them are good for the first contribution to bcbio.

• refactoring of picard usage can't make use of Picard tools via gatk3 #3553
• VEP annotations can have a SOMATIC field and this breaks vcf2db parsing Heads-up: VEP annotations can have a SOMATIC field and this breaks vcf2db parsing #2966
• conflicting annotations from VEP and gnomad Heads up: conflicting annotations from VEP and gnomad #2945
• Using DP to extract somatic AF in strelka2.py might be wrong Using DP to extract somatic AF in strelka2.py might be wrong #2836
• Identical miRNA read counts versus multimapping Identical miRNA read counts versus multimapping #2893
• Configuration check should not allow for duplicated "description" items Configuration check should not allow for duplicated "description" items #2738
• update citations promoting tool citation #2125
• test tools against python3.7 and switch bcbio to python3.7 with dataclasses
• clinvar UnicodeDecodeError: pybedtools issue clinvar UnicodeDecodeError: pybedtools issue #3078, waiting for pybedtools 0.8.2
• CnvKit - 0.9.7b issues CnvKit - 0.9.7b issues #3201 - for new cnvkit 0.9.7 release
• Error in "Getting Started" instructions Error in "Getting Started" instructions #3162
• Remove Sailfish support Remove Sailfish support #3157
• Speed up installation of third party tools Speed up installation of third party tools #3238 *related to *
• Make installation and upgrading more robust Make installation and upgrading more robust #3191
• TypeError in test_atacseq TypeError in test_atacseq #3225
• FileNotFoundError in test_chipseq FileNotFoundError in test_chipseq #3224
• Allow custom genome installation with --no-data Allow custom genome installation with --no-data #3226
• ValueError in test_variantcall_2 ValueError in test_variantcall_2 #3218
• Value error in test_detect_fusions_with_ericscipt_with_disambiguate Value error in test_detect_fusions_with_ericscipt_with_disambiguate #3220
• ValueError in test_detect_fusions_with_ericscipt_without_disambiguate ValueError in test_detect_fusions_with_ericscipt_without_disambiguate #3219
• ValueError in test_variantcall_1 ValueError in test_variantcall_1 #3217
• ValueError in test_fusion ValueError in test_fusion #3216
• Error running bcbio_nextgen.py version sub-command Error running bcbio_nextgen.py version sub-command #3209
• Upgrade to Ubuntu 20.04 LTS Upgrade to Ubuntu 20.04 LTS #3192
• Move Docker tests out of the bcbio-nextgen repo Move Docker tests out of the bcbio-nextgen repo #3176
• Lock conda environments Lock conda environments #3174
• Avoid editing the source code to keep the work directory between test runs Avoid editing the source code to keep the work directory between test runs #3155
• Remove legacy AWS setup documentation and code Remove legacy AWS setup documentation and code #3134
• Remove legacy python2 code (see Remove Python 2 compatibility modules #3247 (comment))
• Drop the incorrect genoype attributes for decomposes SNP's. Drop the incorrect genoype attributes for decomposes SNP's.  #1334
• test cancermulti fails because strelka wants /usr/bin/env python2 test cancermulti fails because strelka wants /usr/bin/env python2 #3019
• Don't require a dummy genome when transcriptome_fasta and transcriptome_gtf are specified Single cell for other species #3264

Reference issues

Closed issues where no work is needed but which contain some valuable information (validations, great expertise and discussion). for better visibility.

• Validations in GRCh37 w and without decoy + RR Would it possible to add decoys to GRCh37 reference? #2489
• Pipeline standardization Pipeline standardization #2260
• Large DEL validations with HuRef benchmark Validation of large deletions calling in bcbio with HuRef benchmark compared to ERDS and CNVNator #2313
• variant annotation for WGS WGSA annotation for noncoding variants in WGS studies #2587
• RFC: Tumor purity/ploidy estimation methods with targeted panels RFC: Tumor purity/ploidy estimation methods with targeted panels #2639
• 'add_log2_depth' issue during CNV analysis (CNVkit) 'add_log2_depth' issue during CNV analysis (CNVkit) #3004
• Which variant callers are compatible to TitanCNA? Which variant callers are compatible to TitanCNA? #2975
• Support for lossy CRAM in bcbio WGrS variant calling pipeline Support for lossy CRAM in bcbio WGrS variant calling pipeline #2531
• RFE: Use germline resources in MuTect2 to reduce artifacts in tumor-only mode RFE: Use germline resources in MuTect2 to reduce artifacts in tumor-only mode #2873
• Encoding errors with CIVIC svprioritize and IPython distributed tasks Encoding errors with CIVIC svprioritize and IPython distributed tasks #2890
• a hint on somatic variants prioritization Gemini annotation by bcbio missing some boolean columns #3418

[roryk]

If anyone wants to tag in and work on any of this stuff, we are happy to help orient.

[lbeltrame]

cfDNA calling: test and integrate ichor

I can help with this, as I've used the package extensively (200+ samples so far). It would probably need some tweaking for low fraction samples.

On cfDNA front it might be worth investigating in silico size selection, which in my tests improves sensitivity a lot. The rationale is explained in https://pubmed.ncbi.nlm.nih.gov/30404863/, which describes a dual approach (both during library preparation and during analysis), but in my experience just filtering BAMs by query size length helps. I have proof-of concept code that does this step, which we use internally.