Skip to content

Latest commit

 

History

History
125 lines (92 loc) · 6.17 KB

README.md

File metadata and controls

125 lines (92 loc) · 6.17 KB
Raw

XAVIER - eXome Analysis and Variant explorER 🔬

tests docs Docker Pulls GitHub Issues MIT license DOI

XAVIER - eXome Analysis and Variant explorER. This is the home of the pipeline, XAVIER. Its long-term goals: to accurately call germline and somatic variants, to infer CNVs, and to boldly annotate variants like no pipeline before!

Overview

Welcome to XAVIER! Before getting started, we highly recommend reading through xavier's documentation.

The xavier pipeline is composed several inter-related sub commands to setup and run the pipeline across different systems. Each of the available sub commands perform different functions:

  • xavier run: Run the XAVIER pipeline with your input files.
  • xavier unlock: Unlocks a previous runs output directory.
  • xavier cache: Cache remote resources locally, coming soon!

XAVIER is a comprehensive whole exome-sequencing pipeline following the Broad's set of best practices. It relies on technologies like Singularity1 to maintain the highest-level of reproducibility. The pipeline consists of a series of data processing and quality-control steps orchestrated by Snakemake2, a flexible and scalable workflow management system, to submit jobs to a cluster or cloud provider.

The pipeline is compatible with data generated from Illumina short-read sequencing technologies. As input, it accepts a set of FastQ or BAM files and can be run locally on a compute instance, on-premise using a cluster, or on the cloud (feature coming soon!). A user can define the method or mode of execution. The pipeline can submit jobs to a cluster using a job scheduler like SLURM, or run on AWS using Tibanna (feature coming soon!). A hybrid approach ensures the pipeline is accessible to all users.

Before getting started, we highly recommend reading through the usage section of each available sub command.

For more information about issues or trouble-shooting a problem, please checkout our FAQ prior to opening an issue on Github.

Dependencies

Requires: singularity>=3.5 snakemake==6.X

Snakemake and singularity must be installed on the target system. Snakemake orchestrates the execution of each step in the pipeline. To guarantee the highest level of reproducibility, each step relies on versioned images from DockerHub. Snakemake uses singaularity to pull these images onto the local filesystem prior to job execution, and as so, snakemake and singularity are the only two dependencies.

Run XAVIER pipeline

Biowulf

# XAVIER is configured to use different execution backends: local or slurm
# view the help page for more information
module load ccbrpipeliner
xavier run --help

# @slurm: uses slurm and singularity execution method
# The slurm MODE will submit jobs to the cluster.
# It is recommended running XAVIER in this mode.

# Please note that you can dry-run the command below
# by providing  --runmode dryrun flag

# Do not run this on the head node!
# Grab an interactive node
sinteractive --mem=110g --cpus-per-task=12 --gres=lscratch:200
module load ccbrpipeliner

# First, initialize the output directory
xavier run \
--input data/*.R?.fastq.gz \
--output /data/$USER/xavier_hg38 \
--genome hg38 \
--pairs pairs.txt \
--targets resources/Agilent_SSv7_allExons_hg38.bed \
--mode slurm \
--runmode init

# Second, do a dry run to visualize outputs
xavier run \
--input data/*.R?.fastq.gz \
--output /data/$USER/xavier_hg38 \
--genome hg38 \
--pairs pairs.txt \
--targets resources/Agilent_SSv7_allExons_hg38.bed \
--mode slurm \
--runmode dryrun

# Then do a complete run
xavier run \
--input data/*.R?.fastq.gz \
--output /data/$USER/xavier_hg38 \
--genome hg38 \
--pairs pairs.txt \
--targets resources/Agilent_SSv7_allExons_hg38.bed \
--mode slurm \
--runmode run

FRCE

# grab an interactive node
srun --export all --pty --x11 bash

# add xavier to path correctly
. /mnt/projects/CCBR-Pipelines/pipelines/guis/latest/bin/setup

# add SIF cache and TMPDIR path
SIFCACHE="/mnt/projects/CCBR-Pipelines/SIFs/XAVIER"
TMPDIR="/scratch/cluster_scratch/$USER"

# run xavier

# Initialize and then dryrun (or run)
xavier run \
--input data/*.R?.fastq.gz \
--output /data/$USER/xavier_hg38 \
--genome hg38 \
--sif-cache $SIFCACHE \
--tmp-dir $TMPDIR \
--pairs pairs.txt \
--targets resources/Agilent_SSv7_allExons_hg38.bed \
--mode slurm \
--runmode init # run

Contribute

This site is a living document, created for and by members like you. XAVIER is maintained by the members of CCBR and is improved by continuous feedback! We encourage you to contribute new content and make improvements to existing content via pull request to our repository.

References

1. Kurtzer GM, Sochat V, Bauer MW (2017). Singularity: Scientific containers for mobility of compute. PLoS ONE 12(5): e0177459.
2. Koster, J. and S. Rahmann (2018). "Snakemake-a scalable bioinformatics workflow engine." Bioinformatics 34(20): 3600.