Demo: MTBLS1 on NMRbox
This demo is meant to get you up and running with SAFER (Spectral Annotation by Feature Extraction and Recovery) on NMRbox so you can annotate your studies on Metabolights, but a few quick steps will allow you to run just about any standard 1H1D dataset with > ~ 10 samples. It takes you from a basic spectral matrix, through parameter file and NMRBox environment setup, running the R pipeline, and browsing the results.
At the end, you'll have:
- a working parameter file
- extracted and quantified, statistically supported features
- predicted locations of the features in both reference and dataset spectra
Thanks to Chris Esselman of Art Edison's Lab at University of Georgia for helping us get this running on NMRbox! We are continuously developing SAFER, and appreciate your patience and reporting should you encounter bugs or want to suggest features. Collabs are welcome.
The data we'll be using is from Salek et al. (2007) which compares human urine (Type 2 Diabetes vs. control, male and female). It is available under study name MTBLS1 here. However, we have processed the data for use in SAFER. There were several data types submitted for this study; we will use noesypr1d 1r files (i.e. processed in Topspin). It has ~130 spectra, plenty of pH-induced peak shifting, and some overlapped regions as well. Additionally, it has author-submitted annotations that we can check later.
SAFER requires a specifically formatted reference library file. (Note: it is simple to make for any spectra, but beyond the scope here). We'll use the GISSMO 700 MHz simulated database slice, data.list_700MHz_mtbls1_annots.RDS for this run, since MTBLS1 was collected at ~700 MHz.
I like to use the following structure in my workspace:
Suppose your user name on NMRbox is username
Make the following folders
/home
└── nmrbox
└── username
└── Documents
└── safer
├── results
├── params
├── matrices
└── libraries
Opening terminal from your home screen, and running the command
mkdir -p Documents/safer/{results,params,matrices,libraries}
will accomplish this.
This makes it easy to loop through results, params, etc., and so my input files can easily be reused. However, there is a lot of flexibility in where things come from/go depending on how you set up the paths in your parameters file (see below or here).
Upload/transfer the files above into the following locations on your NMRbox workspace: data.list_700MHz_mtbls1_annots.RDS -> username/Documents/safer/libraries MTBLS1_1r_noesypr1d_spectralMatrix.RDS -> username/Documents/safer/matrices
Using VNC Viewer file transfer
Click "Send files...". A file selector window will open on your machine. Select the file to upload, hit Enter. Then the following will show up on NMRBox:
Note the "Fetch files to..." field in the window. You can change this. Currently, it's going to the active file browser folder on Box.
Move to ./matrices.
For this run, paste the following into a text file called params_saferDemo.yaml:
params_saferDemo.yaml
dirs: temp: /home/nmrbox/mjudge/Documents/safer/results study: id: MTBLS1 spectrometer.frequency: 700.0 files: spectral.matrix: /home/nmrbox/mjudge/Documents/safer/matrices/MTBLS1_1r_noesypr1d_spectralMatrix.RDS lib.data: /home/nmrbox/mjudge/Documents/safer/libraries/data.list_700MHz_mtbls1_annots.RDS corrpockets: half.window: 0.06 noise.percentile: 0.99 only.region.between: ~ rcutoff: 0.7 storm: correlation.r.cutoff: 0.8 q: 0.01 b: 1.5 number.of.plots: 250 tina: bounds: - -1.0 - 11.0 min.subset: 5.0 prom.ratio: 0.3 do.clustering: no clustering: max.eps: 50 minPts: 2 eps.stepsize: 0.01 nfeats: 10000 plots: max.plots: 600 filtered.out: no filtered.features: no cleaned.clusters: no matching: cluster.profile: representative.feature ref.sig.SD.cutoff: 0.01 max.hits: 5 r.thresh: 0.8 p.thresh: 0.01 filtering: res.area.threshold: 0.25 ppm.tol: 0.1 max.backfits: 1E7 par: ncores: 16 type: FORK galaxy: enabled: no opts: npoints: 32000 debug: enabled: no throttle_features: 100 all.outputs: yes
Key Parameters to Pay Attention To
dirs:
temp:
This is where the results file will be written to for this run. The results will be in an automatically named directory in the provided location.
files:
spectral.matrix:
This is your MTBLS1 data file.lib.data:
This is your GISSMO library file.
storm: correlation.r.cutoff: 0.8 is usually okay. Avoid going lower than 0.7.
matching:
r.thresh:
0.8 is usually okay. Avoid going lower than 0.7.filtering:
ppm.tol:
PPM tolerance for matches; 0.1 ppm = +/- 0.1 ppm.max.backfits:
Raising this gives better results, but requires more compute. Keep this around 10^5 - 10^7 for starters.
par: ncoresThis is the number of CPUs to parallelize across. On NMRbox, 16 is usually okay. If you have more compute, you can increase the number for roughly linear time savings, but RAM demands will increase as well.
opts: npointsThis is the number of points to interpolate the data to. Some datasets have way more spectral points than needed, like 64k or 128k. 32k is usually enough, and reducing this results in big time savings. Going below 16k is not recommended.
Make sure you change username to your actual username. You can either create this file on NMRbox, or make locally and add to NMRbox:
params_saferDemo.yaml -> username/Documents/safer/params
Check to make sure the filepaths in your params file on NMRbox actually correspond to the directories things will be read from/written to! A good practice is to always copy the paths directly from the file browser or terminal to avoid typos etc.
In the top left corner, click this icon:
Search for Github Desktop:
Note: if prompted for a keyring, just hit 'Cancel'. Github will still open.
image
Follow the steps here
Record the path to your clone of the github repo on NMRbox.
We maintain a Docker Image from which a Singularity Image is built on your workspace (it will remain there, so only need to run the first time, or when updating):
Open the Terminal Emulator in your NMRbox workspace (home folder).
Type:
singularity pull docker://mtbls/safer-nmr:latest
Next, open an R session in the terminal:
singularity exec safer-nmr_latest.sif R
You should see something like this:
Load the R package in the R session. Recall your Github clone path from above and replace 'path/to/local/safer/repo' with that. Run the command in the terminal - this makes all the SAFER functions available:
library(SAFERnmr)
Using a current branch directly from Github:
If you want to use the most current version of the repository on any branch (instead of using the Docker version), you can do so by calling
devtools::document('path/to/local/safer/repo')instead oflibrary(SAFERnmr)
It is normal to see a couple warnings at this point.
Almost done! Run the pipeline command (assuming your parameter file is set up and saved!):
pipeline(params_loc = '/home/nmrbox/username/Documents/safer/params/params_saferDemo.yaml')
Visualize your results on the NMRbox browser or download the results directory, unzip, start the R session, rebuild the package, and run locally.):
browse_evidence('path_data_directory')
You will see an address reported in the terminal. Highlight, right click, and select "Open Link" (you won't have to do this if running locally).
You should see the results viewer like this:
To get better resolution, you can click the icon in the upper left corner and type to search 'Resolution Changer', and select:
The dimensions can then be increased:
Search for your metabolite of interest (click the , or browse the heatmap to see high-scoring metabolites. The heatmap shows the score for each metabolite (rows) in each sample (columns). Clicking a point on the heatmap selects that metabolite.
Note: Click and drag to draw a box for zooming/selecting. If you click and drag directly horizontal, only the horizontal axis will be zoomed. Same for the vertical axis. A tutorial on zooming/selecting on a plotly chart can be found here.
Once a metabolite is selected from the search bar or heatmap, the metabolite reference spectrum will be displayed in the left upper left panel, and the sample-specific scores will be plotted in the lower right panel. First, horizontal zoom to include all reference peaks (or peaks of interest. Then use the selection box to select the peaks. Next, in the scores plot, use the selection tool to select one or more high-scoring samples. The default is to display in the lower left panel the positions on the reference spectrum where features matched. There are buttons to toggle between the two currently available plot types: Features on Reference (shows the alignment of features to the reference region being displayed), and Ref-Features on Samples (shows the matched reference regions, fit to your dataset spectra).
To download the data, I find it easiest to right click the dataset in your results directory, select Create Archive (to zip the directory), then use the following rsync command to update a local directory on your machine to match the zip files on the NMRbox directory.
rsync -avz --progress --include '*/' --include '*.zip' --exclude '*/' <username@element.nmrbox.org:~/Documents/safer/results/> <mymachine/.../pipeline_runs_new>
Note: Remove the <> and replace with your directories. In "username@element.nmrbox.org", replace 'username' with your username and 'element' with one of the available VMs on NMRbox (e.g. 'helium'). You will be prompted for your NMRbox password to grant rsync access to your directory.