Follow Relion Gracefully 🔬🚀🌐

---

v5: An almost complete dashboard for easy interaction with your cryo-EM data in Relion, now with partial ✨ #teamtomo ✨ support!

• Data sourced from Relion5 tutorial and Relion4 STA
• Licensed under Non-Profit Open Software License 3.0 (NPOSL-3.0)

FollowRelion1.mp4

✨ Found this helpful in your research? Cite my work! ✨

Dawid Zyla. (2024). dzyla/Follow_Relion_gracefully: v5 (Version v5). Zenodo. https://doi.org/10.5281/zenodo.10465899

---

Description 🔬

v5: 🔆

Version 5 improves the job preview by adopting a dynamic approach. Using Streamlit, it allows users to interact directly with their data. The underlying Python framework facilitates real-time computation of statistics and data from most jobs, enabling users to engage with metadata and select preferred statistics for download and further analysis.

v4:

Version 4 introduced support for multiple projects and job visualization through an online interface using the Hugo framework. While this static job generator enabled job display with example data, it lacked interactive capabilities due to its static nature.

v5 features 💫

• Python script recognizes default_pipeline.star (only Relion 4 and 5 supported) and generates job previews, enhancing data understanding and quality access without opening individual files.
• Streamlit simplifies the setup (no extra software downloads needed) and is operated through a single file follow_relion_gracefully.py.
• ✨ (New) Relion Live dashboard: Monitor live Relion sessions and select MotionCorr/CtfFind statistics. This functionality is inspired and partially based on CNIO_Relion_Tools.
• ✨ (New) Experimental support for running Relion in-browser*! Access and operate all Relion programs directly via a web interface. Currently a Linux-only technical demo, but functioning well.
• ✨ (New) Job selection displays rejected particle locations (both SPA and STA), aiding in data quality assessment.
• ✨ (New) Downloadable results: Access all final MRC and PDF files directly, eliminating the need for network drive mounting.
• ✨ (New) Volume slider: Interactively adjust volume thresholds.
• ✨ (New) Enhanced picking job support: Interact with all micrographs and review picking statistics.
• ✨ (New) View your job flowchart in-browser. While not aesthetically pleasing, it is informative.
• ✨ (New) Interactive metadata plotting and selection tool: Choose data directly from plots and download selected STAR files. Upload your own STAR files to customize statistics.
• ✨ (New) ModelAngelo support: Visualize final protein structures with your maps in-browser.
• ✨ (New) Direct 2D class selection in-browser.
• ✨ (New) Increased security with password-protected dashboard access.
• Multi-platform support (Windows, Linux, Mac OS [untested, but expected to work similarly to Linux]).
• Multi-Project Support 🔀 - Manage all your Relion data in one place. Generate detailed plots and processing statistics from multiple projects effortlessly. Functions well as a Relion electronic notebook📓.
• Partial support for Relion 4 Tomography workflow #teamtomo
• Code rewritten for enhanced speed and stability. #ChatGPT #GPT4
• Publication-ready figures (FSC, class projections, and angular distribution plots).
• Monitoring capabilities for Select and Extract processes: visualize and quantify selected/extracted particles.
• Dark mode🌜: Reduce eye strain during night shifts, enabled via Streamlit settings.
• #OpenSoftwareAcceleratesScience

* not acually running in the browser but allowing to start the job that runs on Linux workstation

FollowRelion2.mp4

FollowRelion3.mp4

Installation 🚀

Minor changes from v4, with a few new libraries added. Tested on Windows 10/11, WSL2, CentOS, and Ubuntu 22.04.

Install Dependencies 🐍

Install dependencies in a conda environment, as Python 3.11 is required and virtual environments are no longer supported (though they might still work).

Conda Instructions

1. Install miniconda3 (no root access required, only if not installed already):

wget -q  -P  .  https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

bash ./Miniconda3-latest-Linux-x86_64.sh -b -f

Activate conda for bash:

conda init bash

Restart the shell or type bash to see the (base) prompt:

(base) dzyla@GPU0

2. Clone the GitHub repository and navigate to the folder:

git clone https://github.com/dzyla/Follow_Relion_gracefully.git

cd Follow_Relion_gracefully

3. Create a conda environment and install dependencies using the environment.yml file:

conda env create --file environment.yml

conda activate FollowRelion

You should now see:

(FollowRelion) dzyla@GPU0

✨Ready to start! ✨

Usage 💻

streamlit run follow_relion_gracefully.py

Additional command line parameters for extra features:

-h, --help            Show this help message and exit.
--i I, --folder I     Path to the default folder.
--path PATH, --relion-path PATH
                        Path to the RELION bin folder.
--p P, --password P   Password for securing your instance.

Example for live server updates and setting up a new project:

To use command line parameters with streamlit, add -- before the parameters:

conda activate FollowRelion

streamlit run follow_relion_gracefully.py -- --p MyPassword$221#& --i /mnt/staging/240105_NewProcessing --path /usr/local/relion5/build/bin/

This sets a password, default processing folder, and specifies the path to Relion executables.

Accessing the Dashboard via Browser 📈

The dashboard should open automatically in your browser. For remote workstations, access it using the provided network URL, ensuring the port is not firewall-blocked.

Remote access example:

(FollowRelion) dzyla@PC-HOME:~/Follow_Relion_gracefully$ streamlit run follow_relion_gracefully.py --server.port 8501 -- --p 1234 --i /mnt/f/linux/Tutorial5.0/

  Local URL: http://localhost:8501
  Network URL: http://172.21.222.176:8501

Open the network URL in your browser to access the dashboard.

For firewall issues, create an SSH tunnel:

ssh -f username@workstation -L 8501:localhost:8501 -N

This allows remote dashboard access on your local computer: http://localhost:8501.

Hosting a GitHub Page 🌐 (Deprecated)

The real-time data calculation in the new implementation makes pre-calculated server hosting obsolete. Advanced users can still use tools like ngrok for remote dashboard access.

Troubleshooting 🔧

• Issues with certain parameters causing errors? Start a new GitHub issue with the problematic STAR files.
• Large volumes (500px+) load slowly, especially for multiple class-3D classifications. Downloading them is suggested.
• Ensure the correct environment is activated (FollowRelion). Deactivate others with conda deactivate.
• Jobs run manually may not be processed, as the script reads from default_pipeline.star. The exception is the Relion Live dashboard.
• Rendering issues in the browser can often be resolved by refreshing (F5).
• Import job previews and ice thickness calculations in Relion Live may be slow.
• Mac support is untested, but it's assumed to work similarly to Linux. Please report any issues!
• Not all Tomo jobs are currently supported, but future updates are planned.
• But I liked Hugo-based implementation, you might think. No worries; after cloning the GitHub repo, run git checkout v4.
• Please note that this code was developed by a Python enthusiast, not a professional developer. It has been tested under standard scenarios to ensure reliability. However, as the author, I cannot be held responsible for any issues or damages that may arise from its use. Users are encouraged to review and test the code thoroughly before implementation in their projects.

To-do 📝

• Preview of the remaining Tomo jobs (need access to fully calculated project) ❔

• Preview of non-default jobs (Multi-Body, External, Particle subtractions, DynaMight) ❔

• Better volume preview ✅

• Better data visualization, more statistics, everything publication-ready ✅

• Possibility to download volumes (ala cryoSPARC) (Is it really necessary?) ✅

• Job flow chart overview. Who is father of whom and which jobs are realted. ✅

• Can you run Relion via static website generator? (probably not) ✅

• Optimization of speed and RAM usage ❔

• Selection of jobs inside given project ❔

• Add job templates for running Relion via browser (basically copying Relion GUI)

• A way to modify and write jobs to default_pipeline.star

• Relion Live job creation (New Movie->Import->MotionCorr->CtfFind->update dashboard->Repeat)

• Add manual picking job that would save picks for further use.

Questions/suggestions?:email:

Dawid Zyla, La Jolla Institute for Immunology

Twitter

dzyla@lji.org