Skip to content

MASSIVE and HPC clusters

Jump to bottom
magnesium2400 edited this page Aug 7, 2023 · 13 revisions

Introduction

As computational neuroscientists, the work we do can get computationally demanding. To facilitate this, we (and many other labs) will use what is called a High-Performance Computing (HPC) cluster.

Our cluster is called MASSIVE, and it consists of many interconnected nodes which function similar to your laptop or desktop PC, but at a much larger scale.

MASSIVE is a cloud computing solution. This means that we connect to MASSIVE and its desktops remotely, so you won’t need a powerful computer to access MASSIVE and it can be accessed anywhere in the world. You may however need a reliable internet connection in order to access the remote desktop without much latency. Similarly, nothing you save on MASSIVE will be saved on your personal computer, so you may need to transfer relevant files if necessary.

Getting Started with MASSIVE

You should read the guidelines and talk to Chao about creating a MASSIVE account.

The Strudel web client (currently in beta) is the simplest method of opening a desktop and what we recommend for beginners.

Step-by-step guide

  1. Go to https://beta.cloud.cvl.org.au/login

  1. Select CVL and click Login
  2. Sign in via AAF and use your regular Monash credentials
  3. Click Desktop on the left

  1. Under the ‘Launch Desktop’ heading, choose a desktop to use - for beginners we suggest P4 (or alternatively Single T4). Click Launch.

  1. Under ‘Pending/Running Desktops’ click connect, and your MASSIVE session should open in a new tab.

  2. When you sign in, you will see the MASSIVE desktop

You can see a handy little ‘time left’ box. You can also see the shortcut bar with ‘Applications’, ‘Places’, ‘System’, and three icons for the file explorer, the terminal, and the Firefox internet browser. You should click on each of these icons to see what they do.

You will end up getting very familiar with the terminal. You can also right-click on the desktop (or in the file explorer) and use the ‘Open a Terminal’ option to open a terminal.

Advanced Use

Choosing a Desktop and Login Node

To launch a desktop within Strudel, you must first select which GPU node you wish to use. These nodes vary in how much memory is available, ranging from the P4 and Single T4 light compute nodes, to the more powerful Dual T4 and A40 heavy compute nodes.

Each node has a login queue based on the availability of resources, with the heavy compute nodes sometimes taking longer to connect to. If you find you’re waiting in the queue for ages, try connecting to a different node and let someone in the team know if you’re still struggling to access a desktop

MASSIVE Modules

Software on MASSIVE is downloaded and stored as modules that can be loaded and unloaded whenever you need them.

If you haven't loaded the appropriate module first, you will be unable to use that software and your commands will fail

See here for a list of installed MASSIVE modules

Here are some useful commands for handling modules on MASSIVE:

To load your chosen module module load [module name]

To unload your chosen module module unload [module name]

To list all currently loaded modules module list

To unload all currently loaded modules module purge

To see basic information for your chosen module, including version numbers installed on MASSIVE module show [module name]basic information for your chosen module, including version numbers installed on MASSIVE

VirtualGL

This is a graphics library for Unix that is needed when running some software on a virtual machine like your MASSIVE desktop

Many visualisation tools will fail if you haven’t loaded the virtualgl module and added ‘vglrun’ before your command

  • e.g., vglrun freeview […]

This detail isn’t included in some of the attached tutorials on external websites or their example code, so make sure to include this where necessary when running code on MASSIVE

Clone this wiki locally