Dependencies

First steps

Follow the instructions here to install necessary software and to set up git on the HPC.

Accessing the HPC

You can access the HPC through your terminal with the following command:

ssh [username]@hpclogin02.fiu.edu

Remember to replace [username] with your FIU username! E.g., tsalo006, mtobia, alaird.

The terminal will now prompt you for your password. Use the same password you use for your email and MyFIU.

Now you should be on the HPC. You will automatically be logged on in your home directory (i.e., /home/[username]). This is a folder that no one except you and the HPC admins can access. You will put a lot of your work here, including materials for the training.

Set up SSH key for your laptop

This will allow you to SSH onto the HPC without having to type in your password. (Note: the following instructions are adapted from this resource).

First, generate authentication keys for your laptop. In your laptop terminal (i.e., not SSHed onto the HPC):

ssh-keygen -t rsa

Just hit Enter for the filename and password (so there is no password).

Now add your laptop's public key to the list of authorized keys on the HPC:

cat .ssh/id_rsa.pub | ssh [username]@hpclogin02.fiu.edu 'cat >> .ssh/authorized_keys'

Remember to replace [username] with your username!

Now you should be able to SSH onto the HPC without typing in your password.

Clone the training-week-2019 repository to the HPC

Now that you’re on the HPC, you're going to use git (which should be set up now) to clone the training repository to your home directory on the HPC.

First, fork the repository to your GitHub account. Go to the repository page on GitHub. You will see a Fork button at the top-right of the page. Click that button and select your username if it asks which account to fork to. It will take a moment, but the forking process will create a copy of the repository under your account (e.g., tsalo/training-week-2019 instead of FIU-Neuro/training-week-2019).

Now, clone your fork to the HPC. In your terminal (you should still be in your home directory), type git clone git@github.com:[username]/training-week-2019.git. Remember to change [username] to your username!

This will download the repository to your home directory. You should now have a folder in your home directory called training-week-2019 with all of the materials we will use for the training.

Loading environments

You will now edit your .bashrc file, which automatically runs every time you open a terminal on the HPC.

nano .bashrc

This opens the file inside the terminal, so you can edit and save the file without opening a new window.

Add the following lines to the bottom of the file:

# Project-specific environments
source /home/data/cis/training-week-2019/environments

prompt1="\[\e[1;39m\]\u@\h:\W\$\[\e[0m\]"
PROMPT_COMMAND='PS1="\[\e[1;33m\]${project_name}\[\e[0;0m\]${prompt1} "'

These lines will allow you to load environments provided for the training with special commands.

Now save and exit the file:

Ctrl-X
y
Enter

To activate the training environment, you'll need to close out of your current terminal and ssh onto the HPC again. Then, simply type training_env in the terminal to activate the environment. To deactivate the environment, type leave_env in the terminal.

Port forwarding

We will use port forwarding from a specific visualization node for our interactive coding sessions. To ssh onto the visualization node, do:

ssh [username]@v003

Now that you have sshed onto the visualization node, you will open a Jupyter notebook and use port forwarding to access it through your preferred web browser. (Note: the following instructions are adapted from this very helpful resource).

First, activate an environment (e.g., training_env) to gain access to Jupyter notebook.

In the terminal that is accessing the HPC, open a Jupyter notebook without opening the web browser:

jupyter notebook --no-browser --port=[portnumber]

This will show something like the following:

[I 14:53:04.250 NotebookApp] Serving notebooks from local directory: /home/bpank001
[I 14:53:04.250 NotebookApp] 0 active kernels
[I 14:53:04.250 NotebookApp] The Jupyter Notebook is running at: http://localhost:8889/?token=f43fc7ced0cc389e5178f1f5c2757f72517073333f44866b
[I 14:53:04.250 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 14:53:04.251 NotebookApp]
   Copy/paste this URL into your browser when you connect for the first time,
   to login with a token:
       http://localhost:8888/?token=f43fc7ced0cc389e5178f1f5c2757f72517073555f44866b

Remember to replace [portnumber] with your assigned port number.

Now, in a separate terminal that isn't accessing the HPC, start an SSH tunnel:

ssh -N -L localhost:[portnumber]:localhost:[portnumber] [username]@v003.fiu.edu

Remember to replace [portnumber] with your assigned port number and [username] with your username.

You will need to leave this terminal open while using the Jupyter notebook.

Now, open your preferred browser and paste the URL from the first window:

http://localhost:8888/?token=f43fc7ced0cc389e5178f1f5c2757f72517073555f44866b

The browser will now show the Jupyter notebook hub, within which you can open an existing notebook or create a new one.