generated from carpentries/workbench-template-md
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
74525d5
commit 4029d4a
Showing
17 changed files
with
10,615 additions
and
121 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
--- | ||
title: "00-introduction" | ||
teaching: 10 | ||
exercises: 2 | ||
--- | ||
|
||
## Introduction | ||
|
||
This workshop is designed for a beginner with little to now | ||
|
||
By the end of this workshop you will have: | ||
|
||
- Created 16 contianers. | ||
|
||
- Executed 12+ commands with containers. | ||
|
||
- Build 2 custom containers. | ||
|
||
- Build 2 websites and viewed them with containers. | ||
|
||
- Run the most common singularity commands (pull, build, exec) | ||
|
||
- Run the exec command command with bind mounts to access data. | ||
|
||
- Encounter 3 gotchas and discuss solutions to deal with them. | ||
|
||
- Gotcha 1: Repositories are managed by other users. | ||
- Solution 1: Keep good documentiaton on how your software was built. | ||
|
||
- Gotcha 2: Cant mount the current working directory. | ||
- Solution 2: Try the **-B $PWD** flag. | ||
|
||
- Gotcha 3: Cant mount the the directory with the data. | ||
- Solution 3: Try the **-B <my_data_dir_path>:<my_data_dir_path>** flag. | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
--- | ||
title: "01-intro-to-computers" | ||
teaching: 30 | ||
exercises: 0 | ||
--- | ||
|
||
![parts example](episodes/fig/01_computer_parts.svg){alt='A collection of computer parts including disks, ram, cpu'} | ||
|
||
## Computer Component Review | ||
|
||
- **CPU:** CPUs are the data process unit, they are composed of multiple cores. For legacy reasons software often refers the number of cores as the number of CPUs, so yeah that is confusing. | ||
|
||
- **RAM (a.k.a MEMORY):** RAM is fast digital storage. Most programs utilize RAM for access to data needed more than once. RAM is generally non-persistent when the powered off RAM memory is lost. | ||
|
||
- **DISK:** Disk is persistent digital storage that is not as fast as RAM. Disk storage can be made up of one or more disks such as hard drives (HDD) and/or Solid State Harddrives (SSD). Multiple disk can be configured together for increased performance and drive failure protection. | ||
|
||
- **NETWORKING:** Switches and network access cards within computers allow for computers to be networked together. | ||
|
||
- **GPU:** A Graphics Processing Unit (GPU) is a computer component that is capable of rendering graphics, these are also useful for conducting certain mathematical calculations. | ||
|
||
## Consumer Computer vs Servers vs HPC vs Sumhpc | ||
|
||
| Component | Home/Busines Computer | Server | Typical Individual Node in HPC | Typical Total HPC System | Individual Node on Sumhpc | Total Sumhpc System | | ||
|-----------|-----------------------|------------|-------------------------------|--------------------------|---------------------------|---------------------| | ||
| CPU (cores)| 4 - 8 | 12 - 128 | 32 - 128 | 1000s | 70\* | 7,000 | | ||
| RAM(GB) | 8 -16 | 64 - 960 | 240 - 3000 | 64,000 | 754 - 3TB | 76.8 TB| | ||
| DISK (TB)| .5 - 1 TB | 8 - 100 | None - 1 TB | 100s (Networked) | NA | 2.7 PB | | ||
| Networking (Gbe)| .1 - 1 | 1 - 10 | 40 - 100 | 40 - 100 | 40 | 40 + | | ||
|
||
|
||
## Computer Ports | ||
|
||
A port is a communication endpoint. | ||
|
||
## Introduction to OS, Virtual Machines and Containers | ||
|
||
|
||
Why can't I use Docker? | ||
Docker images are not secure because they allow users to gain root access to the compute nodes. Singularity effectively runs as the user running the command and does not result in elevated access. Also, docker interacts with the slurm job scheduler in a way that causes resource requests and usages to not match up, making it difficult to keep job queueing fair for all users. In that the clusters are multi-user systems, we want to make sure people can work without worry that others are accessing their data or unfairly using up resources. | ||
|
||
## Important notes on how they relate to singularity | ||
|
||
-CPU There are 2 common CPU architectures in modern systems x86_64 and ARM. Singularity containers are architecture specific. | ||
```--arch string architecture to pull from library (default "amd64")``` | ||
|
||
### citations | ||
|
||
Lesson adapted from: | ||
|
||
https://github.com/TheJacksonLaboratory/intro-to-hpc | ||
|
||
|
||
https://crc.pitt.edu/singularity | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
--- | ||
title: "02-singularity-commands" | ||
teaching: 30 | ||
exercises: 0 | ||
--- | ||
|
||
|
||
### Fequently used singularity commands | ||
|
||
- **singularity pull** | ||
|
||
- **singularity shell** | ||
|
||
- **singularity cache** | ||
|
||
- **singularity build** | ||
|
||
- **singularity exec** | ||
|
||
**These hands on tutorial are being conducted on a base Centos7 OS VM with the following programs installed.** | ||
|
||
- apptainer (including the alias singularity) | ||
|
||
- nano | ||
|
||
- wget | ||
|
||
- unzip | ||
|
||
**The following ports have been made available to the system:** | ||
|
||
- 8080,8787,8789 | ||
|
||
**The following folders have been setup on the remote system:** | ||
|
||
- /projects/my-lab | ||
- /flashscratch | ||
- /workshop_data | ||
|
Oops, something went wrong.