Supercomputing matters. So does user experience. Standing between the mainstream adoption of supercomputing and a new generation of users is the reality that the entry cost to using these systems, both in terms of dollars and in time spent learning the technology, has not significantly changed in the last 20 years. The rise of cloud computing only complicates the learning curve further. Over the last 6 years, the authors have been addressing this gap through the development of a Science-as-a-Service platform enabling users to go from their desktop, to their local data center, to the cloud, and back without sacrificing their existing tool chain or user experience.
In this tutorial, we combine best practices and lessons learned while on-boarding the last 70k new users to TACC’s data center through the Agave Platform. Participants will walk through the process of scaling their application from a local environment to the Jetstream academic cloud and to a high performance computing system at the Texas Advanced Computing Center. They will learn to use multiple container technologies to harmonize app execution between cloud and HPC resources, and they will learn to use modern APIs to orchestrate job execution, capture provenance information, and foster collaboration.
| Time | Presenterr | Topic |
|---|---|---|
| 08:30 - 08:45 | John, Steve | [Introductions](01%20Introduction.ipynb) |
| 08:45 - 09:05 | Rion | [Agave Overview](02%20Agave%20Overview.pdf) |
| 09:05 - 09:15 | Kathy | [Jupyter, Sanbox, and Logging In](03%20Jupyter%2C%20Sandboxes%2C%20and%20Logging%20In.ipynb) |
| 09:15 - 09:30 | Steve | [Code, Build, and Test](04%20Code%20Build%20and%20Test.ipynb) |
| 09:30 - 10:00 | Rion, John | [Hands on with Agave](05%20Hands%20on%20with%20Agave.ipynb) |
| 10:00 - 10:30 | -- | Break |
| 10:30 - 11:00 | Steve,John | [Docker and Singularity](06%20Docker%20and%Singularity.ipynb) |
| 11:00 - 11:15 | Rion | [Automation an Benchmarking](07%20Automation%20and%20Benchmarking.ipynb) |
| 11:15 - 11:45 | Kathy, Rion | [Packaging, publishing, and Portability](08%20Packaging%20publishing%20and%20Portability.ipynb) |
| 11:45 - 12:00 | Steve, John | [Future Directions and Homework)[09%20Future%20Directions%20and%20Homework.ipynb] |
- 01: Requirements and Preparation
- 02: Installation and Infrastructure
- 03: Auth, Notebooks, and the Web Interface
- 04: SciOps and our Sample Application
- 05: Code, Build, and Run Locally
- 06: Containerize Existing Applications
- 07: Automation, Registries, and App Catalogues
- Agave
- CI/CD
- Image publishing
- Image caching
- Runtime environments
- Data scheduling
- Reproducibility anti-patterns