Skip to content

SC17 tutorial - "HPC via HTTP: Portable, Scalable Computing using App Containers and the Agave API"

License

Notifications You must be signed in to change notification settings

agaveplatform/SC17-container-tutorial

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

HPC via HTTP: Portable, Scalable Computing using App Containers and the Agave API

Supercomputing matters. So does user experience. Standing between the mainstream adoption of supercomputing and a new generation of users is the reality that the entry cost to using these systems, both in terms of dollars and in time spent learning the technology, has not significantly changed in the last 20 years. The rise of cloud computing only complicates the learning curve further. Over the last 6 years, the authors have been addressing this gap through the development of a Science-as-a-Service platform enabling users to go from their desktop, to their local data center, to the cloud, and back without sacrificing their existing tool chain or user experience.

In this tutorial, we combine best practices and lessons learned while on-boarding the last 70k new users to TACC’s data center through the Agave Platform. Participants will walk through the process of scaling their application from a local environment to the Jetstream academic cloud and to a high performance computing system at the Texas Advanced Computing Center. They will learn to use multiple container technologies to harmonize app execution between cloud and HPC resources, and they will learn to use modern APIs to orchestrate job execution, capture provenance information, and foster collaboration.

Preview

Intro Video

Schedule

Time Presenterr Topic
08:30 - 08:45 John, Steve [Introductions](01%20Introduction.ipynb)
08:45 - 09:05 Rion [Agave Overview](02%20Agave%20Overview.pdf)
09:05 - 09:15 Kathy [Jupyter, Sanbox, and Logging In](03%20Jupyter%2C%20Sandboxes%2C%20and%20Logging%20In.ipynb)
09:15 - 09:30 Steve [Code, Build, and Test](04%20Code%20Build%20and%20Test.ipynb)
09:30 - 10:00 Rion, John [Hands on with Agave](05%20Hands%20on%20with%20Agave.ipynb)
10:00 - 10:30 -- Break
10:30 - 11:00 Steve,John [Docker and Singularity](06%20Docker%20and%Singularity.ipynb)
11:00 - 11:15 Rion [Automation an Benchmarking](07%20Automation%20and%20Benchmarking.ipynb)
11:15 - 11:45 Kathy, Rion [Packaging, publishing, and Portability](08%20Packaging%20publishing%20and%20Portability.ipynb)
11:45 - 12:00 Steve, John [Future Directions and Homework)[09%20Future%20Directions%20and%20Homework.ipynb]

Table of Contents

  • Agave
  • CI/CD
  • Image publishing
  • Image caching
  • Runtime environments
  • Data scheduling
  • Reproducibility anti-patterns

About

SC17 tutorial - "HPC via HTTP: Portable, Scalable Computing using App Containers and the Agave API"

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •