Turnkey epidemic system

[jim-sheldon]

As a programmer supporting research I would like to push a button that creates a new system (infrastructure, services, etc.) to accommodate any new epidemics. Similar to our turnkey partner infrastructure, we could specify some values in a configuration file (like S3 bucket name, Google document ID) and run a script that would create and run an (minimum viable) application stack that satisfies some set of curator, researcher, and software requirements.

This would include, but not be limited to:

• A Google Sheet document ID (to read data from)
• An S3 bucket (to store data)
• A map website (to visualize data)
• A Github repository (to plan and coordinate work)
• Scripts to handle data

[jim-sheldon]

One possible approach:

Variables & secrets in .env file
- Google Oauth service account credentials
- Google Sheet Document ID
- S3 Bucket name(s)
- Disease name
Bash script
- read from .env file
- terraform CLI
- github CLI

Inputs:
- Terraform
- Github Action files
- .gitignore
- Application secrets (.env)
- Source code (including tests)
- Dockerfile
- README
LICENSE

Outputs (and how to create them):
AWS (Terraform)
- ECR repo
- Batch compute environment (Terraform)
- Batch job queue (Terraform)
- Batch job definition (Terraform)
- Eventbridge rule (Terraform)
- Eventbridge target (Terraform)
- Map DNS (Terraform)
- Map Cloudfront distribution (Terraform)
- S3 buckets
Github (Bash script)
- data repository (Bash script)
- map repository (Bash script)
- actions (Bash script)
-- automatic image -> ECR upload
-- S3 -> Github data copy
-- automated testing
- secrets (Bash script)

Not present yet but may need:
Monitoring (I think we need this)
- Batch success + failure
- Github Action success + failure
Line list
- Entry service
- Display service
- Database

[iamleeg]

It'd be useful to know what the common fields and what the specific extension points are for various diseases, so we can understand what the "day zero" platform looks like and how we would configure/extend it to the specific needs of any outbreak.

Also just foregrounding a couple of implications in this proposal which may well be valid but probably deserve explicit consideration:

• any future deployment will run on AWS.
• any future curation activity will start on Google Sheets.

[Mougk]

This sounds great.

Find attached day_zero schema for review (@outbreakprepared @ksewalk ) :

day_zero_schema.csv

M

[ksewalk]

@Mougk I think this schema is great. The only thing it does not include is the country and status drop down options.

[marinanicole]

@Mougk Schema looks good. My only additional thought is whether we want to add a tab for a summary table.

[iamleeg]

@Mougk is this schema the latest version? Can you add indications of which fields are required/optional (in the Covid-19 line list confirmation date and location are required)?

[Mougk]

Not quite, let's discuss during next standup.

[joe-brilliant]

Dependent on #13 and #15

[tvarrelman]

@Mougk @aimeehan1 @iamleeg Here is the updated day zero schema based on our most recent conversations.
GhDayOneSchema.docx

[jim-sheldon]

• twitter bot

[jim-sheldon]

Broke this out into a number of issues.