Infra for po8klasie.
Goals:
- Minimum cost
- Minimum maintenance
- Survive big spikes in traffic
- Automated deploys, at least to the test instance
- Currently stateless, but has to be able to support statefulness (only in db) in the future
Non goals:
- Automated bootstrapping: initial setup won't happen that often
- Automated updates of infra code (they don't happen often)
Life of a request:
- If it's in Cloudflare cache it gets server from there.
- If not, it is passed (via HTTP, in our setup Cloudflare strips SSL) to the gcp instance external IP port 80.
- Inside the instance it's routed to the nginx container.
- If the path is not
/api/*
it gets served directly from the nginx container. - Otherwise nginx converts the request to the uWSGI protocol and passes it to the uWSGI container.
- The result is served to the client and saved in Cloudflare cache.
Components:
- Nginx image: ghcr.io/po8klasie/po8klasie, built from https://github.com/po8klasie/po8klasie
- uWSGI image: ghcr.io/po8klasie/po8klasie-api, build from https://github.com/po8klasie/po8klasie-api
- Images are automatically built and uploaded to docker hub using GitHub Actions.
- The same action handles pulling and restarting containers on the test deployment + cloudflare cache purge.
- Create an
f1-micro
instance (go to "Compute Engine")- We suggest
europe-west4
: cheaper than Frankfurt, still pretty good latency. - OS image: latest stable of Container Optimized OS (10GB is fine)
- Select "Allow HTTP traffic"
- We suggest
- Log into the instance by clicking the "SSH" button.
- Open port 80:
sudo iptables -w -A INPUT -p tcp --dport 80 -j ACCEPT
git pull https://github.com/po8klasie/infra
cd infra
- Create
django-secret-key.env
andpublic-sentry-dsn.env
files, withDJANGO_SECRET_KEY="<key>"
andPUBLIC_SENTRY_DSN="<dsn>"
as content, respectively. docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v "$PWD:$PWD" -w="$PWD" docker/compose:1.25.5 up -d
- To see logs:
docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v "$PWD:$PWD" -w="$PWD" docker/compose:1.25.5 logs -f
. You can safely Ctrl-C this command, it won't stop any contaienrs. - To update:
docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v "$PWD:$PWD" -w="$PWD" docker/compose:1.25.5 pull
docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v "$PWD:$PWD" -w="$PWD" docker/compose:1.25.5 up -d
- Open port 80:
- Create a service account, download the json key, set is as
GCE_SA_KEY
secret in github. Add IAM roles:Compute OS Admin Login
andService Account User
. - Create an API token with permissions
Zone.Cache Purge
, save asCLOUDFLARE_TOKEN
secret. - The rest of the secrets are self explanatory.
- DNS -> pick a (sub)domain that points to the gcp instance external IP. Proxy status "Proxied"
- SSL/TLS
- Overview tab
- Flexible
- Edge Certificates tab
- Always Use HTTPS -> On
- HSTS
- Enable -> On
- max-age -> 12 months
- includeSubDomains -> On
- Preload -> On
- No-Sniff Header -> On
- Minimum TLS Version -> TLS 1.2
- Opportunistic Encryption -> Off
- TLS 1.3 -> On
- Automatic HTTPS Rewrites -> On
- Certificate Transparency Monitoring -> On
- Overview tab
- Caching
- Cachin Level -> Standard
- Browser Cache TTL -> Respect Existing Headers
- Always Online -> On
- Page Rules
{your domain}/*
- Cache Level -> Cache Everything
- Edge Cache TTL -> a month
- Network
- HTTP/3 (with QUIC) -> On
- 0-RTT Connection Resumption -> On
- WebSockets -> Off
- Scrape Shield -> Yours to choose