Skip to content

Production Deployment

Seth Erickson edited this page Jan 21, 2022 · 15 revisions

Pre-Flight Check

You will be deploying the latest code in the main branch to production. Make sure you've done any additional testing that's required outside of the usual automated tests in CircleCI. Verify that there are no running CI builds and the most current build on the main branch is green.

Announce Maintenance Window

Canned text for maintenance window:

ScholarSphere maintenance is scheduled for January 22, 6-7AM EST. During that time, site functionality may be limited.

If maintenance includes use of "read-only" mode, add this sentence to the above:

New submissions and revisions will not be permitted.

Included this in the "Announcement" form of the Application settings.

Tag

This can be done the day before the deployment, if needed.

From your local terminal:

git pull origin main

Create a tag for the release. Tags should follow the regex /v\d+/\d+\d+*/

git tag -a vX.Y.Z -m "Tagging X.Y.Z release"
git push --tags

Check the CI build page and verify a release-image job is running. Once that's done, check the scholarpshere-config repo for a PR

When you're ready to perform the deploy, proceed to merging the PR.

Manual PR

If for some reason the CI build fails to submit the PR, we can do it manually

git clone git@github.com:psu-stewardship/scholarsphere-config.git 
cd scholarsphere-config 
git checkout -b vX.Y.Z

Edit argocd-prod/prod.yaml change spec.source.helm.values.image.tag to the tag you created in the previous step.

git add argocd-prod/prod.yaml
git commit -m 'release vX.Y.Z'
git push -u origin vX.Y.Z

Submit a PR to the scholarsphere-config repo.

Deploy

When you're ready to do the actual deployment, simply merge the PR!

https://github.com/psu-stewardship/scholarsphere-config/pulls

Validate

ArgoCD

Visit the project page

Check the application's details. Next to Images you should see scholarsphere:vX.Y.Z. You can filter to show only pods, and all pods should be green.

kubectl

Make sure you're using the prod context for kubectl (https://sites.psu.edu/dltdocs/?p=4954)

Change to the scholarsphere namespace and get a listing of pods:

kubens scholarsphere
kubectl get pods -l app.kubernetes.io/name=scholarsphere -o custom-columns=NAME:.metadata.name,STATUS:.status.phase,IMAGE:'.spec.containers[0].image'

curl

The old-fashioned way:

curl --silent https://scholarsphere.k8s.libraries.psu.edu/health/all.json | jq

Clone this wiki locally