Integrate with Github Actions

[yong-jie]

Problem

I think there is value in trying out github actions as our CI provider, with the main reason being support for concurrent builds without hoarding travis' concurrent builds limit.

Approach

Traditionally, our CI/CD pipeline consists of the following steps done in sequence:

Lint -> Test -> Build -> Push to ECR -> Deploy to EB

With Github Actions, we can do Lint, Test and Build+Push concurrently, before Deploying to EB after the previous 3 jobs have finished. This an optimistic approach because it builds and pushes the docker image to ECR without checking if Lint and Test have passed. Here is the primary tradeoff:

Pros:

• Overall time to deployment drastically reduces because the build step happens concurrently with tests.

Cons:

• Potentially wasted computation if either Lint or Test builds fail.

Rationale:

• Tests and lint rarely fails in the deployment merge because we would have undergone both the release checklist and all other checks before merging into release.
• This approach is still safe because the final deployment step still waits for all of Lint, Test, and Build+Push to pass before executing.

Things to note

Syntax is lacking
The Github Actions syntax is still in its early phases of development and is missing some quality-of-life configuration options. This means I had to rely on some workarounds to get some things done.

An example is the dedicated job called gatekeep which helps determine whether the build & deploy jobs need to be executed. The workaround is necessary because I can't utilise the env context inside the if statements for jobs, so something like if: branch == env.STAGING || branch == env.PRODUCTION is not possible.

GoGovSG/.github/workflows/ci.yml

Lines 56 to 72 in b522871

	gatekeep:
	name: Determine if Build & Deploy is needed
	outputs:
	proceed: ${{ steps.determine_proceed.outputs.proceed }}
	runs-on: ubuntu-18.04
	steps:
	- shell: python
	id: determine_proceed
	run: |
	import os
	ref = os.environ['GITHUB_REF']
	prod = os.environ['PRODUCTION_BRANCH']
	stage = os.environ['STAGING_BRANCH']
	if ref == prod or ref == stage:
	print('::set-output name=proceed::true')
	else:
	print('::set-output name=proceed::false')

Weak env var selection mechanism
Our builds use different versions of env vars depending on the branch that the Github Action is being run on. At present, I could not find a nice way to select variants of env vars depending on the branch, so I have resorted to using scripts to set our necessary EB env vars.

GoGovSG/.github/workflows/ci.yml

Lines 129 to 146 in b522871

	- name: Select Elastic Beanstalk variables
	shell: python
	run: |
	import os
	branch = os.environ['GITHUB_REF']
	staging = os.environ['STAGING_BRANCH']
	production = os.environ['PRODUCTION_BRANCH']
	eb_app_staging = os.environ['EB_APP_STAGING']
	eb_env_staging = os.environ['EB_ENV_STAGING']
	eb_app_production = os.environ['EB_APP_PRODUCTION']
	eb_env_production = os.environ['EB_ENV_PRODUCTION']
	if branch == staging:
	print('::set-output name=eb_app::' + eb_app_staging)
	print('::set-output name=eb_env::' + eb_env_staging)
	elif branch == production:
	print('::set-output name=eb_app::' + eb_app_production)
	print('::set-output name=eb_env::' + eb_env_production)
	id: select_eb_vars

Lack of inter-workflow dependency/message-passing
It's a universally good practice to split blocks of logic/code into different files. This is not yet supported in Github Actions and I am lumping both CI and CD into a single workflow file for the time being.

List of things done

• Build (tsc)
• Lint
• Test
• Integrate with Coveralls
• Build (Docker)
• Send to ECR
• Set up conditional jobs
• Deploy to EB

[LoneRifle]

lgtm otherwise; please:

• review comments
• figure out if we can immediately replace Travis