CATLab's Workflows

CATLab's Workflows for Docker projects

These templates were made to be used in projects managed with Gitflow with the little difference that the release branches are not used since the releases are automated.

It is also advised to select the "develop" branch as the default branch so that issues merged into "develop" are closed automatically. It may also be interesting to protect the develop branch and check the checkbox to automatically delete branch when pull request are merged (in GitHub repository > Settings > General).

Here is the workflow we follow to implement new features and fix bugs.

Blue blocks are automated tasks.

At the top of this diagram, we see all the steps from the creation of an issue to the merge of the related pull request. When someone finds a bug or thinks of a new feature, (s)he opens an issue. The contributors of the projects then discuss on the newly created issue. If the bug is really a bug or if the feature is interesting, a branch to fix the bug or implement the feature is created. A pull request of this branch that targets develop is created. Commits are pushed to GitHub and contributors get to see the modifications. Each time commits are pushed to GitHub, the CI workflow is triggered. It automatically builds and tests the image. Moreover, it is advised to enable the Codacy GitHub App to lint the Dockerfile. At the same time, contributors can review the code and see the results of the CI and Codacy analysis (directly on the PR page). After some iterations of the commit/review loop, contributors agree on the completeness of the PR and, if the CI workflow pass and the code is beautiful, they can merge the pull request. To help the contributors, the results of the tests are published in the pull request discussion.

Using the CI workflow enables the docker image to be tested each time some code is published on GitHub to detect problems as early as possible. Moreover, the Dockerfile is linted with to enforce a high level of quality.

Here is the workflow we follow to release new versions.

Blue blocks are automated tasks.

It starts by the creation of a milestone followed by the selection of the issues to treat and their labellisation. The milestone title must match the regex '^[0-9]+.[0-9]+.[0-9]+$'. Putting the right label on the issues is important for changelog generation. A closed issue labeled "bug" will appear under the "Fixed bugs" section. A closed issue labeled "enhancement" will appear under the "Implemented enhancements" section. Contributors implement the issues (see the first diagram). When the last pull request of the milestone is merged into develop, the CI workflow is executed. If the CI workflow passes, the CD workflow is triggered. This workflow starts by merging develop that contains all the newly implemented features and bug fixes, into master (fast-forward). It then tags master with the name of the milestone (the new version). A changelog is generated with this tool and used as the body of the GitHub Release that is then created. Finally, the milestone is closed and the CD workflow ends. As soon as master is tagged, Docker Hub Build builds the new version of the image and publish it on Docker Hub.

Docker CI workflow template

Name: Docker image Continuous Integration Workflow

This workflow builds a docker image each time commits are pushed to GitHub or a pull request is opened or synchronized. It also runs a container of this image to test it with automated integration tests. Moreover, the hadolint linter is run on the Dockerfile and the yamllint linter is run on the whole project.

How to use and tailor the template to your project

1. On GitHub click on the "Actions" tab of the home page of your project and click "Set up this workflow" or copy this workflow template in ".github/workflows/" in your project
2. Tailor it to your project
   1. Give custom values to the environment variables
   2. Modify the job that tests the Docker image
   3. Set the location of the yamllint config file in the "lint-YAML" job or remove this job if it is not necessary
   4. Set up the Codacy GitHub app if you want. You may remove the lint-Dockerfile job if you set up Codacy.
3. Commit and push to GitHub
4. Watch live logs

Recommended YAML lint configuration

---
extends: default

rules:
  line-length:
    max: 80
    level: warning
  truthy: disable

To use it, create a file named ".yamllint.yml" at the root of the project and paste the above content in it.

Docker CD Workflow template

Name: Docker image Continuous Delivery to Docker Hub Workflow

This workflow is triggered each time the CI workflow succeed on develop. It looks for the first milestone that is completed. It must have a title matching the pattern '[0-9]+\.[0-9]+\.[0-9]+'. It then merges the develop branch into the master branch and tags the master branch with the title of the milestone. Finally, it creates a new Release on GitHub where it writes an automatically generated changelog. It also close the milestone (which is 100% completed).

How to use and tailor the template to your project

1. Go to: GitHub project's page > Settings tab > Secrets > Add a new secret and add 2 secrets named "DOCKERHUB_USERNAME" and "DOCKERHUB_PASSWORD". Alternatively, you can remove the step Push to DockerHub and set up Docker Hub Build to build new images when new tags are published or when master is updated.
2. On GitHub click on the "Actions" tab of the home page of your project and click "Set up this workflow" or copy this workflow template in ".github/workflows/" in your project
3. Tailor it to your project
   1. Give custom values to the environment variables
   2. Change the releases titles by changing the value of "release_name" of the job "Create GitHub Release"
4. Commit and push to GitHub
5. Watch live logs

CATLab's Workflow for Java projects

These templates were made to be used in Maven Java projects managed with Gitflow.

Here is the workflow implemented :

Blue blocks are automated tasks.

For each push or pull request on any branch, the GitHub Action CI process builds the jar for Java 8 and 11, then tests that rely on the jar are executed. In parallel, a SonarCloud analysis is executed.
Once the Github Action CI process is completed, if it is successful and on the master branch only, another GitHub Action is triggered in order to close the open milestone if it has only closed issues (issues count > 0 and issues not closed = 0).
Once the milestone is closed, this event triggers another GitHub Action that creates a new tag from the version indicated in pom.xml (which should have been manually modified during the development and must match the milestone version) and makes a draft release, including the generated jar file, whose description contains the list of issues and pull requests of the closed milestone sorted by their label.