Skip to content

Commit

Permalink
Speed up workflow testing by getting away from Docker (PR #21)
Browse files Browse the repository at this point in the history
## Feature

* feat: requirement checker for merge testing
* feat: use `golangci-lint` v1.36.0 -> latest
* feat: coverage action w/out docker
* feat: merge-tests.yaml w/out docker
* feat: use pre-installed shellcheck
* feat: ".shellceckrc"
  - For older version compatibility. (CI uses old version)

## Fixes

* fix: shell check errors
* fix: golangci-lint.yaml as dispatch workflow only
  - Since merge-tests.yaml does the same thing limit it to `workflow_dispatch` as an individual test
  • Loading branch information
KEINOS authored Feb 14, 2021
1 parent 1314f7c commit 9df541f
Show file tree
Hide file tree
Showing 11 changed files with 251 additions and 59 deletions.
6 changes: 3 additions & 3 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -75,9 +75,9 @@ RUN url_download="https://github.com/koalaman/shellcheck/releases/download/lates
&& shellcheck --version \
&& rm -r "$path_tmp_dir"

# golangci-lint - The fast Go linters runner.
# binary will be $(go env GOPATH)/bin/golangci-lint
RUN curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.36.0 \
# golangci-lint - The fast Go linters runner. Version=latest
# binary will be installed under: $(go env GOPATH)/bin/golangci-lint
RUN curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin \
# Smoke test
&& golangci-lint --version

Expand Down
33 changes: 17 additions & 16 deletions .devcontainer/README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,26 @@
<!-- markdownlint-disable MD033 -->
# Dockerfile for CIs and Development
# Dockerfile for GitHub and VSCode Users To Develop

## For CIs
This directory is for [GitHub Codespaces](https://github.com/features/codespaces) and/or [VS Code + Docker](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack) users for development.

The [CI](https://en.wikipedia.org/wiki/Continuous_integration) will run the [Dockerfile](Dockerfile) to build the image and then run the tests in a container.
It includes most of the necessary packages and tools for developing Golang app. Aiming to provide the same environment to develop the app.

- Current CI
- [GitHub Actions](https://docs.github.com/en/free-pro-team@latest/actions): [../.github/workflows/](https://github.com/KEINOS/Hello-Cobra/tree/main/.github/workflows)
## Developing Online

## For DEVs
If GitHub detects this directory (`.devcontainer`) in the repo, then you will be able to develop online via [GitHub Codespaces](https://github.com/features/codespaces).

This directory is for [GitHub Codespaces](https://github.com/features/codespaces) and [VS Code + Docker](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack) users to develop the cloned/forked repo.
## VS Code + Docker User

### VS Code + Docker User
The container contains VS Code Server as well. If you already have installed the "[Remote - Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)" extension, then press "<kbd>F1</kbd>" and select "`Remote-Containers: Open in Container`".

If you already have installed the "[Remote - Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)" extension, press "<kbd>F1</kbd>" and select "`Remote-Containers: Open in Container`". After a while, you'll get most of the environment needed to develop and debug.
After a while, you'll get most of the environment needed to develop and debug.

- File Description
- [cobra.yaml](cobra.yaml): Default `cobra` command Settings.
- [devcontainer.json](devcontainer.json): VSCode Extensions to be installed.
- [Dockerfile](Dockerfile): Alpine based Golang development container.
- [postCreateCommand.sh](postCreateCommand.sh): Initialization script that runs after the container and the VSCode server is up and before connection from VSCode.
- [settings.vscode.json](settings.vscode.json): Additional VSCode Settings.
- [welcome.sh](welcome.sh): Bash script to display the welcome message in the terminal when first login. It will display the basic info and TIPs to use.
## File Description

- [cobra.yaml](cobra.yaml): Default `cobra` command Settings. Used for `$ cobra add ..`
- [devcontainer.env](devcontainer.env): ENV variables to be loaded once when the container's created.
- [devcontainer.json](devcontainer.json): VSCode Extensions to be installed and env settings.
- [Dockerfile](Dockerfile): Debian 10 (buster) based Golang development container.
- [postCreateCommand.sh](postCreateCommand.sh): Initialization script that runs after the container and the VSCode server is up.
- [README.md](README.md): This file. ;-)
- [welcome.sh](welcome.sh): Bash script to display the basic info and TIPs to use in the first shell login.
77 changes: 77 additions & 0 deletions .github/check-requirements.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
#!/bin/sh
# =============================================================================
# This script checks if the commands and packages required for merge testing
# are installed.
# =============================================================================

# -----------------------------------------------------------------------------
# Constants
# -----------------------------------------------------------------------------
SUCCESS=0
FAILURE=1
TRUE=0
FALSE=1

# -----------------------------------------------------------------------------
# Functions
# -----------------------------------------------------------------------------
isAvailable() {
command_tmp="${1:?Command name missing}"
msg_error="${2:?Error message missing}"
url_reference="${3:?'URL for reference missing'}"

printf -- ' %s ... ' "$command_tmp"
if ! which "$command_tmp" 1>/dev/null 2>/dev/null; then
flag_covered_all=$FALSE
echo 'NG'
echo >&2 " - ABOUT : ${msg_error}"
echo >&2 " - DETAILS: ${url_reference}"
return $FALSE
fi
echo 'OK'
}

# -----------------------------------------------------------------------------
# Main
# -----------------------------------------------------------------------------
flag_covered_all=$TRUE

echo 'Checking requirements for:'

echo
echo 'Shell scripts:'
isAvailable \
shellcheck \
'"shellcheck" is a static analysis tool for shell scripts.' \
'https://github.com/koalaman/shellcheck'

isAvailable shfmt \
'"shfmt" is a linter for shell scripts to support POSIX Shell, Bash, and mksh.' \
'https://github.com/mvdan/sh'

echo
echo 'Go programs:'
isAvailable \
go \
'"go" is required as a matter of course.' \
'https://golang.org/'

isAvailable \
gofmt \
'"gofmt" is a formatter for golang.' \
'https://golang.org/cmd/gofmt/'

isAvailable \
golangci-lint \
'"golangci-lint" is is a Go linters aggregator.' \
'https://golangci-lint.run/'

if [ $flag_covered_all -ne 0 ]; then
echo
echo >&2 "Some requirements missing."
exit $FAILURE
fi

echo
echo 'OK - All requirements were installed! You are good to Go testing!'
exit $SUCCESS
4 changes: 2 additions & 2 deletions .github/mergify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ pull_request_rules:
- name: automatic merge on CI success and 2 approved reviews
conditions:
- "#approved-reviews-by>=2"
- "check-success=docker_test"
- "check-success=merge_tests"
- base=main
- -draft
actions:
Expand All @@ -11,7 +11,7 @@ pull_request_rules:
strict: smart+fasttrack
- name: automatic merge on CI success if only markdown and/or Golang files were changed
conditions:
- "check-success=docker_test"
- "check-success=merge_tests"
- files~=.\.(?i)(md|go)$
- base=main
- -draft
Expand Down
54 changes: 39 additions & 15 deletions .github/workflows/coverage-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,21 +5,45 @@ on:
pull_request:

jobs:
docker_test:
runs-on: ubuntu-latest
coverage:
name: Unit test and coverage

strategy:
matrix:
platform: [ubuntu-latest, macos-latest, windows-latest]

runs-on: ${{ matrix.platform }}

steps:
- name: Checkout the code
uses: actions/checkout@v2
- name: Build the test image
- uses: actions/checkout@v2

- uses: actions/setup-go@v2
with:
go-version: 1.15.x

- name: Use Cache
uses: actions/cache@v2
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Download Modules
if: steps.cache.outputs.cache-hit != 'true'
run: go mod download

- name: Run coverage (Windows)
if: runner.os == 'Windows'
run: |
pwd && \
cd ./.devcontainer && \
docker build -t test:ci .
- name: Run the tests on the container
go install
go mod tidy
go vet -v ./...
go test -cover -v ./...
- name: Run coverage (Linux/macOS)
if: runner.os != 'Windows'
run: |
pwd && \
docker run \
-v $(pwd):/workspaces/Hello-Cobra \
-w /workspaces/Hello-Cobra \
test:ci \
/bin/bash ./.github/run-tests-coverage.sh --verbose
go install
go mod tidy
/bin/bash ./.github/run-tests-coverage.sh --verbose
1 change: 0 additions & 1 deletion .github/workflows/golangci-lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ name: golangci-lint

on:
workflow_dispatch:
pull_request:

jobs:
golangci:
Expand Down
56 changes: 44 additions & 12 deletions .github/workflows/merge-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,21 +5,53 @@ on:
pull_request:

jobs:
docker_test:
merge_tests:
runs-on: ubuntu-latest
steps:
- name: Checkout the code
uses: actions/checkout@v2
- name: Build the test image

- uses: actions/setup-go@v2
with:
go-version: 1.15.x

- name: Use Cache
uses: actions/cache@v2
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Download Modules
if: steps.cache.outputs.cache-hit != 'true'
run: go mod download

- name: Run shellcheck (pre-installed)
run: |
pwd && \
cd ./.devcontainer && \
docker build -t test:ci .
- name: Run full-tests on the container
find . -name '*.sh' -type f -print0 | xargs -0 shellcheck
- name: Install and run shfmt
run: |
GO111MODULE=on go get mvdan.cc/sh/v3/cmd/shfmt
find . -name '*.sh' -type f -print0 | xargs -0 shfmt -d
- name: Run gofmt
uses: Jerome1337/gofmt-action@v1.0.4
with:
gofmt-path: '.'
gofmt-flags: '-d -e' # display diffs and report all errors

- name: Run golangci-lint
uses: golangci/golangci-lint-action@v2
with:
version: v1.36
args: --config ./.github/golangci.yml
# use pre-installed Go
skip-go-installation: true

- name: Run requirement check
run: |
pwd && \
docker run \
-v $(pwd):/workspaces/Hello-Cobra \
-w /workspaces/Hello-Cobra \
test:ci \
/bin/bash ./.github/run-tests-merge.sh
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin
./.github/check-requirements.sh
./.github/run-tests-merge.sh
8 changes: 8 additions & 0 deletions .shellcheckrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# =============================================================================
# ShellCheck Configuration
# =============================================================================
# See: https://github.com/koalaman/shellcheck/wiki/

# Disable
# SC2230: https://github.com/koalaman/shellcheck/wiki/SC2230
disable=SC2230
6 changes: 4 additions & 2 deletions bin/.gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# We do not want to version control the built binary.
# Though we need this directory for access control purposes.
# Output directory of built/compiled application via "../build-app.sh"
#
# We do not want to version control the built binary. Though we need
# this directory for access control purposes.
*
!.gitignore
14 changes: 7 additions & 7 deletions build-app.sh
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ GOARCH:
For supported architectures specify '--list' option.
GOARM:
The 3rd argument is the ARM version. Such as:
The 3rd argument is the ARM variant/version. Such as:
"5", "6", "7".(Default: empty)
Expand All @@ -69,20 +69,20 @@ Sample usage:
./build-app.sh --list
./build-app.sh -l
# Linux (Intel)
# Build Linux (Intel) binary
./build-app.sh linux
# macOS
# Build macOS binary
./build-app.sh darwin #Equivalent to: ./build-app.sh darwin amd64
./build-app.sh darwin arm64
# Windows10
# Build Windows10 binary
./build-app.sh windows
# Raspberry Pi 3
# Build Raspberry Pi 3 binary
./build-app.sh linux arm 7
# QNAP ARM5
# Build QNAP ARM5 binary
./build-app linux arm 5
HEREDOC
Expand All @@ -104,7 +104,7 @@ listPlatforms() {
echo "$list" | indentSTDIN 1>&2
exit $FAILURE
}
echo 'List of available platforms. (OS/Architecture)'
echo 'List of available platforms to build. (GOOS/GOARCH)'
echo "$list" | indentSTDIN
exit $SUCCESS
}
Expand Down
Loading

0 comments on commit 9df541f

Please sign in to comment.