[Travis CI] use "ShellCheck" to validate the shell scripts in xCAT commits

[immarvin]

ShellCheck is a GPLv3 tool that gives warnings and suggestions for bash/sh shell scripts, we can leverage it to impose a strict check on the shell scripts in xCAT

From https://github.com/koalaman/shellcheck:
The goals of ShellCheck are

• To point out and clarify typical beginner's syntax issues that cause a shell to give cryptic error messages.

• To point out and clarify typical intermediate level semantic problems that cause a shell to behave strangely and counter-intuitively.

• To point out subtle caveats, corner cases and pitfalls that may cause an advanced user's otherwise working script to fail under future circumstances.
  Travis CI Setup

If you want to use ShellCheck in Travis CI, you can most easily install it via apt:

language: bash
addons:
  apt:
    sources:
    - debian-sid    # Grab ShellCheck from the Debian repo
    packages:
    - shellcheck

[daniceexi]

👍 Good to see the action that we are looking for the ways to improve ourselves.

[hu-weihua]

@immarvin, thanks for your information. It is very helpful for test.

[samveen]

Packages for shellcheck are also available for RH/Centos from EPEL (named ShellCheck).

[samveen]

@immarvin @hu-weihua A lot of the scripts are expected to run on very diverse systems. For those systems that provide bash, it may be good to create a matrix of expected bash versions against the base systems as a lot of shell features are introduced by newer major Bash versions.

In this particular case, I see a script that expects to work on both AIX and linux. However I am unsure of what bash version is available on AIX, which means that I cannot make changes to it because I might be introducing gotchas depending on the version of bash available on AIX.

[immarvin]

hi @samveen , thanks for the explanation.
I remember that AIX does not ship bash by default, postscripts for AIX are supposed to be run with ksh(or sh linked to ksh). And now we do not cover AIX verification in each xcat release.

So I think you do not need to consider AIX support while modifying the postscripts unless:

1. the postscript for AIX only, i.e, the postscripts with the name aix* or with she-bang #!/bin/ksh
2. the code slices for AIX only, i.e, the code in conditional branches like if [ $OS == "AIX" ]; then

For the scripts that are run with bash, the bash version is determined by the bash shipped in linux distributions we support, we suppose the postscripts can be run on all the linux distributions which are announced to be supported in the xcat release notes

[samveen]

@immarvin For any scripts with AIX specific code paths inside common scripts will still need non-bash specific code in the common areas. This makes the script syntax fixes a lot harder.

On a side note, Is AIX still supported, and how long will it remain as a supported system?

[immarvin]

hi @samveen , checked this with team, for xCAT releases after 2.12, we do not announce support for AIX any longer

[samveen]

Can I safely expect Bash >= 4.0 as a minimum available on all platforms where xCAT is expecting to run? If it is not explicitly mentioned, can this be added as an explicit requirement for xCAT servers and for the OS Images for all provisioned nodes?

[immarvin]

hi @samveen , sles11.4 is still in the support list of xCAT 2.13.x and sles11.4 ships bash 3.x

[samveen]

@immarvin I am working on shellchecking the scripts inside xCAT-genesis-scripts/ which are confirmed to run on Centos 7.4 and therefore bash >= 4.0 . The rest can proceed on a case by case basis.

[hu-weihua]

Due to xCAT team will use IBM internal github to manage task. the related task is https://github.ibm.com/xcat2/task_management/issues/177.

[gurevichmark]

@samveen Is your plan to go through all xCAT scripts and make them pass shellcheck ?
Is your plan after that to add shellcheck to TravisCI to run it for each PR ?

[samveen]

@gurevichmark Yes, I was planning on getting as much of the shell portion of the project shellchecked as possible.
As to adding CI checks, I hadn't really thought about it, but I can take a look and see if I can help with that too.