Skip to content

Latest commit

 

History

History
195 lines (127 loc) · 6.83 KB

DEVELOPMENT.md

File metadata and controls

195 lines (127 loc) · 6.83 KB
Raw

Development and Testing

For development and testing purposes, it is essential to use a virtual environment. It is recommended that pipenv is used.

Setup Pipenv

To start developing, install pip and pipenv on your system. Note the latter is done at user level to keep the system installation of python clean which is important on a Mac (at least):

sudo easy_install pip

Install pipenv (the --user is important, do not use sudo)

pip install --user pipenv

Check that pipenv is in the binary path

pipenv --version

If not, find the user base binary directory

python -m site --user-base
#~ /Users/<username>/Library/Python/3.7

Append bin to the directory returned and add this to your path by updating ~/.profile. For example you might add the following:

export PATH=~/Library/Python/3.7/bin/:$PATH

Setup Development Environment

Clone GitHub repository

git clone git@github.com:ARMmbed/continuous-delivery-scripts.git

Setup Pipenv to use Python 3 (Python 2 is not supported) and install package development dependencies:

cd continuous-delivery-scripts/
pipenv install --dev

Unit Tests, Code Formatting and Static Analysis

Shell into virtual environment:

pipenv shell

Run unit tests:

pytest

Note that other test runners can be used (e.g. green) as long as they support test written using unittest.TestCase.

Run code formatter (it will format files in place):

black .

Run static analysis (note that no output means all is well):

flake8

Perform static type check:

mypy -p continuous_delivery_scripts

Documenting code

Inclusion of docstrings is needed in all areas of the code for Flake8 checks in the CI to pass.

We use google-style docstrings.

To set up google-style docstring prompts in Pycharm, in the menu navigate to Preferences > Tools > Python Integrated Tools and in the dropdown for docstring format select 'Google'.

For longer explanations, you can also include markdown. Markdown can also be kept in separate files in the docs/user_docs folder and included in a docstring in the relevant place using the reST include as follows:

    .. include:: ../docs/user_docs/documentation.md

Building docs locally

You can do a preview build of the documentation locally by running:

cd-generate-docs

This will generate the docs and output them to local_docs. This should only be a preview. Since documentation is automatically generated by the CI you shouldn't commit any docs html files manually.

Viewing docs generated by the CI

Documentation only gets committed back to this repo to the docs directory during a release and this is what gets published to Github pages. Don't modify any of the files in this directory by hand.

Type hints

Type hints should be used in the code wherever possible. Since the documentation shows the function signatures with the type hints there is no need to include additional type information in the docstrings.

Code Climate

Code Climate is integrated with our GitHub flow. Failing the configured rules will yield a pull request not mergeable.

If you prefer to view the Code Climate report on your machine, prior to sending a pull request, you can use the cli provided by Code Climate.

Plugins for various tools are also available:

Dependency upgrades

For dependency upgrades, dependabot is relied upon and news files are auto-generated in order to document such change. Nonetheless, due to a change in GitHub actions, secrets are not available in the build triggered by the pull request unless they are re-run manually. So please re-run every dependabot PR CI jobs.

Releasing

Release Types

The CI supports three release flows:

  • development for snapshot releases
  • release for stable releases
  • beta for pre-releases
Type Purpose Version Number Format GitHub Release News Files Deleted
Release General Availability <minor>.<major>.<patch> Yes Yes
Beta Integration Testing <minor>.<major>.<patch>-beta.<commit number> Yes No
Development Development Testing <minor>.<major>.<patch>-dev+<git hash> No No

⚠️ releases can be made from any branches but it is recommended that they are only made from the master branch.

Release workflow

  1. Navigate to the GitHub Actions page.
  2. Select the Run Workflow button and type which kind of release you would like to make (i.e. release, beta or development).

Version Numbers

The version number will be automatically calculated, based on the news files.

Detecting secrets

So that no secrets are committed back to the repository, a combination of two tools are run in CI:

  • GitLeaks : Scans the git history for usual secrets (e.g. AWS keys, etc.)
  • detect-secrets: Scans only the current state of the repository for anything which can look like secrets (strings with high entropy)

For the latter, False positive keys are stored in the baseline which detect-secrets checks against when it runs

Baseline & False positives

To flag individual false positives add comment # pragma: allowlist secret to line with secret

To add all suspected secrets in the repository (excluding ones with an allow secret comment), run detect-secrets scan --all-files --exclude-files 'Pipfile\.lock$' --exclude-files '.*\.html$' --exclude-files '.*\.properties$' --exclude-files 'ci.yml' --exclude-files '\.git' --exclude-files '.*_version.py' > .secrets.baseline

If on Windows: then change the encoding of the .secrets.baseline file to UTF-8 then convert all \ to / in the .secrets.baseline file