github_tests_validator_app

github_tests_validator_app is a GitHub Application you can deploy on your own to retrieve test results from forked projects.

Here is how it works :

The main purpose of this project is for education.

Often, Computer Science teachers can have a main repository and ask their students to fork their repository to complete the courses from their repository.

Students never merge their repository into the main one, so it can be tedious for teachers to retrieve informations about the results from their exercices.

By deploying a GitHub App and asking their students to install this app on their GitHub forked projects, teachers can retrieve automatically the results from their tests (using pytest) into Google Sheets.

Prerequisites

1. You need to create a GitHub App in your organization or personal account on GitHub
2. For GCP cloud run deployment, you may need to install terraform and the gcloud CLI.

GitHub App

Your GitHub App needs the following informations :

Repository permissions

• Actions: RO
• Checks: RW # create a check on PR
• Issues: RW # comment the issue
• Contents: RO
• Pull requests: RW # comment the issue

Subscribe to events

• Pull request
• Push
• Workflow job

Deploy locally

You first need to create a .env file by filling up the values from the .env.template file. More details about each of these values below.

You can start the GitHub App backend locally with the following command :

docker run -d --rm -p 127.0.0.1:8080:8080 --env-file .env ghcr.io/artefactory/github_tests_validator_app:latest

Then you can use smee.io or ngrok to serve your applications running locally.

Deploy in the cloud

With Cloud Run, you have an example terraform configuration here.

But you can deploy the application on many Serverless Container services on any cloud by making sure that :

• The secrets defined in the .env file are available for the container at runtime as environment variables
• The container can receive HTTP requests
• The container can login to any data warehouse with a SQLAlchemy Connection URI : Bigquery, Snowflake, Redshift

Environment variables details

• GH_APP_ID : Auto-generated ID of the GitHub App you created during the Prerequisites step.
• GH_APP_KEY : Private Key of the GitHub App you created during the Prerequisites step.
• GH_PAT : GitHub personal access token you must create that has access to the GitHub repository containing the tests and the original repository which was forked (both could be the same repository).
• SQLALCHEMY_URI : Database URI with SQLAlchemy format
• LOGGING : "LOCAL" if you are deploying locally, "GCP" if you are deploying on Google Cloud Run.
• GH_TESTS_REPO_NAME : (Optional, only if you are using a git submodule for the tests folder) Name of the repository containing the tests (could be convenient if you have a repository with the exercices, and another one with the solutions and you want to have the same tests in both repositories by providing a submodule defined in a third repository).

Contributing

Check the guide here : CONTRIBUTING

Makefile usage

Makefile contains many functions for fast assembling and convenient work.

1. Download Poetry

make download-poetry; export PATH="$HOME/.local/bin:$PATH"

2. Install all dependencies and pre-commit hooks

make install

If you do not want to install pre-commit hooks, run the command with the NO_PRE_COMMIT flag:

make install NO_PRE_COMMIT=1

3. Check the security of your code

make check-safety

This command launches a Poetry and Pip integrity check as well as identifies security issues with Safety and Bandit. By default, the build will crash if any of the items fail. But you can set STRICT=0 for the entire build, and then you can configure strictness for each item separately.

make check-safety STRICT=0

or only for safety:

make check-safety STRICT=0 SAFETY_STRICT=1

multiple

make check-safety STRICT=0 PIP_STRICT=1 SAFETY_STRICT=1

List of flags for check-safety (can be set to 1 or 0): STRICT, POETRY_STRICT, PIP_STRICT, SAFETY_STRICT, BANDIT_STRICT.

4. Check the codestyle

The command is similar to check-safety but to check the code style, obviously. It uses Black, Darglint, Isort, and Mypy inside.

make check-style

It may also contain the STRICT flag.

make check-style STRICT=0

List of flags for check-style (can be set to 1 or 0): STRICT, BLACK_STRICT, DARGLINT_STRICT, ISORT_STRICT, MYPY_STRICT.

5. Run all the codestyle formatters

Codestyle uses pre-commit hooks, so ensure you've run make install before.

make format-code

6. Run tests

make test

7. Run all the linters

make lint

the same as:

make test && make check-safety && make check-style

List of flags for lint (can be set to 1 or 0): STRICT, POETRY_STRICT, PIP_STRICT, SAFETY_STRICT, BANDIT_STRICT, BLACK_STRICT, DARGLINT_STRICT, PYUPGRADE_STRICT, ISORT_STRICT, MYPY_STRICT.

8. Build docker

make docker

which is equivalent to:

make docker VERSION=latest

More information here.

9. Cleanup docker

make clean_docker

or to remove all build

make clean

More information here.

10. Activate virtualenv

poetry shell

To deactivate the virtual environment :

deactivate

11. Deploy on GCP

make deploy_gcp