FORK THIS REPOSITORY
AND REPLACE THE FOLLOWING
WITH YOUR OWN README/JOB DOCUMENTATION
A GitHub Template Repository that you can use for Data Manager Job implementations.
This repository is a minimal template for Data Manager Jobs. Although it contains
scaffolding to test and build a Python-based Job, including GitHub actions to
build and publish the implementation container image you can replace the
src files and associated Dockerfile to support any language you choose.
Ultimately the Job is published as a container image, you simply have to
provide the implementation and a suitable data-manager/jobs.yaml
definition.
From a fork you should be able to build and run the tests for the example Job that it defines, start with this, and you'll know you're starting with a working framework: -
python -m venv venv
source venv/bin/activate
python -m pip install -r build-requirements.txt
docker-compose build
jote
deactivate
Note: You MUST provide at least one test for every Job your repository defines, and you MUST use our Job Tester (im-jote) to run those tests - it's what we will use and if it fails the Job Tester we are unlikely to deploy the image.
You must have at least one manifest file and at least one job definition file. This template contains a single working example.
As well as replacing this README with your own you will want to adjust the following additional files: -
DEVELOPER-READEM.mdto name the repository you've forked todata-manager/manifest.yamldata-manager/job.yaml- The GitHub Actions, which expects repository secrets
DOCKERHUB_USERNAMEandDOCKERHUB_TOKEN - Adjust the
.gitignoreto satisfy your won tooling - Add tests (and test data)
To assist in local execution on the ARM64 (Apple M1) series of processors your job container image must compile for its architecture. The GitHub actions supplied in this template do that for you by employing the Docker buildx actions.
You can test that your intended mage builds for the ARM64 processor using the notes in this public buildx gist.