Django-Celery-GrowthMonitor

A Django helper to monitor jobs running Celery tasks

Features

• Utilities to track progress of Celery tasks via in-database jobs
• Designed for jobs with user-uploaded files

Requirements

• Python >= 3.5
• Django >= 2.2
• Celery >= 4.0.2
• echoices >= 2.6.0
• autoslug >= 1.9.7

Installation

Using PyPI

1. Run pip install django-celery-growthmonitor

Using the source code

1. Make sure pandoc is installed
2. Run ./pypi_packager.sh
3. Run pip install dist/django_celery_growthmonitor-x.y.z-[...].wheel, where x.y.z must be replaced by the actual version number and [...] depends on your packaging configuration

Usage

('state', echoices.fields.make_echoicefield(default=celery_growthmonitor.models.AJob.EState.CREATED, echoices=celery_growthmonitor.models.AJob.EState, editable=False)),
('status', echoices.fields.make_echoicefield(default=celery_growthmonitor.models.AJob.EStatus.ACTIVE, echoices=celery_growthmonitor.models.AJob.EStatus, editable=False)),

from .celery import app

@app.task
def my_task(holder: JobHolder, *args):
    job = holder.get_job()
    if job.has_failed():
        # Just skip the whole if the previous task failed
        return holder.pre_serialization()
    # Some processing
    ...
    job.save()
    return holder.pre_serialization()

Helpers

Automatically set the job failed on task failure using custom base Task class

from celery_growthmonitor.models.task import JobFailedOnFailureTask

@app.task(base=JobFailedOnFailureTask, bind=True)
def my_task(self, holder: JobHolder):
    pass

Admin

from django.contrib import admin

from celery_growthmonitor.admin import AJobAdmin

@admin.register(MyJob)
class MyJobAdmin(AJobAdmin):
    fields = AJobAdmin.fields + ('my_extra_field',)
    readonly_fields = AJobAdmin.readonly_fields + ('my_extra_field',)