Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

15692: Introduce background jobs #16927

Merged
merged 38 commits into from
Jul 30, 2024

Commits on Jun 17, 2024

  1. Introduce reusable BackgroundJob framework

    A new abstract class can be used to implement job function classes. It
    handles the necessary logic for starting and stopping jobs, including
    exception handling and rescheduling of recurring jobs.
    
    This commit also includes the migration of data source jobs to the new
    framework.
    alehaa committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    5fab8e4 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. Restore using import_string for jobs

    Using the 'import_string()' utility from Django allows the job script
    class to be simplified, as module imports no longer need to avoid loops.
    This should make it easier to queue and maintain jobs.
    alehaa committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    957bc3d View commit details
    Browse the repository at this point in the history
  2. Use SyncDataSourceJob for management command

    Instead of maintaining two separate job execution logics, the same job
    is now used for both background and interactive execution.
    alehaa committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    db591d4 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. Implement BackgroundJob for running scripts

    The independent implementations of interactive and background script
    execution have been merged into a single BackgroundJob implementation.
    alehaa committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    53a4420 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2024

  1. Configuration menu
    Copy the full SHA
    7fb1875 View commit details
    Browse the repository at this point in the history
  2. Ensure consitent code style

    alehaa committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    212262d View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. Introduce reusable ScheduledJob

    A new abstract class can be used to implement job function classes that
    specialize in scheduling. These use the same logic as regular
    BackgroundJobs, but ensure that they are only scheduled once at any given
    time.
    alehaa committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    9dc6099 View commit details
    Browse the repository at this point in the history
  2. Introduce reusable SystemJob

    A new abstract class can be used to implement job function classes that
    specialize in system background tasks (e.g. synchronization or
    housekeeping). In addition to the features of the BackgroundJob and
    ScheduledJob classes, these implement additional logic to not need to be
    bound to an existing NetBox object and to setup job schedules on plugin
    load instead of an interactive request.
    alehaa committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    4880d81 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    d78ddfc View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. Configuration menu
    Copy the full SHA
    fd8d537 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. Revert "Use SyncDataSourceJob for management"

    This partially reverts commit db591d4. The 'run_now' parameter of
    'enqueue()' remains, as its being used by following commits.
    alehaa committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    15f888c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7d15ec0 View commit details
    Browse the repository at this point in the history
  3. Fix logger for ScriptJob

    alehaa committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    9f1989c View commit details
    Browse the repository at this point in the history
  4. Remove job name for scripts

    Because scripts are already linked through the Job Instance field, the
    name is displayed twice. Removing this reduces redundancy and opens up
    the possibility of simplifying the BackgroundJob framework in future
    commits.
    alehaa committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    257976d View commit details
    Browse the repository at this point in the history
  5. Merge ScheduledJob into BackgroundJob

    Instead of using separate classes, the logic of ScheduledJob is now
    merged into the generic BackgroundJob class. This allows reusing the
    same logic, but dynamically deciding whether to enqueue the same job
    once or multiple times.
    alehaa committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    58089c7 View commit details
    Browse the repository at this point in the history
  6. Add name attribute for BackgroundJob

    Instead of defining individual names on enqueue, BackgroundJob classes
    can now set a job name in their meta class. This is equivalent to other
    Django classes and NetBox scripts.
    alehaa committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    fb75389 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    654e6e7 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    62380fb View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    d6432fb View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    b3f122a View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. Merge branch '15692-cherry' into 15692-background-jobs

    Unfortunately, work on this branch was done twice. However, most of the
    commits affected different sections of the code. This merge commit
    combines both efforts, but removes duplicate work.
    alehaa committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    3e1cc1b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    bcad8cf View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0b15ecf View commit details
    Browse the repository at this point in the history
  4. Remove legacy JobResultStatusChoices

    ChoiceSet was moved to core in 40572b5.
    alehaa committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    309ad29 View commit details
    Browse the repository at this point in the history
  5. Use queue 'low' for system jobs by default

    System jobs usually perform low-priority background tasks and therefore
    can use a different queue than 'default', which is used for regular jobs
    related to specific objects.
    alehaa committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    b17b205 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    60e4e81 View commit details
    Browse the repository at this point in the history
  7. Fix enqueue interval jobs

    As the job's name is set by enqueue(), it must not be passed in handle()
    to avoid duplicate kwargs with the same name.
    alehaa committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    bd4a21c View commit details
    Browse the repository at this point in the history
  8. Honor schedule_at for job's enqueue_once

    Not only can a job's interval change, but so can the time at which it is
    scheduled to run. If a specific scheduled time is set, it will also be
    checked against the current job schedule. If there are any changes, the
    job is rescheduled with the new time.
    alehaa committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    4c2ba09 View commit details
    Browse the repository at this point in the history
  9. Switch BackgroundJob to regular methods

    Instead of using a class method for run(), a regular method is used for
    this purpose. This gives the possibility to add more convenience methods
    in the future, e.g. for interacting with the job object or for logging,
    as implemented for scripts.
    alehaa committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    c047bf4 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    e65e87c View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    3fc3d37 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. Configuration menu
    Copy the full SHA
    cecc2b8 View commit details
    Browse the repository at this point in the history
  2. Touch up docs

    jeremystretch committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    c098d1c View commit details
    Browse the repository at this point in the history
  3. Revert "Use queue 'low' for system jobs by default"

    This reverts commit b17b205.
    alehaa committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    b9cf078 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. Remove system background job

    This commit reverts commits 4880d81 and 0b15ecf. Using the database
    'connection_created' signal for job registration feels a little wrong at
    this point, as it would trigger registration very often. However, the
    background job framework is prepared for this use case and can be used
    by plugins once the auto-registration of jobs is solved.
    alehaa committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    32ebe7b View commit details
    Browse the repository at this point in the history
  2. Fix runscript management command

    Defining names for background jobs was disabled with fb75389. The
    preceeding changes in 257976d did forget the management command.
    alehaa committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    ecf8e79 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    7f0a4e3 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    85b9f65 View commit details
    Browse the repository at this point in the history