-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
15692: Introduce background jobs #16927
15692: Introduce background jobs #16927
Commits on Jun 17, 2024
-
Introduce reusable BackgroundJob framework
A new abstract class can be used to implement job function classes. It handles the necessary logic for starting and stopping jobs, including exception handling and rescheduling of recurring jobs. This commit also includes the migration of data source jobs to the new framework.
Configuration menu - View commit details
-
Copy full SHA for 5fab8e4 - Browse repository at this point
Copy the full SHA 5fab8e4View commit details
Commits on Jun 20, 2024
-
Restore using import_string for jobs
Using the 'import_string()' utility from Django allows the job script class to be simplified, as module imports no longer need to avoid loops. This should make it easier to queue and maintain jobs.
Configuration menu - View commit details
-
Copy full SHA for 957bc3d - Browse repository at this point
Copy the full SHA 957bc3dView commit details -
Use SyncDataSourceJob for management command
Instead of maintaining two separate job execution logics, the same job is now used for both background and interactive execution.
Configuration menu - View commit details
-
Copy full SHA for db591d4 - Browse repository at this point
Copy the full SHA db591d4View commit details
Commits on Jun 21, 2024
-
Implement BackgroundJob for running scripts
The independent implementations of interactive and background script execution have been merged into a single BackgroundJob implementation.
Configuration menu - View commit details
-
Copy full SHA for 53a4420 - Browse repository at this point
Copy the full SHA 53a4420View commit details
Commits on Jun 30, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 7fb1875 - Browse repository at this point
Copy the full SHA 7fb1875View commit details -
Configuration menu - View commit details
-
Copy full SHA for 212262d - Browse repository at this point
Copy the full SHA 212262dView commit details
Commits on Jul 1, 2024
-
Introduce reusable ScheduledJob
A new abstract class can be used to implement job function classes that specialize in scheduling. These use the same logic as regular BackgroundJobs, but ensure that they are only scheduled once at any given time.
Configuration menu - View commit details
-
Copy full SHA for 9dc6099 - Browse repository at this point
Copy the full SHA 9dc6099View commit details -
A new abstract class can be used to implement job function classes that specialize in system background tasks (e.g. synchronization or housekeeping). In addition to the features of the BackgroundJob and ScheduledJob classes, these implement additional logic to not need to be bound to an existing NetBox object and to setup job schedules on plugin load instead of an interactive request.
Configuration menu - View commit details
-
Copy full SHA for 4880d81 - Browse repository at this point
Copy the full SHA 4880d81View commit details -
Configuration menu - View commit details
-
Copy full SHA for d78ddfc - Browse repository at this point
Copy the full SHA d78ddfcView commit details
Commits on Jul 16, 2024
-
Configuration menu - View commit details
-
Copy full SHA for fd8d537 - Browse repository at this point
Copy the full SHA fd8d537View commit details
Commits on Jul 24, 2024
-
Revert "Use SyncDataSourceJob for management"
This partially reverts commit db591d4. The 'run_now' parameter of 'enqueue()' remains, as its being used by following commits.
Configuration menu - View commit details
-
Copy full SHA for 15f888c - Browse repository at this point
Copy the full SHA 15f888cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7d15ec0 - Browse repository at this point
Copy the full SHA 7d15ec0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9f1989c - Browse repository at this point
Copy the full SHA 9f1989cView commit details -
Because scripts are already linked through the Job Instance field, the name is displayed twice. Removing this reduces redundancy and opens up the possibility of simplifying the BackgroundJob framework in future commits.
Configuration menu - View commit details
-
Copy full SHA for 257976d - Browse repository at this point
Copy the full SHA 257976dView commit details -
Merge ScheduledJob into BackgroundJob
Instead of using separate classes, the logic of ScheduledJob is now merged into the generic BackgroundJob class. This allows reusing the same logic, but dynamically deciding whether to enqueue the same job once or multiple times.
Configuration menu - View commit details
-
Copy full SHA for 58089c7 - Browse repository at this point
Copy the full SHA 58089c7View commit details -
Add name attribute for BackgroundJob
Instead of defining individual names on enqueue, BackgroundJob classes can now set a job name in their meta class. This is equivalent to other Django classes and NetBox scripts.
Configuration menu - View commit details
-
Copy full SHA for fb75389 - Browse repository at this point
Copy the full SHA fb75389View commit details -
Configuration menu - View commit details
-
Copy full SHA for 654e6e7 - Browse repository at this point
Copy the full SHA 654e6e7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 62380fb - Browse repository at this point
Copy the full SHA 62380fbView commit details -
Configuration menu - View commit details
-
Copy full SHA for d6432fb - Browse repository at this point
Copy the full SHA d6432fbView commit details -
Configuration menu - View commit details
-
Copy full SHA for b3f122a - Browse repository at this point
Copy the full SHA b3f122aView commit details
Commits on Jul 25, 2024
-
Merge branch '15692-cherry' into 15692-background-jobs
Unfortunately, work on this branch was done twice. However, most of the commits affected different sections of the code. This merge commit combines both efforts, but removes duplicate work.
Configuration menu - View commit details
-
Copy full SHA for 3e1cc1b - Browse repository at this point
Copy the full SHA 3e1cc1bView commit details -
Configuration menu - View commit details
-
Copy full SHA for bcad8cf - Browse repository at this point
Copy the full SHA bcad8cfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0b15ecf - Browse repository at this point
Copy the full SHA 0b15ecfView commit details -
Remove legacy JobResultStatusChoices
ChoiceSet was moved to core in 40572b5.
Configuration menu - View commit details
-
Copy full SHA for 309ad29 - Browse repository at this point
Copy the full SHA 309ad29View commit details -
Use queue 'low' for system jobs by default
System jobs usually perform low-priority background tasks and therefore can use a different queue than 'default', which is used for regular jobs related to specific objects.
Configuration menu - View commit details
-
Copy full SHA for b17b205 - Browse repository at this point
Copy the full SHA b17b205View commit details -
Configuration menu - View commit details
-
Copy full SHA for 60e4e81 - Browse repository at this point
Copy the full SHA 60e4e81View commit details -
As the job's name is set by enqueue(), it must not be passed in handle() to avoid duplicate kwargs with the same name.
Configuration menu - View commit details
-
Copy full SHA for bd4a21c - Browse repository at this point
Copy the full SHA bd4a21cView commit details -
Honor schedule_at for job's enqueue_once
Not only can a job's interval change, but so can the time at which it is scheduled to run. If a specific scheduled time is set, it will also be checked against the current job schedule. If there are any changes, the job is rescheduled with the new time.
Configuration menu - View commit details
-
Copy full SHA for 4c2ba09 - Browse repository at this point
Copy the full SHA 4c2ba09View commit details -
Switch BackgroundJob to regular methods
Instead of using a class method for run(), a regular method is used for this purpose. This gives the possibility to add more convenience methods in the future, e.g. for interacting with the job object or for logging, as implemented for scripts.
Configuration menu - View commit details
-
Copy full SHA for c047bf4 - Browse repository at this point
Copy the full SHA c047bf4View commit details -
Configuration menu - View commit details
-
Copy full SHA for e65e87c - Browse repository at this point
Copy the full SHA e65e87cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3fc3d37 - Browse repository at this point
Copy the full SHA 3fc3d37View commit details
Commits on Jul 29, 2024
-
Configuration menu - View commit details
-
Copy full SHA for cecc2b8 - Browse repository at this point
Copy the full SHA cecc2b8View commit details -
Configuration menu - View commit details
-
Copy full SHA for c098d1c - Browse repository at this point
Copy the full SHA c098d1cView commit details -
Revert "Use queue 'low' for system jobs by default"
This reverts commit b17b205.
Configuration menu - View commit details
-
Copy full SHA for b9cf078 - Browse repository at this point
Copy the full SHA b9cf078View commit details
Commits on Jul 30, 2024
-
This commit reverts commits 4880d81 and 0b15ecf. Using the database 'connection_created' signal for job registration feels a little wrong at this point, as it would trigger registration very often. However, the background job framework is prepared for this use case and can be used by plugins once the auto-registration of jobs is solved.
Configuration menu - View commit details
-
Copy full SHA for 32ebe7b - Browse repository at this point
Copy the full SHA 32ebe7bView commit details -
Configuration menu - View commit details
-
Copy full SHA for ecf8e79 - Browse repository at this point
Copy the full SHA ecf8e79View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7f0a4e3 - Browse repository at this point
Copy the full SHA 7f0a4e3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 85b9f65 - Browse repository at this point
Copy the full SHA 85b9f65View commit details