- Contribution Name: Maintenance Worker
- Implementation Owner: Unknown
- Start Date: 05-12-2020
- Target Date: 15-12-2020
- Appwrite Issue: None
Add a new Appwrite worker to handle periodic maintenance tasks like deleting old logs, abuse records, and inactive sessions or tokens.
Large Appwrite systems create lots of logs and data that might not be relevant after X time.
Lots of data is being stored on Appwrite which might not be relevant. Deleting the data can save a lot of system resources.
A new worker and maintenance tasks will be added to Appwrite and will run on preset time as part of Appwrite background workers.
Create a simple PHP daemon that will be act as a new possible entrypoint for the Appwrite container.
A nice example that could use as a starting point is available here: https://stackoverflow.com/a/44420339/2299554
The sole purpose of the new worker is to execute repetitive server maintenance task at preset intervals (per-hour & per-day for start).
The worker don't need to execute the tasks itself but rather pass them through to the relevant worker message queue (for now mainly the deletes worker), but this might change as more tasks are added and we get a clearer pattern of the actual tasks.
Code Structure
The worker script should be located at: ./app/tasks and use the cli framework to be consistent with other CLI tasks Appwrite executes. The Console class should also be used for different logs. We might want to add a loop function to the Console class, but this should be disccused separately.
Add a simple bash script names maintenance on ./bin to allow the entry point to have a nice simple command, the same file should also have relevant references in the main Dockerfile to make it work.
Tasks:
The maintenance tasks that should be included in day 1 are:
- Delete all system logs older then
X days(add a new method to Audit lib) - Delete all abuse logs older then
X days(add a new method to Abuse lib) - Delete all functions executions older then
X days(delete using exisiting document deletion method available at the delete worker)
New Env Vars
Add a new env variable to determine the amount of X days for data deletion. Default value should be 30 days. As with most env vars and settings in Appwrite we should define this time interval in seconds for consistency and predictability. The key of the new variable should be named: _APP_SYSTEM_RETENTION_TIME (please suggest better name if any).
Docs
Add the new env varibale option to https://appwrite.io/docs/environment-variables under System Settings.
Haven't found any specific data that might be relevant for our exact use case.
Should we also run database optimization scripts as might be recommended after deleting a lot of data?
In the long run we could use this feature to run lots of system health related tasks.