Admin commands are accessed via the tf
command line tool. See tf --help
for complete usage.
For task decorators:
pip install taskflows
For additional service/scheduling functionality:
sudo apt install libdbus-glib-1-dev
loginctl enable-linger
pip install taskflows[service]
Task execution metadata is stored in SQLite (default) or Postgresql. To use a personal database, set environment variable TASKFLOWS_DB_URL
to your database URL. If using Postgresql, TASKFLOWS_DB_SCHEMA may also be set to use a custom schema (default schema is taskflows).
Turn any function (optionally async) into a task that logs metadata to the database and sends alerts, allows retries, etc..
alerts=[
Alerts(
send_to=[
Slack(
bot_token=os.getenv("SLACK_BOT_TOKEN"),
channel="critical_alerts"
),
Email(
addr="sender@gmail.com",
password=os.getenv("EMAIL_PWD"),
receiver_addr=["someone@gmail.com", "someone@yahoo.com"]
)
],
send_on=["start", "error", "finish"]
)
]
@task(
name='some-task',
required=True,
retries=1,
timeout=30,
alerts=alerts
)
async def hello():
print("Hi.")
Tasks can send alerts via Slack and/or Email, as shown in the above example. Internally, alerts are sent using the alert-msgs package.
Task start/finish times, status, retry count, return values can be found in the task_runs
table.
Any errors that occurred during the execution of a task can be found in the task_errors
table.
Note: To use services, your system must have systemd (the init system on most modern Linux distributions)
Services run commands on a specified schedule. See Service for service configuration options.
To create the service(s), use the create
method (e.g. srv.create()
), or use the CLI create
command (e.g. taskflows create my_services.py
)
from taskflows import Calendar, Service
see Calendar for more options.
srv = Service(
name="something",
start_command="docker start something",
start_schedule=Calendar("Mon-Sun 14:00 America/New_York"),
)
run_time = datetime.now() + timedelta(minutes=30)
srv = Service(
name='write-message',
start_command="bash -c 'echo hello >> hello.txt'",
start_schedule=Calendar.from_datetime(run_time),
)
Run command after system boot, then again every 5 minutes after start of previous run. Skip run if CPU usage is over 80% for the last 5 minutes.
see Periodic and constraints for more options.
Service(
name="my-periodic-task",
start_command="docker start something",
start_schedule=Periodic(start_on="boot", period=60*5, relative_to="start"),
system_load_constraints=CPUPressure(max_percent=80, timespan="5min", silent=True)
)
TASKFLOWS_DB_URL TASKFLOWS_DB_SCHEMA TASKFLOWS_DISPLAY_TIMEZONE TASKFLOWS_DOCKER_LOG_DRIVER TASKFLOWS_FLUENT_BIT_HOST TASKFLOWS_FLUENT_BIT_PORT