Add ECS worker option to use most recent revision in task definition family

[kevingrismore]

Closes #368

Adds an option to the ECS worker for using the most recent revision in the task definition family.

Using the family name from the family field on either the work pool or the deployment's job_variables ("prefect" if left empty), the most recent revision of that family's task definition is retrieved and compared against the task definition constructed by the worker. If they are a match, the retrieved task definition's ARN is used rather than proceeding with registration of a new task definition. These steps are only taken if the task definition cache is empty or the cached definition fails to validate.

This approach is similar to the one implemented in Jenkins and should reduce occurrences of task definition registration when used effectively.

It should be noted that careful family name management is required across deployments which share matching configuration, else new registration of task definitions not present in the worker's cache within a single family may reduce the usefulness of this option.

Example

A new worker with an empty cache picks up the most recent revision from the prefect family since the deployment's configuration matches:

Worker 'ECSWorker 03fd9887-2943-4229-ba6c-fa713727af7c' started!
22:17:19.500 | INFO    | prefect.flow_runs.worker - Worker 'ECSWorker 03fd9887-2943-4229-ba6c-fa713727af7c' submitting flow run '77ceb587-3fee-448a-bca9-96c7b47c4616'
22:17:20.273 | INFO    | prefect.flow_runs.worker - Retrieving most recent active revision from ECS task family 'prefect'...
22:17:20.409 | INFO    | prefect.flow_runs.worker - Using ECS task definition 'arn:aws:ecs:***:task-definition/prefect:29'...
22:17:20.834 | INFO    | prefect.flow_runs.worker - Creating ECS task run...
22:17:21.382 | INFO    | prefect.flow_runs.worker - Waiting for ECS task run to start...
22:17:21.438 | INFO    | prefect.flow_runs.worker - ECS task status is PROVISIONING.
22:17:31.534 | INFO    | prefect.flow_runs.worker - ECS task status is PENDING.
22:17:51.751 | INFO    | prefect.flow_runs.worker - ECS task status is RUNNING.
22:17:56.870 | INFO    | prefect.flow_runs.worker - Completed submission of flow run '77ceb587-3fee-448a-bca9-96c7b47c4616'
22:18:17.034 | INFO    | prefect.flow_runs.worker - ECS task status is DEPROVISIONING.
22:18:27.137 | INFO    | prefect.flow_runs.worker - ECS task status is STOPPED.
22:18:27.139 | INFO    | prefect.flow_runs.worker - Container 'prefect' exited successfully.

Upon a second run of the deployment, the in-memory cache is used instead:

22:18:40.072 | INFO    | prefect.flow_runs.worker - Worker 'ECSWorker 03fd9887-2943-4229-ba6c-fa713727af7c' submitting flow run '8932a7f5-e2bb-4ab8-bcfc-a397c3c1781e'
22:18:40.892 | INFO    | prefect.flow_runs.worker - Retrieving ECS task definition 'arn:aws:ecs:***:task-definition/prefect:29'...
22:18:41.038 | INFO    | prefect.flow_runs.worker - Using ECS task definition 'arn:aws:ecs:***:task-definition/prefect:29'...
22:18:41.359 | INFO    | prefect.flow_runs.worker - Creating ECS task run...
22:18:41.921 | INFO    | prefect.flow_runs.worker - Waiting for ECS task run to start...
22:18:41.968 | INFO    | prefect.flow_runs.worker - ECS task status is PROVISIONING.
22:18:52.059 | INFO    | prefect.flow_runs.worker - ECS task status is PENDING.
22:19:12.269 | INFO    | prefect.flow_runs.worker - ECS task status is RUNNING.
22:19:17.372 | INFO    | prefect.flow_runs.worker - Completed submission of flow run '8932a7f5-e2bb-4ab8-bcfc-a397c3c1781e'
22:19:37.536 | INFO    | prefect.flow_runs.worker - ECS task status is DEPROVISIONING.
22:19:52.690 | INFO    | prefect.flow_runs.worker - ECS task status is STOPPED.
22:19:52.691 | INFO    | prefect.flow_runs.worker - Container 'prefect' exited successfully.

Checklist

• References any related issue by including "Closes #" or "Closes ".
  • If no issue exists and your change is not a small fix, please create an issue first.
• Includes tests or only affects documentation.
• Passes pre-commit checks.
  • Run pre-commit install && pre-commit run --all locally for formatting and linting.
• Includes screenshots of documentation updates.
  • Run mkdocs serve view documentation locally.
• Summarizes PR's changes in CHANGELOG.md

[zzstoatzz]

hey @kevingrismore - you should be able to go ahead and nix the CHANGELOG from your branch as we've removed it from main

[kevingrismore]

love the unnesting!