ECS Works Pools Should Support Specifying Volumes for Flow Runs

[rjruizes]

With ECS Task blocks, you can supply EFS Volume parameters in JSON for the task definition like so

TASK_DEFINITION = {
    ...
    "containerDefinitions": [
        {
            "name": "prefect",
            ...
            "mountPoints": [EFS_MOUNT_POINT],
        },
    ],
    "volumes": [EFS_VOLUME],
}

But no such variables exist for templating an ECS job.

Expectation / Proposal

Volumes and mountPoints should be a JSON variable in the template

[desertaxle]

Thanks for the issue @rjruizes! Custom fields like this are available in the ECS worker via templating, so we recommend using the ECS worker for more configurability. To achieve the same customization in the ECSTask block, you can use the task_customizations field. Let us know if you have further questions about either of those approaches!

[rjruizes]

Ah I understand the templating better now. I modified the base template in the UI (under Advanced) for my ECS Worker Pool to allow for supplying volumes, since it's not built into prefect-aws base template. Example:

      "volumes": "{{ volumes }}",
...
      "volumes": {
        "title": "Volumes",
        "description": "",
        "type": "object",
        "additionalProperties": {
          "type": "string"
        },
        "default": [
          {
            "name": "efs-nibrs",
            "efsVolumeConfiguration": {
              "fileSystemId": "fs-abcdefghjijklmno"
            }
          }
        ]
      },

The UI under Basic then updated the form to show the change.