Stop container when dependencies are non-resolvable

[angelcar]

Summary

Addresses #2579 where a task can be stuck in PENDING for ever when container dependencies cannot be fulfilled.

Implementation details

Adding the concept of terminal dependency error to detect cases where container dependencies can never succeed. When a terminal dependency error is detected, we simply forcefully stop the container with the "bad" dependencies.

This new behavior triggers a chain reaction that eventually stops the task if an essential container has terminal dependency errors. If essential containers don't have terminal dependency errors, then the task is allowed to start.

Containers that are forcefully stopped are marked with exitCode 143. This is the code docker assigns when a container receives SIGTERM. Although we technically don't send SIGTERM to containers stopped due to terminal dependency errors, this exit code is the closest to reality. However, customers will be able to see a text description of the exact error when this edge case is hit.

For example (from AWS console):

Status reason	CannotStartContainerError: dependency graph: failed to resolve container ordering dependency [container-1(public.ecr.aws/u5s6c6g2/just-sleep) (STOPPED->RUNNING) - Exit: 1] for target [container-2(public.ecr.aws/u5s6c6g2/just-sleep) (NONE->RUNNING)] as dep
143
["/sleep","10"]

Testing

New tests cover the changes: yes

Description for the changelog

Bug - Fix an issue where a task can be stuck in PENDING for ever when container dependencies can never be fulfilled.

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[singholt]

🙌

[fierlion]

(Thanks for the well reasoned PR with clear tests which act as a walkthrough of the changes.)