[ECS] Inferentia support

[samuelkarp]

This issue is tracking adding support for Inferentia devices to the ECS variant of Bottlerocket.

This will require adding the Neuron driver and runtime.

[GSvensk]

Does this apply to EKS as well?

[zmrow]

Does this apply to EKS as well?

This issue is to track support for the ECS variant. I'm not sure if EKS would work the same way but we'd probably want to track them separately.

[stmcginnis]

Is this still needed?

[prashant-prodigal]

are we taking this up in next release @stmcginnis ? Do we have an expected timeline for this? We plan to use this heavily.

[stmcginnis]

Sorry for the delay!

No, this will not be included in the next release, unfortunately. There isn't a timeline currently, but it's good to have comments like your's to help gauge interest and have a little extra data to back prioritization.

Contributions are welcome if anyone wants to take a crack at adding it, but until then this is on the list, but not currently assigned anyone to work on it.

[AWS-tomofun]

This is a comment to recommend this issue.

Demand for AI/ML has increased, and requests from users to use GPU or Inferentia chip on ECS are increasing.
There is a plan to support GPU in Fargate, and I think the options for using GPU in ECS will expand for them.
However, since the Inferentia chip doesn't support either EC2 or Fargate, I think this issue is one solution that supports a variety of use cases like these.

[prashant-prodigal]

Hey, any progress on this @AWS-tomofun @stmcginnis ?