aws: Use modern instance types by default

[cgwalters]

OK so I went down a bit of a rabbit hole because @sinnykumari hit a bug where trying to use kernel-rt in AWS via the MCO caused the instance to apparently not have networking on startup.

The real bug here is that the installer is defaulting to old "m4" instance types. AWS now has "m5" instance types which have enhanced networking among other things. And notably, kernel-rt does include the ena driver, but apparently not the xen_netfront one used for older types.

There's a lot of history and related PRs here, among them:

• Support regions without m4 machines #1163
• update instance family from m4 to m5 #1087

And 06e58e0 is highly relevant.

Looking at e.g. us-east-2 (Ohio), it's actually cheaper to use m5.xlarge versus m4.xlarge:

m5.xlarge | 4 | 16 | 16 GiB | EBS Only | $0.192 per Hour
-- | -- | -- | -- | -- | --

m4.xlarge | 4 | 13 | 16 GiB | EBS Only | $0.20 per Hour
-- | -- | -- | -- | -- | --

[cgwalters]

Also, one thing we should consider introducing is some sort of operator that upgrades machine types on IPI/machineAPI installed clusters.

[cgwalters]

I guess really what I'm aiming for in this issue is switching over at least us-east-1 AKA the One True Region since that's what our CI and cluster-bot defaults to among others.

[cgwalters]

Also related #2909

[sdodson]

Also related https://bugzilla.redhat.com/show_bug.cgi?id=1710981

[pierreprinetti]

/label platform/aws

[pittar]

The one and only Canadian region (ca-central-1) finally added a 3rd AZ. Unfortunately, it does not have "m4" instances. Anyone attempting a stock IPI install in Canada will get the following error:

ERROR Error: Error launching source instance: Unsupported: Your requested instance type (m4.xlarge) is not supported in your requested Availability Zone (ca-central-1d). Please retry your request by not specifying an Availability Zone or choosing ca-central-1a, ca-central-1b.

I've added a similar comment to the BZ linked above. The work around is to generate the install-config.yaml and specify the m5 machine type instead, but a lot of folks installing for the first time won't know this, and the error above will be their first experience installing OpenShift 4 with IPI (at least in Canada).

[cgwalters]

@pittar If the other ca-central-1 zones support m4, then I think then we just need a commit like 06e58e0

(But, this is clearly just kicking the can down the road a little bit)

[pittar]

@cgwalters , sorry if my under-caffeinated Sunday morning brain is parsing that code incorrectly, but would that snippet default to m5 if m4 is not available in enough AZs?

The issue I have is that in Canada, only 2 of 3 zones support m4. All 3 zones support m5. Unfortunately, there are only 3 AZs in Canada, which is why the m4 default is an issue.

Without deep knowledge on the installer, yes, I think ca-central-1 would need to be added to this list:
https://github.com/openshift/installer/blob/master/pkg/types/aws/defaults/platform.go#L8

[abhinavdahiya]

#3051

that code in 4.5 would allow the CA to automatically use m5 since that instance type is available in all AZs.

[kikisdeliveryservice]

@cgwalters

As a note @sinnykumari tried today on aws which should be using m5 and rt kernel still didnt' work.. ☹️

[cgwalters]

As a note @sinnykumari tried today on aws which should be using m5 and rt kernel still didnt' work.. frowning_face

I think the recent changes here just prefer m5 - in e.g. us-east-1 I think (particularly in some of the oldest availability zones) there isn't much if any m5 hardware still. And I think OpenShift CI still heavily uses us-east-1.