eu-west-3 doesn't have m4.xlarge #1127
Labels
Comments
crawford
added
kind/bug
wking
added a commit
to wking/openshift-installer
that referenced
this issue
May 23, 2019
Some regions have per-zone support differences for available instance
classes. For example, us-west-2 has m4 instances in zones a through
c, but not in zone d. It supports m5 instances in all four zones [1].
Unfortunately, we can't switch to just using the reserved-instance
offerings, because Paris claims those for m4.large in all three of its
zones:
$ AWS_PROFILE=ci aws --region eu-west-3 ec2 describe-reserved-instances-offerings --instance-tenancy default --instance-type m4.large --product-description 'Linux/UNIX' --filters Name=scope,Values='Availability Zone' | jq -r '[.ReservedInstancesOfferings[].AvailabilityZone] | sort | unique[]'
eu-west-3a
eu-west-3b
eu-west-3c
but still does not price on-demand compute instances for that region:
$ AWS_PROFILE=ci aws --region us-east-1 pricing get-products --service-code AmazonEC2 --filters Field=tenancy,Type=TERM_MATCH,Value=Shared Field=productFamily,Type=TERM_MATCH,Value='Compute Instance' Field=operatingSystem,Type=TERM_MATCH,Value=Linux Field=instanceFamily,Type=TERM_MATCH,Value='General purpose' | jq -r '[.PriceList[] | fromjson | .product.attributes | select(.instanceType == "m4.large").location] | sort | unique[]'
AWS GovCloud (US)
Asia Pacific (Mumbai)
Asia Pacific (Osaka-Local)
Asia Pacific (Seoul)
Asia Pacific (Singapore)
Asia Pacific (Sydney)
Asia Pacific (Tokyo)
Canada (Central)
EU (Frankfurt)
EU (Ireland)
EU (London)
South America (Sao Paulo)
US East (N. Virginia)
US East (Ohio)
US West (N. California)
US West (Oregon)
So the new test here only assumes a match when both APIs claim support
for the instance class in a given region to avoid re-breaking Paris
[2].
After rerolling the test to account for that sort of thing, I made the
two default changes to address:
$ AWS_PROFILE=ci go test -count 1 .
--- FAIL: TestGetDefaultInstanceClass (57.36s)
--- FAIL: TestGetDefaultInstanceClass/US_West_(Oregon) (1.22s)
default_instance_class_test.go:176: map[m4:map[us-west-2c:{} us-west-2a:{} us-west-2b:{}] m5:map[us-west-2b:{} us-west-2c:{} us-west-2d:{} us-west-2a:{}]]
default_instance_class_test.go:177:
Error Trace: default_instance_class_test.go:177
Error: Not equal:
expected: "m4"
actual : "m5"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-m4
+m5
Test: TestGetDefaultInstanceClass/US_West_(Oregon)
FAIL
FAIL github.com/openshift/installer/platformtests/aws 57.374s
The zone-error fallback covers regions that I don't have access too,
where requesting zones returns errors like (us-gov-east-1):
AuthFailure: AWS was not able to validate the provided access credentials
and (ap-northeast-3):
OptInRequired: You are not subscribed to this service. Please go to http://aws.amazon.com to subscribe.
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1713157#c1
[2]: openshift#1127
wking
added a commit
to wking/openshift-installer
that referenced
this issue
May 23, 2019
Some regions have per-zone support differences for available instance
classes. For example, us-west-2 has m4 instances in zones a through
c, but not in zone d. It supports m5 instances in all four zones [1].
Unfortunately, we can't switch to just using the reserved-instance
offerings, because Paris claims those for m4.large in all three of its
zones:
$ AWS_PROFILE=ci aws --region eu-west-3 ec2 describe-reserved-instances-offerings --instance-tenancy default --instance-type m4.large --product-description 'Linux/UNIX' --filters Name=scope,Values='Availability Zone' | jq -r '[.ReservedInstancesOfferings[].AvailabilityZone] | sort | unique[]'
eu-west-3a
eu-west-3b
eu-west-3c
but still does not price on-demand compute instances for that region:
$ AWS_PROFILE=ci aws --region us-east-1 pricing get-products --service-code AmazonEC2 --filters Field=tenancy,Type=TERM_MATCH,Value=Shared Field=productFamily,Type=TERM_MATCH,Value='Compute Instance' Field=operatingSystem,Type=TERM_MATCH,Value=Linux Field=instanceFamily,Type=TERM_MATCH,Value='General purpose' | jq -r '[.PriceList[] | fromjson | .product.attributes | select(.instanceType == "m4.large").location] | sort | unique[]'
AWS GovCloud (US)
Asia Pacific (Mumbai)
Asia Pacific (Osaka-Local)
Asia Pacific (Seoul)
Asia Pacific (Singapore)
Asia Pacific (Sydney)
Asia Pacific (Tokyo)
Canada (Central)
EU (Frankfurt)
EU (Ireland)
EU (London)
South America (Sao Paulo)
US East (N. Virginia)
US East (Ohio)
US West (N. California)
US West (Oregon)
So the new test here only assumes a match when both APIs claim support
for the instance class in a given region to avoid re-breaking Paris
[2].
After rerolling the test to account for that sort of thing, I made the
default change to address:
$ AWS_PROFILE=ci go test -count 1 .
--- FAIL: TestGetDefaultInstanceClass (57.36s)
--- FAIL: TestGetDefaultInstanceClass/US_West_(Oregon) (1.22s)
default_instance_class_test.go:176: map[m4:map[us-west-2c:{} us-west-2a:{} us-west-2b:{}] m5:map[us-west-2b:{} us-west-2c:{} us-west-2d:{} us-west-2a:{}]]
default_instance_class_test.go:177:
Error Trace: default_instance_class_test.go:177
Error: Not equal:
expected: "m4"
actual : "m5"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-m4
+m5
Test: TestGetDefaultInstanceClass/US_West_(Oregon)
FAIL
FAIL github.com/openshift/installer/platformtests/aws 57.374s
The zone-error fallback covers regions that I don't have access too,
where requesting zones returns errors like (us-gov-east-1):
AuthFailure: AWS was not able to validate the provided access credentials
and (ap-northeast-3):
OptInRequired: You are not subscribed to this service. Please go to http://aws.amazon.com to subscribe.
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1713157#c1
[2]: openshift#1127
wking
added a commit
to wking/openshift-installer
that referenced
this issue
May 23, 2019
Some regions have per-zone support differences for available instance
classes. For example, us-west-2 has m4 instances in zones a through
c, but not in zone d. It supports m5 instances in all four zones [1].
Unfortunately, we can't switch to just using the reserved-instance
offerings, because Paris claims those for m4.large in all three of its
zones:
$ AWS_PROFILE=ci aws --region eu-west-3 ec2 describe-reserved-instances-offerings --instance-tenancy default --instance-type m4.large --product-description 'Linux/UNIX' --filters Name=scope,Values='Availability Zone' | jq -r '[.ReservedInstancesOfferings[].AvailabilityZone] | sort | unique[]'
eu-west-3a
eu-west-3b
eu-west-3c
but still does not price on-demand compute instances for that region:
$ AWS_PROFILE=ci aws --region us-east-1 pricing get-products --service-code AmazonEC2 --filters Field=tenancy,Type=TERM_MATCH,Value=Shared Field=productFamily,Type=TERM_MATCH,Value='Compute Instance' Field=operatingSystem,Type=TERM_MATCH,Value=Linux Field=instanceFamily,Type=TERM_MATCH,Value='General purpose' | jq -r '[.PriceList[] | fromjson | .product.attributes | select(.instanceType == "m4.large").location] | sort | unique[]'
AWS GovCloud (US)
Asia Pacific (Mumbai)
Asia Pacific (Osaka-Local)
Asia Pacific (Seoul)
Asia Pacific (Singapore)
Asia Pacific (Sydney)
Asia Pacific (Tokyo)
Canada (Central)
EU (Frankfurt)
EU (Ireland)
EU (London)
South America (Sao Paulo)
US East (N. Virginia)
US East (Ohio)
US West (N. California)
US West (Oregon)
So the new test here only assumes a match when both APIs claim support
for the instance class in a given region to avoid re-breaking Paris
[2].
After rerolling the test to account for that sort of thing, I made the
default change to address:
$ AWS_PROFILE=ci go test -count 1 .
--- FAIL: TestGetDefaultInstanceClass (57.36s)
--- FAIL: TestGetDefaultInstanceClass/US_West_(Oregon) (1.22s)
default_instance_class_test.go:176: map[m4:map[us-west-2c:{} us-west-2a:{} us-west-2b:{}] m5:map[us-west-2b:{} us-west-2c:{} us-west-2d:{} us-west-2a:{}]]
default_instance_class_test.go:177:
Error Trace: default_instance_class_test.go:177
Error: Not equal:
expected: "m4"
actual : "m5"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-m4
+m5
Test: TestGetDefaultInstanceClass/US_West_(Oregon)
FAIL
FAIL github.com/openshift/installer/platformtests/aws 57.374s
The zone-error fallback covers regions that I don't have access too,
where requesting zones returns errors like (us-gov-east-1):
AuthFailure: AWS was not able to validate the provided access credentials
and (ap-northeast-3):
OptInRequired: You are not subscribed to this service. Please go to http://aws.amazon.com to subscribe.
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1713157#c1
[2]: openshift#1127
openshift-cherrypick-robot
pushed a commit
to openshift-cherrypick-robot/installer
that referenced
this issue
May 23, 2019
Some regions have per-zone support differences for available instance
classes. For example, us-west-2 has m4 instances in zones a through
c, but not in zone d. It supports m5 instances in all four zones [1].
Unfortunately, we can't switch to just using the reserved-instance
offerings, because Paris claims those for m4.large in all three of its
zones:
$ AWS_PROFILE=ci aws --region eu-west-3 ec2 describe-reserved-instances-offerings --instance-tenancy default --instance-type m4.large --product-description 'Linux/UNIX' --filters Name=scope,Values='Availability Zone' | jq -r '[.ReservedInstancesOfferings[].AvailabilityZone] | sort | unique[]'
eu-west-3a
eu-west-3b
eu-west-3c
but still does not price on-demand compute instances for that region:
$ AWS_PROFILE=ci aws --region us-east-1 pricing get-products --service-code AmazonEC2 --filters Field=tenancy,Type=TERM_MATCH,Value=Shared Field=productFamily,Type=TERM_MATCH,Value='Compute Instance' Field=operatingSystem,Type=TERM_MATCH,Value=Linux Field=instanceFamily,Type=TERM_MATCH,Value='General purpose' | jq -r '[.PriceList[] | fromjson | .product.attributes | select(.instanceType == "m4.large").location] | sort | unique[]'
AWS GovCloud (US)
Asia Pacific (Mumbai)
Asia Pacific (Osaka-Local)
Asia Pacific (Seoul)
Asia Pacific (Singapore)
Asia Pacific (Sydney)
Asia Pacific (Tokyo)
Canada (Central)
EU (Frankfurt)
EU (Ireland)
EU (London)
South America (Sao Paulo)
US East (N. Virginia)
US East (Ohio)
US West (N. California)
US West (Oregon)
So the new test here only assumes a match when both APIs claim support
for the instance class in a given region to avoid re-breaking Paris
[2].
After rerolling the test to account for that sort of thing, I made the
default change to address:
$ AWS_PROFILE=ci go test -count 1 .
--- FAIL: TestGetDefaultInstanceClass (57.36s)
--- FAIL: TestGetDefaultInstanceClass/US_West_(Oregon) (1.22s)
default_instance_class_test.go:176: map[m4:map[us-west-2c:{} us-west-2a:{} us-west-2b:{}] m5:map[us-west-2b:{} us-west-2c:{} us-west-2d:{} us-west-2a:{}]]
default_instance_class_test.go:177:
Error Trace: default_instance_class_test.go:177
Error: Not equal:
expected: "m4"
actual : "m5"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-m4
+m5
Test: TestGetDefaultInstanceClass/US_West_(Oregon)
FAIL
FAIL github.com/openshift/installer/platformtests/aws 57.374s
The zone-error fallback covers regions that I don't have access too,
where requesting zones returns errors like (us-gov-east-1):
AuthFailure: AWS was not able to validate the provided access credentials
and (ap-northeast-3):
OptInRequired: You are not subscribed to this service. Please go to http://aws.amazon.com to subscribe.
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1713157#c1
[2]: openshift#1127
jhixson74
pushed a commit
to jhixson74/installer
that referenced
this issue
Jun 5, 2019
Some regions have per-zone support differences for available instance
classes. For example, us-west-2 has m4 instances in zones a through
c, but not in zone d. It supports m5 instances in all four zones [1].
Unfortunately, we can't switch to just using the reserved-instance
offerings, because Paris claims those for m4.large in all three of its
zones:
$ AWS_PROFILE=ci aws --region eu-west-3 ec2 describe-reserved-instances-offerings --instance-tenancy default --instance-type m4.large --product-description 'Linux/UNIX' --filters Name=scope,Values='Availability Zone' | jq -r '[.ReservedInstancesOfferings[].AvailabilityZone] | sort | unique[]'
eu-west-3a
eu-west-3b
eu-west-3c
but still does not price on-demand compute instances for that region:
$ AWS_PROFILE=ci aws --region us-east-1 pricing get-products --service-code AmazonEC2 --filters Field=tenancy,Type=TERM_MATCH,Value=Shared Field=productFamily,Type=TERM_MATCH,Value='Compute Instance' Field=operatingSystem,Type=TERM_MATCH,Value=Linux Field=instanceFamily,Type=TERM_MATCH,Value='General purpose' | jq -r '[.PriceList[] | fromjson | .product.attributes | select(.instanceType == "m4.large").location] | sort | unique[]'
AWS GovCloud (US)
Asia Pacific (Mumbai)
Asia Pacific (Osaka-Local)
Asia Pacific (Seoul)
Asia Pacific (Singapore)
Asia Pacific (Sydney)
Asia Pacific (Tokyo)
Canada (Central)
EU (Frankfurt)
EU (Ireland)
EU (London)
South America (Sao Paulo)
US East (N. Virginia)
US East (Ohio)
US West (N. California)
US West (Oregon)
So the new test here only assumes a match when both APIs claim support
for the instance class in a given region to avoid re-breaking Paris
[2].
After rerolling the test to account for that sort of thing, I made the
default change to address:
$ AWS_PROFILE=ci go test -count 1 .
--- FAIL: TestGetDefaultInstanceClass (57.36s)
--- FAIL: TestGetDefaultInstanceClass/US_West_(Oregon) (1.22s)
default_instance_class_test.go:176: map[m4:map[us-west-2c:{} us-west-2a:{} us-west-2b:{}] m5:map[us-west-2b:{} us-west-2c:{} us-west-2d:{} us-west-2a:{}]]
default_instance_class_test.go:177:
Error Trace: default_instance_class_test.go:177
Error: Not equal:
expected: "m4"
actual : "m5"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-m4
+m5
Test: TestGetDefaultInstanceClass/US_West_(Oregon)
FAIL
FAIL github.com/openshift/installer/platformtests/aws 57.374s
The zone-error fallback covers regions that I don't have access too,
where requesting zones returns errors like (us-gov-east-1):
AuthFailure: AWS was not able to validate the provided access credentials
and (ap-northeast-3):
OptInRequired: You are not subscribed to this service. Please go to http://aws.amazon.com to subscribe.
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1713157#c1
[2]: openshift#1127
JacobTanenbaum
pushed a commit
to JacobTanenbaum/installer
that referenced
this issue
Jun 7, 2019
Some regions have per-zone support differences for available instance
classes. For example, us-west-2 has m4 instances in zones a through
c, but not in zone d. It supports m5 instances in all four zones [1].
Unfortunately, we can't switch to just using the reserved-instance
offerings, because Paris claims those for m4.large in all three of its
zones:
$ AWS_PROFILE=ci aws --region eu-west-3 ec2 describe-reserved-instances-offerings --instance-tenancy default --instance-type m4.large --product-description 'Linux/UNIX' --filters Name=scope,Values='Availability Zone' | jq -r '[.ReservedInstancesOfferings[].AvailabilityZone] | sort | unique[]'
eu-west-3a
eu-west-3b
eu-west-3c
but still does not price on-demand compute instances for that region:
$ AWS_PROFILE=ci aws --region us-east-1 pricing get-products --service-code AmazonEC2 --filters Field=tenancy,Type=TERM_MATCH,Value=Shared Field=productFamily,Type=TERM_MATCH,Value='Compute Instance' Field=operatingSystem,Type=TERM_MATCH,Value=Linux Field=instanceFamily,Type=TERM_MATCH,Value='General purpose' | jq -r '[.PriceList[] | fromjson | .product.attributes | select(.instanceType == "m4.large").location] | sort | unique[]'
AWS GovCloud (US)
Asia Pacific (Mumbai)
Asia Pacific (Osaka-Local)
Asia Pacific (Seoul)
Asia Pacific (Singapore)
Asia Pacific (Sydney)
Asia Pacific (Tokyo)
Canada (Central)
EU (Frankfurt)
EU (Ireland)
EU (London)
South America (Sao Paulo)
US East (N. Virginia)
US East (Ohio)
US West (N. California)
US West (Oregon)
So the new test here only assumes a match when both APIs claim support
for the instance class in a given region to avoid re-breaking Paris
[2].
After rerolling the test to account for that sort of thing, I made the
default change to address:
$ AWS_PROFILE=ci go test -count 1 .
--- FAIL: TestGetDefaultInstanceClass (57.36s)
--- FAIL: TestGetDefaultInstanceClass/US_West_(Oregon) (1.22s)
default_instance_class_test.go:176: map[m4:map[us-west-2c:{} us-west-2a:{} us-west-2b:{}] m5:map[us-west-2b:{} us-west-2c:{} us-west-2d:{} us-west-2a:{}]]
default_instance_class_test.go:177:
Error Trace: default_instance_class_test.go:177
Error: Not equal:
expected: "m4"
actual : "m5"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-m4
+m5
Test: TestGetDefaultInstanceClass/US_West_(Oregon)
FAIL
FAIL github.com/openshift/installer/platformtests/aws 57.374s
The zone-error fallback covers regions that I don't have access too,
where requesting zones returns errors like (us-gov-east-1):
AuthFailure: AWS was not able to validate the provided access credentials
and (ap-northeast-3):
OptInRequired: You are not subscribed to this service. Please go to http://aws.amazon.com to subscribe.
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1713157#c1
[2]: openshift#1127
jhixson74
pushed a commit
to jhixson74/installer
that referenced
this issue
Jun 12, 2019
Some regions have per-zone support differences for available instance
classes. For example, us-west-2 has m4 instances in zones a through
c, but not in zone d. It supports m5 instances in all four zones [1].
Unfortunately, we can't switch to just using the reserved-instance
offerings, because Paris claims those for m4.large in all three of its
zones:
$ AWS_PROFILE=ci aws --region eu-west-3 ec2 describe-reserved-instances-offerings --instance-tenancy default --instance-type m4.large --product-description 'Linux/UNIX' --filters Name=scope,Values='Availability Zone' | jq -r '[.ReservedInstancesOfferings[].AvailabilityZone] | sort | unique[]'
eu-west-3a
eu-west-3b
eu-west-3c
but still does not price on-demand compute instances for that region:
$ AWS_PROFILE=ci aws --region us-east-1 pricing get-products --service-code AmazonEC2 --filters Field=tenancy,Type=TERM_MATCH,Value=Shared Field=productFamily,Type=TERM_MATCH,Value='Compute Instance' Field=operatingSystem,Type=TERM_MATCH,Value=Linux Field=instanceFamily,Type=TERM_MATCH,Value='General purpose' | jq -r '[.PriceList[] | fromjson | .product.attributes | select(.instanceType == "m4.large").location] | sort | unique[]'
AWS GovCloud (US)
Asia Pacific (Mumbai)
Asia Pacific (Osaka-Local)
Asia Pacific (Seoul)
Asia Pacific (Singapore)
Asia Pacific (Sydney)
Asia Pacific (Tokyo)
Canada (Central)
EU (Frankfurt)
EU (Ireland)
EU (London)
South America (Sao Paulo)
US East (N. Virginia)
US East (Ohio)
US West (N. California)
US West (Oregon)
So the new test here only assumes a match when both APIs claim support
for the instance class in a given region to avoid re-breaking Paris
[2].
After rerolling the test to account for that sort of thing, I made the
default change to address:
$ AWS_PROFILE=ci go test -count 1 .
--- FAIL: TestGetDefaultInstanceClass (57.36s)
--- FAIL: TestGetDefaultInstanceClass/US_West_(Oregon) (1.22s)
default_instance_class_test.go:176: map[m4:map[us-west-2c:{} us-west-2a:{} us-west-2b:{}] m5:map[us-west-2b:{} us-west-2c:{} us-west-2d:{} us-west-2a:{}]]
default_instance_class_test.go:177:
Error Trace: default_instance_class_test.go:177
Error: Not equal:
expected: "m4"
actual : "m5"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-m4
+m5
Test: TestGetDefaultInstanceClass/US_West_(Oregon)
FAIL
FAIL github.com/openshift/installer/platformtests/aws 57.374s
The zone-error fallback covers regions that I don't have access too,
where requesting zones returns errors like (us-gov-east-1):
AuthFailure: AWS was not able to validate the provided access credentials
and (ap-northeast-3):
OptInRequired: You are not subscribed to this service. Please go to http://aws.amazon.com to subscribe.
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1713157#c1
[2]: openshift#1127
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Version
Platform (aws|libvirt|openstack):
aws
What happened?
Right now a stock aws cluster doesn't work in eu-west-3. This is because m4.xlarge is not available.
What you expected to happen?
We should query the list of supported instance types and choose the best option for a given region. See https://stackoverflow.com/questions/45283036/list-all-ec2-instance-types-in-a-region-or-az for tips on using the pricing API to gather this info.
How to reproduce it (as minimally and precisely as possible)?
A stock aws cluster in eu-west-3 will reproduce this problem as of commit 3a604a3
The text was updated successfully, but these errors were encountered: