ClientFactory to support regional STS Endpoints

[andrew-glenn]

STS is now a regional-endpoint based services. Our calls to STS clients should reference a specific region rather than rely on the global endpoint.

[jaymccon]

Where are you picking up calls to the global endpoint? I don't see any calls to sts in client_factory.py and the only place I find sts is called in taskcat seems to already specify a region for the client.

https://github.com/aws-quickstart/taskcat/blob/b1316ff5c7bddcccd24091e7bad50163538505a0/taskcat/stacker.py#L1334-L1336

[vsnyc]

That may or may not work, but the documentation states the following:

AWS STS recommends that you use both the setRegion and setEndpoint methods to make calls to a Regional endpoint. You can use the setRegion method alone for manually enabled Regions, such as Asia Pacific (Hong Kong). In this case, the calls are directed to the STS Regional endpoint. To learn how to manually enable a Region, see Managing AWS Regions in the AWS General Reference. If you use the setRegion method alone for Regions enabled by default, the calls are directed to the global endpoint of https://sts.amazonaws.com.

[vsnyc]

Just tested it as well, you do need the endpoint. See snippet below.

>>> import boto3
>>> sts1 = boto3.client('sts', region_name='us-west-2', endpoint_url='https://sts.us-west-2.amazonaws.com')
>>> sts2 = boto3.client('sts', region_name='us-west-2')
>>> print(sts1._endpoint)
sts(https://sts.us-west-2.amazonaws.com)
>>> print(sts2._endpoint)
sts(https://sts.amazonaws.com)

[jaymccon]

Thanks for the research @vsnyc. Let's look at patching for now, even though, imho, intricacies like this should be handled in boto3/botocore

[jaymccon]

boto/boto3#1859

boto/boto3#1939

[jaymccon]

we'll need to discover the endpoint url (seeing as it differs for some partitions) and sadly there doesn't seem to be a direct public method for this boto/boto3#1166

[jaymccon]

UPDATE: previos versions of this returned the global endpoint for sts.

quick attempt at a generic endpoint finder function:

import botocore.loaders as boto_loader
import botocore.regions as boto_regions

def get_endpoint(service, region):
    data = boto_loader.create_loader().load_data("endpoints")
    endpoint_data = boto_regions.EndpointResolver(data).construct_endpoint(service, region)
    if not endpoint_data:
       raise TaskcatException(f"unable to resolve endpoint for {service} in {region}")
    return f"https://{service}.{region}.{endpoint_data['dnsSuffix']}"

[andrew-glenn]

@jaymccon - Thanks. I'll be working on the PR within the coming days. That helps get me off the ground.

[andrew-glenn]

This issue is now slated for the v0.9 branch.