Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why does AmazonWebServiceClient.setRegion computes Signer with isRegionIdAsSignerParam=false? #842

Closed
zz22394 opened this issue Sep 13, 2016 · 6 comments

Comments

@zz22394
Copy link

zz22394 commented Sep 13, 2016

Dear team:

I have a question,
https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/AmazonWebServiceClient.java#L342
Why does AmazonWebServiceClient.setRegion computes Signer with isRegionIdAsSignerParam=false?

I am trying to use this library to connect an S3-Compatible storage service.
I use the override region file and set region to the internal one.

System.setProperty(SDKGlobalConfiguration.REGIONS_FILE_OVERRIDE_SYSTEM_PROPERTY, "/tmp/sso_regions.xml");
Region ssoBeta = RegionUtils.getRegion("beta");
client.setRegion(ssoBeta);

However, It fails when authration and got 400 Bad request.
In the request header, SDK library sent

Authorization: AWS4-HMAC-SHA256 Credential=AK....MyKey...A/20160913/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=bbd27c....MyKey...9b5b3

I traced the source code,
AmazonS3Client.setRegion(Region region)
-->super.setRegion(region)
-->'computeSignerByServiceRegion(serviceNameForSigner, region.getName(), signerRegionOverride, false)'
Because AWSS3V4Signer use Region in header, regionId should be used.
So the isRegionIdAsSignerParam parameter here should be true, am I right?

@shorea
Copy link
Contributor

shorea commented Sep 13, 2016

Wow this code is confusing. We'll actually try and set the region in the signer provider which might recompute the region if there's a redirect (https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/internal/auth/S3SignerProvider.java#L53).

The region on the signer itself is apparently more of an override, similar to the signerRegionOverride on the client (https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/auth/AWS4Signer.java#L79). When not set it too will attempt to parse the region from the hostname (https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/auth/internal/AWS4SignerRequestParams.java#L88)

What's your custom endpoints file look like? Would you mind sharing or is it sensitive?

@zz22394
Copy link
Author

zz22394 commented Sep 14, 2016

@shorea Thanks for your reply.
Our custom endpoints file(/tmp/sso_regions.xml):

<XML>
  <Regions>
    <Region>
      <Name>beta</Name>
      <Domain>beta.storage.testdomain.com</Domain>
      <Endpoint>
        <ServiceName>s3</ServiceName>
        <Http>true</Http>
        <Https>true</Https>
        <Hostname>beta.storage.testdomain.com</Hostname>
      </Endpoint>
    </Region>
    <Region>
      <Name>region01</Name>
      <Domain>region01.storage.storage-testdomain.com</Domain>
      <Endpoint>
        <ServiceName>s3</ServiceName>
        <Http>true</Http>
        <Https>true</Https>
        <Hostname>region01.storage.storage-testdomain.com</Hostname>
      </Endpoint>
    </Region>
    <Region>
      <Name>region02</Name>
      <Domain>region02.storage.storage-testdomain.com</Domain>
      <Endpoint>
        <ServiceName>s3</ServiceName>
        <Http>true</Http>
        <Https>true</Https>
        <Hostname>region02.storage.storage-testdomain.com</Hostname>
      </Endpoint>
    </Region>
  </Regions>
  <Services>
    <Service>
      <Name>s3</Name>
      <FullName>testdomain Simple Storage Service</FullName>
      <RegionName>beta</RegionName>
      <RegionName>region01</RegionName>
      <RegionName>region02</RegionName>
    </Service>
  </Services>
</XML>

Because there is no "s3" hint in the Hostname (eg. beta.storage.testdomain.com), maybe we will have to make some change to the AwsHostNameUtils to compute our internal region name.
https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/util/AwsHostNameUtils.java#L120

@shorea
Copy link
Contributor

shorea commented Sep 14, 2016

I believe you can add a mapping in an overridden config file like so.
https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/resources/com/amazonaws/internal/config/awssdk_config_default.json#L145

The override file is expected to be at the root of the classpath with the name 'awssdk_config_override.json'

static final String CONFIG_OVERRIDE_RESOURCE = "awssdk_config_override.json";

@zz22394
Copy link
Author

zz22394 commented Sep 16, 2016

Thanks a lot, I will have a try

@zz22394
Copy link
Author

zz22394 commented Sep 16, 2016

Thanks!
it worked.
I put a customized awssdk_config_override.json in my proj/src/main/resources/ folder,
then the region could be read correctly.

With my edited sso_regions.xml file, the AWS SDK got our internal S3-compatible correctly.

Thanks a lot, it helped me.

@shorea
Copy link
Contributor

shorea commented Sep 16, 2016

Good to hear!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants