SigV4 fails when standard port included in --endpoint-url

[Diggs]

If the --endpoint-url includes a port that is the default for the specified scheme (e.g. 80 for HTTP, and 443 for HTTPS), then SigV4 includes this port in the host header used when generating the string to sign, however the port is stripped from the host header before being sent to the server. This results in a 403 as the client and server generate different strings to sign.

This can be reproduced using:

export AWS_DEFAULT_REGION=us-east-2
aws --debug --endpoint-url http://s3.us-east-2.amazonaws.com:80 s3 ls s3://`

We have tested this on our s3-compatible service, as well as the real s3 and both servers behave the same. When testing with a non-standard port, the aws cli includes the port in the string to sign AND sends it in the host header, ensuring the client and server generate the same string to sign.

RFC 2616 (https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.23) says A "host" without any trailing port information implies the default port for the service requested (e.g., "80" for an HTTP URL) so I'm assuming the underlying HTTP implementation is stripping it out of the host header because it's not technically required, however by this point it has already been included in the string to sign.

Included below is debug output and corresponding tcpdump packet trace outputs that show this behaviour. We tested on v1.11.132 and v1.11.168. We believe this used to behave differently in earlier releases.

v1.11.168 - Standard port (80) - tested against the real S3 service:

AWS CLI debug:

2017-10-11 02:33:12,971 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
GET
/

host:s3.us-east-2.amazonaws.com:80
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20171011T023312Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2017-10-11 02:33:12,971 - MainThread - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20171011T023312Z
20171011/us-east-2/s3/aws4_request
8ca4947bf0ced90c3de7062d50152206671a964e6f944b788a6c25faae67dac2
2017-10-11 02:33:12,971 - MainThread - botocore.auth - DEBUG - Signature:
8484d1fded73c7fe256dbe32a91d44cd81875951fe0417b60fc3537aa3ebb1ab

Packet trace:

GET / HTTP/1.1
Host: s3.us-east-2.amazonaws.com
Accept-Encoding: identity
X-Amz-Content-SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20171011T023312Z
Authorization: AWS4-HMAC-SHA256 Credential=AKIAJDSRKO5FJFY4XBBA/20171011/us-east-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=8484d1fded73c7fe256dbe32a91d44cd81875951fe0417b60fc3537aa3ebb1ab
User-Agent: aws-cli/1.11.168 Python/2.7.12 Linux/4.9.51-10.52.amzn1.x86_64 botocore/1.7.26

v1.11.168 - Standard port (80) - tested against our s3-compatible service:

AWS CLI debug:

2017-10-11 02:23:41,645 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
GET
/test
delimiter=%2F&encoding-type=url&prefix=
host:ip-172-31-27-129.us-west-2.compute.internal:80
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20171011T022341Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2017-10-11 02:23:41,645 - MainThread - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20171011T022341Z
20171011/us-east-1/s3/aws4_request
88fe3b577c5805f459721019eaa898325b730990822d07fe9b9dde774f9f819f
2017-10-11 02:23:41,646 - MainThread - botocore.auth - DEBUG - Signature:
4082a2084a676d76637cd5ac8a322909033b3b736dcd7cb6632934af7dca5348

Packet trace:

GET /test?delimiter=%2F&prefix=&encoding-type=url HTTP/1.1
Host: ip-172-31-27-129.us-west-2.compute.internal
Accept-Encoding: identity
X-Amz-Content-SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20171011T022341Z
Authorization: AWS4-HMAC-SHA256 Credential=0HM3763OLNVDTNEHKMAD/20171011/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=4082a2084a676d76637cd5ac8a322909033b3b736dcd7cb6632934af7dca5348
User-Agent: aws-cli/1.11.168 Python/2.7.12 Linux/4.9.51-10.52.amzn1.x86_64 botocore/1.7.26

v1.11.132 - Non-standard port (8080) - tested against our s3-compatible service:

AWS CLI debug:

2017-10-11 02:15:41,251 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
GET
/test
delimiter=%2F&encoding-type=url&prefix=
host:ip-172-31-27-129.us-west-2.compute.internal:8080
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20171011T021541Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2017-10-11 02:15:41,251 - MainThread - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20171011T021541Z
20171011/us-east-1/s3/aws4_request
6ec513327cd2637fc3b9b800f445828afa2d290550e47287fc83d75db67732e4
2017-10-11 02:15:41,251 - MainThread - botocore.auth - DEBUG - Signature:
2ff28009983318a01c8a0ca5021b6975c954099a13167136174ed6acf927109d

Packet trace:

GET /test?delimiter=%2F&prefix=&encoding-type=url HTTP/1.1
Host: ip-172-31-27-129.us-west-2.compute.internal:8080
Accept-Encoding: identity
X-Amz-Content-SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20171011T021541Z
Authorization: AWS4-HMAC-SHA256 Credential=0HM3763OLNVDTNEHKMAD/20171011/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=2ff28009983318a01c8a0ca5021b6975c954099a13167136174ed6acf927109d
User-Agent: aws-cli/1.11.132 Python/2.7.12 Linux/4.9.51-10.52.amzn1.x86_64 botocore/1.5.95

[jamesls]

Thanks for reporting, I can repro this issue as well.

Regarding the differences in behavior, that's likely due to us switching the default signature used for S3 from the older signature version to sigv4 (which signs the host header) in more recent versions of the CLI.

We'll get this fixed.

[JordonPhillips]

Looking into this, it seems we're also passing along the auth info since we're setting the host header to just be the netloc.

[JordonPhillips]

To be more clear, that's only the auth info from the url. So we'll happily set the host to username:password@example.com if that's what you set the endpoint url to.