Fixed v4 signing for unordered multi value query parameters

[stevenh]

Remove sorting of query string values when calculating v4 signing as this is not part of the spec:
http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

The spec only requires the keys, not values, to be sorted which is achieved by Query.Encode().

Also fixed TestBuildCanonicalRequest, which was incorrectly testing without this important part of the build process leading to a false success.

[jasdel]

Thanks for putting this PR together, verifying the v4 signature, and correcting the test case.

[sleavitt]

I'm not sure if I'm understanding what the ultimate goal is, but if the use case here is that the SDK should calculate the signature correctly regardless, it appears that the Elasticsearch Kibana plugin breaks if you do not sort the field values prior to calculating the signature.

The examples below vary only the search_fields fields in the query string as to the order in which they are presented.

---

Example where SDK calculates signature incorrectly:
GET /_plugin/kibana/api/saved_objects/?type=dashboard&per_page=1000&page=1&search_fields=title&search_fields=description HTTP/1.1
Host: .es.amazonaws.com

HTTP/1.1 403 Forbidden
Access-Control-Allow-Origin: *
Content-Length: 192
Content-Type: application/json
X-Amzn-Requestid: 57a3c8e1-8ac2-11e8-b305-f151ed41ab7c
Date: Wed, 18 Jul 2018 19:39:53 GMT

{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."}

---

Example where SDK calculates signature correctly:
GET /_plugin/kibana/api/saved_objects/?type=dashboard&per_page=1000&page=1&search_fields=description&search_fields=title HTTP/1.1
Host: .es.amazonaws.com

HTTP/1.1 200 OK
Cache-Control: no-cache
Content-Type: application/json; charset=utf-8
Date: Wed, 18 Jul 2018 19:40:28 GMT
Kbn-Name: kibana
Kbn-Version: 6.2.3
Vary: accept-encoding
X-Amzn-Requestid: 6c38cb6e-8ac2-11e8-b305-f151ed41ab7c
Transfer-Encoding: chunked

... (chunked encoding and JSON object of search results)

---

I am of the opinion that the SDK should handle this automatically, that I should not have to recreate the URL with the fields sorted by key and value in the correct order so that the SDK can sign the request correctly. I believe that this is the same issue, so I am adding this as a comment here. Please advise if I need to open a new issue or if this is sufficient.

[jasdel]

Hi @sleavitt Thanks for reaching out to us. This PR was reverted in v1.10.36 due to the undocumented expectation of the v4 signature to sort both query string key and value.

Are you running into an issue with the SDK's current version (v1.14.29)? If so please open a new issue.

[sleavitt]

Thank you for making me look at my glide.{yaml,lock} files. You may have just saved me a fair amount of work. It looks like the version it's using is version ddfd17e (tag v1.4.10), which is obviously pretty old. I will update to v1.14.29 and try again. If I continue to have issues, I will open a new ticket. Sorry for what may be nothing more than a false alarm at this point.

[sleavitt]

Feel free to disregard my previous comment. Didn't work on v1.4.10, works fine on v1.14.29. Sorry for the false alarm.