S3: Crash in GetObjectCallable

[mikewang2016]

I got crash stack as below:

#0 0x00007f6a8d86c1c0 in Aws::External::tinyxml2::StrPair::GetStr() ()
#1 0x00007f6a8d86d09c in Aws::External::tinyxml2::XMLNode::Value() const ()
#2 0x00007f6a8d8a6dd9 in Aws::Utils::Xml::XmlNode::GetName() const ()
#3 0x00007f6a8d883943 in Aws::Client::AWSXMLClient::BuildAWSError(std::shared_ptrAws::Http::HttpResponse const&) const ()
#4 0x00007f6a8d880f69 in Aws::Client::AWSClient::AttemptOneRequest(std::basic_string<char, std::char_traits, Aws::Allocator > const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod) const ()
<and more ...>

Last piece of trace looks like below:

[TRACE] 2016-12-20 09:16:45 CurlHttpClient [139718591420160] 16360 bytes written to response.
[TRACE] 2016-12-20 09:16:45 CurlHttpClient [139718591420160] 1048 bytes written to response.
[DEBUG] 2016-12-20 09:16:45 CurlHandleContainer [139718591420160] Releasing curl handle 0x7f12c0028560
[DEBUG] 2016-12-20 09:16:45 CurlHandleContainer [139718591420160] Notified waiting threads.
[DEBUG] 2016-12-20 09:16:45 AWSClient [139718591420160] Request returned error. Attempting to generate appropriate error codes from response
[TRACE] 2016-12-20 09:16:45 AWSClient [139718591420160] Error response is a Ã^KXÅ<9f>mpÉ<99>mi^XM0Ë^D^]É^D<99>l^L .

It seems the content of error response it strange which cause the crash.

For those not crash request, it looks like this:

[TRACE] 2016-12-20 09:16:45 CurlHttpClient [139718591420160] 1048 bytes written to response.
[DEBUG] 2016-12-20 09:16:45 CurlHandleContainer [139718591420160] Releasing curl handle 0x7f12c0028560
[DEBUG] 2016-12-20 09:16:45 CurlHandleContainer [139718591420160] Notified waiting threads.
[DEBUG] 2016-12-20 09:16:45 AWSClient [139718591420160] Request returned error. Attempting to generate appropriate error codes from response
[TRACE] 2016-12-20 09:16:45 AWSClient [139718591420160] Error response is
[WARN] 2016-12-20 09:16:45 AWSErrorMarshaller [139718591420160] Encountered Unknown AWSError
XML_ERROR_PARSING_TEXT
Unable to generate a proper httpResponse from the response stream. Response code: 4294967295:
[TRACE] 2016-12-20 09:16:45 AWSClient [139718591420160] Request was either successful, or we are now out of retries.
[TRACE] 2016-12-20 09:16:45 AWSClient [139718591420160] No content body, content-length headers

[bretambrose]

We'll need some more information to be able to possibly help with this.

What platform are you on? Are you building from source or using a packaged version? If building, what is your cmake command line? How are you integrating the SDK with your application (cmake export file, etc...)? How reproducible is the crash? Have you tried debugging the crash (in particular what is in the httpResponse being parsed and what came across the wire for the response)?

[mikewang2016]

I think the problem is with this code "HttpResponseCode::REQUEST_NOT_MADE". This is defined as "-1" and used in CurlHttpClient.cpp. But it's never processed specially. If I register "ContinueRequest" handler and cancel the request in the middle, and then "AWSXMLClient::BuildAWSError" will be trying to parse some binary data as XML. That will cause high chance (but not always) for crash.

[mikewang2016]

A quick fix can look like below:

static bool DoesResponseGenerateError(const std::shared_ptr<HttpResponse>& response)
{
    if (!response) return true;

    int responseCode = static_cast<int>(response->GetResponseCode());
    return response == nullptr || (responseCode < SUCCESS_RESPONSE_MIN && responseCode >0) || responseCode > SUCCESS_RESPONSE_MAX;

}

[oliora]

It seems that we faced a different issue than the creator of this ticket. Thus I've created #781 for our problem and have deleted my comments from this issue. It's possible that my issue #781 is related to this issue.

[singku]

Fixed