“Could not resolve host” error with curl_multi on cURL version 7.64 #3629

Schnitzel · 2019-02-28T15:37:58Z

Basically copying this over from https://stackoverflow.com/questions/54771917/could-not-resolve-host-error-with-curl-multi-on-curl-version-7-64
I'm not the author of the StackOverflow question, but I have the exact same issue.

I did this

php script:

<?php
$url = "https://www.example.com";

$options = array(
    CURLOPT_RETURNTRANSFER => true,     // don't echo page
    CURLOPT_VERBOSE =>true,
);

$cm = curl_multi_init();


// step 1 of 3 (adding requests)

//request 1
$ch1 = curl_init( $url );
$verboseH1 = fopen('php://temp', 'w+');
$options[CURLOPT_STDERR] = $verboseH1;
curl_setopt_array( $ch1, $options);
curl_multi_add_handle($cm, $ch1);

//request 2
$ch2 = curl_init( $url );
$verboseH2 = fopen('php://temp', 'w+');
$options[CURLOPT_STDERR] = $verboseH2;
curl_setopt_array( $ch2, $options);
curl_multi_add_handle($cm, $ch2);

//request 3
$ch3 = curl_init( $url );
$verboseH3 = fopen('php://temp', 'w+');
$options[CURLOPT_STDERR] = $verboseH3;
curl_setopt_array( $ch3, $options);
curl_multi_add_handle($cm, $ch3);

// step 2 of 3 (executing requests in parallel)
// for more information about these strange loops , check http://php.net/manual/en/function.curl-multi-init.php#115055
$active = null;    
do {
    $mrc = curl_multi_exec($cm, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);

while ($active && $mrc == CURLM_OK) {
    if (curl_multi_select($cm) == -1) {
        usleep(100);
    }
    do {
        $mrc = curl_multi_exec($cm, $active);
    } while ($mrc == CURLM_CALL_MULTI_PERFORM);
}

// step 3 of 3 (getting results)
echo "<h1>Requet 1</h1>\n";
echo "errmsg1: '" .  curl_error($ch1) . "'\n";
rewind($verboseH1);
echo "verbose1:\n" .  stream_get_contents( $verboseH1) . "\n";

echo "===============================================================\n";
echo "<h1>Requet 2</h1>\n";
echo "errmsg2: '" .  curl_error($ch2) . "'\n";
rewind($verboseH2);
echo "verbose2:\n" .  stream_get_contents( $verboseH2) . "\n";

echo "===============================================================\n";
echo "<h1>Requet 3</h1>\n";
echo "errmsg3: '" .  curl_error($ch3) . "'\n";
rewind($verboseH3);
echo "verbose3:\n" .  stream_get_contents( $verboseH3) . "\n";

exit;

output:

<h1>Requet 1</h1>
errmsg1: ''
verbose1:
* Expire in 0 ms for 6 (transfer 0x2b750f0)
* Expire in 1 ms for 1 (transfer 0x2b750f0)
* Expire in 0 ms for 1 (transfer 0x2b750f0)
* Expire in 0 ms for 1 (transfer 0x2b750f0)
*   Trying 93.184.216.34...
* TCP_NODELAY set
* Expire in 149999 ms for 3 (transfer 0x2b750f0)
* Expire in 200 ms for 4 (transfer 0x2b750f0)
* Connected to www.example.com (93.184.216.34) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Los Angeles; O=Internet Corporation for Assigned Names and Numbers; OU=Technology; CN=www.example.org
*  start date: Nov 28 00:00:00 2018 GMT
*  expire date: Dec  2 12:00:00 2020 GMT
*  subjectAltName: host "www.example.com" matched cert's "www.example.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x2b750f0)
> GET / HTTP/2
Host: www.example.com
Accept: */*

* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200 
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Tue, 19 Feb 2019 17:01:16 GMT
< etag: "1541025663+ident"
< expires: Tue, 26 Feb 2019 17:01:16 GMT
< last-modified: Fri, 09 Aug 2013 23:54:35 GMT
< server: ECS (dfw/27B2)
< vary: Accept-Encoding
< x-cache: HIT
< content-length: 1270
< 
* Connection #0 to host www.example.com left intact

===============================================================
<h1>Requet 2</h1>
errmsg2: 'Could not resolve host: www.example.com'
verbose2:
* Expire in 0 ms for 6 (transfer 0x2b7a9a0)
* Found bundle for host www.example.com: 0x2b86800 [serially]
* Server doesn't support multi-use (yet)
* Expire in 1 ms for 1 (transfer 0x2b7a9a0)
* Expire in 0 ms for 1 (transfer 0x2b7a9a0)
* Hostname 'www.example.com' was found in DNS cache
* Expire in 0 ms for 1 (transfer 0x2b7a9a0)
*   Trying 93.184.216.34...
* TCP_NODELAY set
* Expire in 149999 ms for 3 (transfer 0x2b7a9a0)
* Expire in 200 ms for 4 (transfer 0x2b7a9a0)
* Connected to www.example.com (93.184.216.34) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Los Angeles; O=Internet Corporation for Assigned Names and Numbers; OU=Technology; CN=www.example.org
*  start date: Nov 28 00:00:00 2018 GMT
*  expire date: Dec  2 12:00:00 2020 GMT
*  subjectAltName: host "www.example.com" matched cert's "www.example.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x2b7a9a0)
> GET / HTTP/2
Host: www.example.com
Accept: */*

* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200 
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Tue, 19 Feb 2019 17:01:16 GMT
< etag: "1541025663+ident"
< expires: Tue, 26 Feb 2019 17:01:16 GMT
< last-modified: Fri, 09 Aug 2013 23:54:35 GMT
< server: ECS (dfw/562D)
< vary: Accept-Encoding
< x-cache: HIT
< content-length: 1270
< 
* Could not resolve host: www.example.com
* Closing connection 1

===============================================================
<h1>Requet 3</h1>
errmsg3: 'Could not resolve host: www.example.com'
verbose3:
* Expire in 0 ms for 6 (transfer 0x2b80250)
* Found bundle for host www.example.com: 0x2b86800 [serially]
* Server doesn't support multi-use (yet)
* Expire in 1 ms for 1 (transfer 0x2b80250)
* Expire in 0 ms for 1 (transfer 0x2b80250)
* Hostname 'www.example.com' was found in DNS cache
* Expire in 0 ms for 1 (transfer 0x2b80250)
*   Trying 93.184.216.34...
* TCP_NODELAY set
* Expire in 150000 ms for 3 (transfer 0x2b80250)
* Expire in 200 ms for 4 (transfer 0x2b80250)
* Connected to www.example.com (93.184.216.34) port 443 (#2)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Los Angeles; O=Internet Corporation for Assigned Names and Numbers; OU=Technology; CN=www.example.org
*  start date: Nov 28 00:00:00 2018 GMT
*  expire date: Dec  2 12:00:00 2020 GMT
*  subjectAltName: host "www.example.com" matched cert's "www.example.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x2b80250)
> GET / HTTP/2
Host: www.example.com
Accept: */*

* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200 
< accept-ranges: bytes
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Tue, 19 Feb 2019 17:01:16 GMT
< etag: "1541025663"
< expires: Tue, 26 Feb 2019 17:01:16 GMT
< last-modified: Fri, 09 Aug 2013 23:54:35 GMT
< server: ECS (dfw/F4AA)
< vary: Accept-Encoding
< x-cache: HIT
< content-length: 1270
< 
* Could not resolve host: www.example.com
* Closing connection 2

Reproduce:

This will upgrade curl to 7.64.0 which has the issue:

docker pull php:7.2-alpine
docker run --rm -it php:7.2-alpine sh
apk upgrade curl
curl https://gist.githubusercontent.com/Schnitzel/ed65648571f25647eb0bc7b2e5a1b998/raw/6876ce533e6085656fac039492027c98372f27bf/test.php > test.php
php test.php

I expected the following

This will keep curl on 7.63.0 which does not have the issue

docker pull php:7.2-alpine
docker run --rm -it php:7.2-alpine sh
curl https://gist.githubusercontent.com/Schnitzel/ed65648571f25647eb0bc7b2e5a1b998/raw/6876ce533e6085656fac039492027c98372f27bf/test.php > test.php
php test.php

curl/libcurl version

$ curl --version
curl 7.64.0 (x86_64-alpine-linux-musl) libcurl/7.64.0 OpenSSL/1.1.1a zlib/1.2.11 libssh2/1.8.0 nghttp2/1.35.1
Release-Date: 2019-02-06
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy

operating system

cat /etc/alpine-release
3.9.0

Could this maybe have to do with #3446 ?

The text was updated successfully, but these errors were encountered:

bagder · 2019-02-28T16:07:06Z

It's a little unclear what this actually says the problem is. Reading the PHP code, it seems to suggest that even if the transfers run OK, curl_error can return an error string saying it failed to resolve the host name. Is that correct?

curl_error is a PHP binding function that doesn't exist in libcurl itself. The curl_error implementation in PHP shows that it basically just returns the error buffer for the handle that was used for the transfer.

This code seems to ignore this detail that is specifically noted in the man page:

Do not rely on the contents of the buffer unless an error code was returned.

But okay, I created a version of this code written in plain C that to the best of my understanding uses the same logic as the PHP code.

Running this with the latest libcurl I cannot reproduce this problem.

What makes you even suspect this is related to #3446 ?

Schnitzel · 2019-02-28T16:49:14Z

wow, thanks very much for your immediate help :)

Do not rely on the contents of the buffer unless an error code was returned.

Oh wow I think you are correct. It looks like that the return code is correctly HTTP/2 200.

Is there anything that changed though in 7.63 and 7.64 which could make the buffer be filled with a message like Could not resolve host: www.example.com even though the response from the server was returned correctly?

What makes you even suspect this is related to #3446 ?

Sorry for confusing here, I was just searching through the git issues and found a very suspect issue regarding multi connections in curl. Realizing now that the response actually worked, I agree that it has nothing to do with it.

bagder · 2019-02-28T19:18:39Z

I've managed to reproduce. The work-around is still to not check the error buffer if there's no error. Will file a PR with a fix as soon as I have one.

When a transfer is done, the resolver thread will be brought down. That could accidentally generate an error message in the error buffer even though this is not an error situationand the transfer would still return OK. An application that still reads the error buffer could find a "Could not resolve host: [host name]" message there and get confused. Reported-by: Michael Schmid Fixes #3629

bagder · 2019-02-28T20:18:54Z

For the record: I just needed to make sure I ran my sample code using a libcurl built to use the threaded resolver and it occurred at once.

CURLOPT_ERRORBUFFER doc says "Do not rely on the contents of the buffer unless an error code was returned." [1] Prior to this change the error buffer was returned even if no error had occurred, and that buffer may contain incorrect information in such a case. [2] [1]: https://curl.haxx.se/libcurl/c/CURLOPT_ERRORBUFFER.html [2]: curl/curl#3629 Closes #xxxx

CURLOPT_ERRORBUFFER doc says "Do not rely on the contents of the buffer unless an error code was returned." [1] Prior to this change the error buffer was returned even if no error had occurred, and that buffer may contain incorrect information in such a case. [2] [1]: https://curl.haxx.se/libcurl/c/CURLOPT_ERRORBUFFER.html [2]: curl/curl#3629

Schnitzel mentioned this issue Feb 28, 2019

drupal-scaffold plugin fails after latest image release uselagoon/lagoon#923

Closed

Schnitzel mentioned this issue Feb 28, 2019

PrestissimoFileFetcher errors with newest Curl Version 7.64.0 drupal-composer/drupal-scaffold#102

Open

bagder mentioned this issue Feb 28, 2019

threaded-resolver: shutdown the resolver thread without error message #3630

Closed

jay mentioned this issue Mar 1, 2019

curl_error: return an empty string if no error occurred php/php-src#3901

Closed

bagder closed this as completed in 754ae10 Mar 1, 2019

jzakrzewski mentioned this issue Mar 6, 2019

Issue with curl_multi and curl_error returning errors for successful requests in PHP #3652

Closed

This was referenced Mar 15, 2019

main/curl: enable check() alpinelinux/aports#6673

Closed

community/php7: add patch for curl alpinelinux/aports#6674

Closed

lock bot locked as resolved and limited conversation to collaborators Jun 2, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

“Could not resolve host” error with curl_multi on cURL version 7.64 #3629

“Could not resolve host” error with curl_multi on cURL version 7.64 #3629

Schnitzel commented Feb 28, 2019

bagder commented Feb 28, 2019

Schnitzel commented Feb 28, 2019

bagder commented Feb 28, 2019

bagder commented Feb 28, 2019

“Could not resolve host” error with curl_multi on cURL version 7.64 #3629

“Could not resolve host” error with curl_multi on cURL version 7.64 #3629

Comments

Schnitzel commented Feb 28, 2019

I did this

Reproduce:

I expected the following

curl/libcurl version

operating system

bagder commented Feb 28, 2019

Schnitzel commented Feb 28, 2019

bagder commented Feb 28, 2019

bagder commented Feb 28, 2019