bug: all requests on the upstream return 500 When the upstream node contains an invalid domain name, even if the node's weight is zero

[redynasc]

Issue description

all requests on the upstream return 500 When the upstream node contains an invalid domain name, even if the node's weight is zero

Environment

• apisix version : 1.5
• OS: Linux d1-beta-node-0004 3.10.0-693.2.2.el7.x86_64 change: added doc of how to load plugin. #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
• OpenResty / Nginx version: (cmd: nginx -V or openresty -V)
• nginx version: openresty/1.17.8.2
  built by gcc 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC)
  built with OpenSSL 1.1.1g 21 Apr 2020
  TLS SNI support enabled
  configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt='-O2 -DNGX_MAX_ERROR_STR=49152 -I/usr/local/openresty/pcre/include' --add-module=../ngx_devel_kit-0.3.1 --add-module=../echo-nginx-module-0.62 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 --add-module=../set-misc-nginx-module-0.32 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.08 --add-module=../srcache-nginx-module-0.32 --add-module=../ngx_lua-0.10.17 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.7 --add-module=../rds-json-nginx-module-0.15 --add-module=../rds-csv-nginx-module-0.09 --add-module=../ngx_stream_lua-0.0.8 --with-ld-opt='-Wl,-rpath,/usr/local/openresty/luajit/lib -L/usr/local/openresty/pcre/lib -Wl,-rpath,/usr/local/openresty/pcre/lib' --with-openssl=/tmp/openssl-1.1.1g --with-file-aio --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-md5-asm --with-sha1-asm --with-pcre-jit --with-stream --with-stream_ssl_module --with-threads --with-openssl-opt=-g --with-stream --with-stream_ssl_preread_module

Minimal test code / Steps to reproduce the issue

1.Create an upstream, like this
"nodes":{
"domain1-ok:8080":100,
"domain2-ok:8080":100,
"domain3-bad:8080":0
}
2. then then requests return 500

What's the actual result? (including assertion message & call stack if applicable)

In most cases, the DNS timeout of the zero node will affect the entire upstream requests

What's the expected result?

The request on the node which the weight is not zero return ok

[spacewander]

Is there any reason to use a node without weight? We don't expect the user will create zero weight node, and it might cause some issue in roundrobin or other balancer.

[redynasc]

Is there any reason to use a node without weight? We don't expect the user will create zero weight node, and it might cause some issue in roundrobin or other balancer.

In this case, config the upstream like this.
"checker":{
...
},
"nodes":{
"prod-service:8080":100,
"disaster-recovery-service:8080":0
}
Normally, the request is forwarded to node1. When node1 is unavailable, it will be automatically transferred to node2 through the health check.

[spacewander]

It seems the zero weight can accidentally work well. I guess the trick is came from #1809.

But I don't think we need to "fix" it as even the zero weight node should be health checked. If the domain name is invalid, how can it work? This zero weight node should be checked as the others.

[spacewander]

Even there is not check for the upstream, I don't think there is a reason to allow an invalid domain for backup node.

[redynasc]

In most cases, the domain name of the disaster recovery service is ok, but because it is a public domain name, sometimes the DNS resolution timeout will affect all requests.

[spacewander]

OK, let's move the discussion to #3342.