Kong warn info 2019/07/29 17:40:54 [warn] 26261#0: *553 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer

[TreeZhiyuan]

While doing stress testing the Kong error log output the following message:
2019/07/29 19:22:34 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "DELETE /upstreams/abtest-upstream-register/targets/507948ac-e627-4c9d-af6c-ce6818c3b16a HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:34 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "DELETE /upstreams/abtest-upstream-register/targets/507948ac-e627-4c9d-af6c-ce6818c3b16a HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:34 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "DELETE /upstreams/abtest-upstream-register/targets/507948ac-e627-4c9d-af6c-ce6818c3b16a HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:34 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "DELETE /upstreams/abtest-upstream-register/targets/507948ac-e627-4c9d-af6c-ce6818c3b16a HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:34 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "DELETE /upstreams/abtest-upstream-register/targets/507948ac-e627-4c9d-af6c-ce6818c3b16a HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:34 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "DELETE /upstreams/abtest-upstream-register/targets/507948ac-e627-4c9d-af6c-ce6818c3b16a HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:34 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "DELETE /upstreams/abtest-upstream-register/targets/507948ac-e627-4c9d-af6c-ce6818c3b16a HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:34 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "DELETE /upstreams/abtest-upstream-register/targets/507948ac-e627-4c9d-af6c-ce6818c3b16a HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:34 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "DELETE /upstreams/abtest-upstream-register/targets/507948ac-e627-4c9d-af6c-ce6818c3b16a HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:34 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "DELETE /upstreams/abtest-upstream-register/targets/507948ac-e627-4c9d-af6c-ce6818c3b16a HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:34 [warn] 26261#0: *29582901 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:34 [warn] 26261#0: *29582901 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:34 [warn] 26261#0: *29582901 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:34 [warn] 26261#0: *29582901 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:34 [warn] 26261#0: *29582901 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:34 [warn] 26261#0: *29582901 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:34 [warn] 26261#0: *29582901 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:34 [warn] 26261#0: *29582901 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:34 [warn] 26261#0: *29582901 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:34 [warn] 26261#0: *29582901 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:46 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "POST /upstreams/abtest-upstream-register/targets HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:46 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "POST /upstreams/abtest-upstream-register/targets HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:46 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "POST /upstreams/abtest-upstream-register/targets HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:46 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "POST /upstreams/abtest-upstream-register/targets HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:46 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "POST /upstreams/abtest-upstream-register/targets HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:46 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "POST /upstreams/abtest-upstream-register/targets HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:46 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "POST /upstreams/abtest-upstream-register/targets HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:46 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "POST /upstreams/abtest-upstream-register/targets HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:46 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "POST /upstreams/abtest-upstream-register/targets HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:46 [warn] 26262#0: *29582877 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, client: 192.168.88.161, server: kong_admin, request: "POST /upstreams/abtest-upstream-register/targets HTTP/1.1", host: "ab.test.com.cn"
2019/07/29 19:22:46 [warn] 26261#0: *29582929 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:46 [warn] 26261#0: *29582929 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:46 [warn] 26261#0: *29582929 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:46 [warn] 26261#0: *29582929 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:46 [warn] 26261#0: *29582929 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:46 [warn] 26261#0: *29582929 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:46 [warn] 26261#0: *29582929 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:46 [warn] 26261#0: *29582929 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:46 [warn] 26261#0: *29582929 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer
2019/07/29 19:22:46 [warn] 26261#0: *29582929 [lua] balancer.lua:716: redistributeIndices(): [ringbalancer] redistributed indices, size=10000, dropped=10000, assigned=0, left unassigned=10000, context: ngx.timer

Does those messages mean there are 10000 request was drop,those drop request response time is 0ms???

• Kong version (v0.13.0)
• Postgres 10.3
• Centos 7

[Tieske]

No, it means that your load balancer is empty. There are no targets left.

The default has 10000 positions, which will be distributed over the targets you add. In this case there are no targets left and they remain unassigned

[hishamhm]

@Tieske In any case, a [warn] level message saying dropped=10000 in a forward proxy is a pretty scary thing and likely to cause further user confusion. Maybe that's a case for renaming the message to removed= (or even deleted=, to better match the DELETE operation). (To be honest, I'm not understanding why it says dropped and assigned=0 in the POST /upstreams/_/targets operations as well.)

Alternatively, if the above messages are nothing to worry about, do we need the internals of the load balancer to be exposed as a warning each time it does its normal operation?

[Tieske]

@hishamhm that message is only logged at the end of a balancer update, which happens on add/delete targets, or on requery the dns for a target, with changed dns results. That's when indices get redistributed.

see: https://github.com/Kong/lua-resty-dns-client/blob/master/src/resty/dns/balancer/ring.lua#L238-L241

The warning is logged only when "assigned" hits 0

I think the OP was scripting against the mgt api, causing a number of those messages to appear in a short window.