Should round-robin node selection take into account queued calls?

[abless]

OKHttp defines a maximum number of concurrent calls per host. This is currently set to 64. If I have a multi-node service where one of the nodes is performing poorly, I can run into a situation where - even with round-robin as my node selection strategy - I end up making calls to a node that become queued up, even if those calls could have been dispatched to other nodes much more quickly.

I've seen instances of this where at any given time I'll have 64 running threads for calls to one node (with hundreds queued up) and only single-digit threads for calls to other nodes. RemotingOkHttpClient.redirectToNewRequest would be able to take into account the queue (since it has access to the dispatcher) - what do you think of supplying a new node selection strategy that takes that queue saturation into account?

[stale]

This issue has been automatically marked as stale because it has not been touched in the last 60 days. Please comment if you'd like to keep it open, otherwise it'll be closed in 7 days time.