[serve] Fix issue http proxy downscaling issues#36652

Merged

edoakes merged 20 commits intoray-project:masterfrom GeneDer:fix-issue-http-proxy-downscaling-on-large-request

Jun 26, 2023

+599-125

Commits on Jun 18, 2023

[serve] address issue when downscale nodes with ongoing requests and stop routing to nodes without replica
GeneDer
committed
manual test and fixes issues
GeneDer
committed
Merge branch 'master' into fix-issue-http-proxy-downscaling-on-large-request
GeneDer
committed

Commits on Jun 20, 2023

add unit test for get_running_replica_node_ids() and get_node_ids_with_running_replicas()
GeneDer
committed
add test to test http state and proxies
GeneDer
committed