Skip to content

Working with load balancers

Jump to bottom
allenxwang edited this page Jan 6, 2014 · 11 revisions

Overview

Ribbon provides software load balancers to communicate with cluster of servers. The load balancers provide the following basic functionalities:

  • Supply the public DNS name or IP of individual servers to communication client
  • Rotate among a list of servers according to certain logic

Certain load balancers can also provide advanced features like

  • Establishing affinity between clients and servers by dividing them into zones (like racks in a data center) and favor servers in the same zone to reduce latency
  • Keeping statistics of servers and avoid servers with high latency or frequent failures
  • Keeping statistics of zones and avoid zones that might be in outage

Utilizing the advanced features requires using one of the client provided in Ribbon as it has integration with load balancer and provides input to the load balancer statistics.

Components of load balancer

  • Rule - a logic component to determine which server to return from a list
  • Ping - a component running in background to ensure liveness of servers
  • ServerList - this can be static or dynamic. If it is dynamic (as used by DynamicServerListLoadBalancer), a background thread will refresh and filter the list at certain interval

These components can be either set programmatically or be part of client configuration properties and created via reflection. These are the related properties names (should be prefixed by <clientName>.<nameSpace>. in the property file):

NFLoadBalancerClassName
NFLoadBalancerRuleClassName
NFLoadBalancerPingClassName
NIWSServerListClassName
NIWSServerListFilterClassName

The behavior of these components are often runtime changeable by changing a property via Archaius

Common rules

RoundRobinRule

This rule simply choose servers by round robin. It is often used as the default rule or fallback of more advanced rules.

AvailabilityFilteringRule

This rule will skip servers that are deemed "circuit tripped" or with high concurrent connection count.

By default, an instance is circuit tripped if the RestClient fails to make a connection to it for the last three times. Once an instance is circuit tripped, it will remain in this state for 30 seconds before the circuit is deemed as closed again. However, if it continues to fail connections, it will become "circuit tripped" again and the wait time for it to become "circuit closed" will increase exponentially to the number of consecutive failures.

The following properties can be set via Archaius ConfigurationManager:

# successive connection failures threshold to put the server in circuit tripped state, default 3
niws.loadbalancer.<clientName>.connectionFailureCountThreshold

# Maximal period that an instance can remain in "unusable" state regardless of the exponential increase, default 30
niws.loadbalancer.<clientName>.circuitTripMaxTimeoutSeconds

# threshold of concurrent connections count to skip the server, default is Integer.MAX_INT
<clientName>.<clientConfigNameSpace>.ActiveConnectionsLimit

WeightedResponseTimeRule

For this rule, each server is given a weight according to its average response time. The longer the response time, the less weight it will get. The rule randomly picks a server where the possibility is determined by server's weight.

To enable WeightedResponseTimeRule, set it with the load balancer via API or set the following property

<clientName>.<clientConfigNameSpace>.NFLoadBalancerRuleClassName=com.netflix.loadbalancer.WeightedResponseTimeRule

ServerList

Adhoc static server list

You can always set a static list of servers programatially with BaseLoadBalancer or its subclasses with the API BaseLoadBalancer.setServersList()

ConfigurationBasedServerList

This is the default ServerList implementation for a load balancer.

You can set the list of servers as a property with Archaius ConfigurationManager. For example

sample-client.ribbon.listOfServers=www.microsoft.com:80,www.yahoo.com:80,www.google.com:80

If the property is dynamically changed, the list of servers will also change for the load balancer.

DiscoveryEnabledNIWSServerList

This ServerList implementation gets the server list from Eureka client. The server cluster must be identified via VipAddress in a property. For example

myClient.ribbon.NIWSServerListClassName=com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList 
# the server must register itself with Eureka server with VipAddress "myservice"
myClient.ribbon.DeploymentContextBasedVipAddresses=myservice

ServerListFilter

ServerListFilter is a component used by DynamicServerListLoadBalancer to filter the servers returned from a ServerList implementations. There are two implementations of ServerListFilter in Ribbon:

ZoneAffinityServerListFilter

Filters out the servers that are not in the same zone as the client, unless there are not servers available in the client zone. This filter can be enabled by specifying the following properties (assuming client name is "myclient" and client property namespace is "ribbon"):

myclient.ribbon.EnableZoneAffinity=true

ServerListSubsetFilter

This filter makes sure that the client only sees a fixed subset of overall servers returned by the ServerList implementation. It can also replace servers in the subset of poor availability with new servers periodically. To enable this filter, specify the following properties

myClient.ribbon.NIWSServerListClassName=com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList 
# the server must register itself with Eureka server with VipAddress "myservice"
myClient.ribbon.DeploymentContextBasedVipAddresses=myservice
myClient.ribbon.NIWSServerListFilterClassName=com.netflix.loadbalancer.ServerListSubsetFilter
# only show client 5 servers. default is 20.
myClient.ribbon.ServerListSubsetFilter.size=5
Clone this wiki locally