Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sticky sessions implementation #713

Merged
merged 24 commits into from
Mar 27, 2017
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
fe192ea
don't ignore return code when adding via header
vankoven Mar 18, 2017
ee925ce
Readme: add description of sticky sessions
vankoven Mar 18, 2017
fad63ed
sess: fix parsing sess_lifetime
vankoven Mar 18, 2017
ebe3545
sched: allow to register same scheduler after removing
vankoven Dec 19, 2016
f9fe3c6
sched: add new callback for scheduling from server
vankoven Mar 19, 2017
ee9673a
sched: fix probable hash scheduling error
vankoven Mar 19, 2017
12ed42f
server: remove unused member
vankoven Mar 19, 2017
37bea03
sess: add cfg parser for sticky sessions
vankoven Mar 19, 2017
1471751
tests: divide functional and stress tests for RR scheduler
vankoven Mar 20, 2017
c23758c
tests: functional test for cookies support
vankoven Mar 20, 2017
745911d
tests: add functional tests for sticky sessions
vankoven Mar 21, 2017
0611223
sched: fix sticky session sheduling order
vankoven Mar 21, 2017
c5c5f67
sticky: allow setting sticky options per server group
vankoven Mar 21, 2017
573b035
tests: improve stability
vankoven Mar 22, 2017
5a8dc2c
srv_group: fix error in initialization
vankoven Mar 22, 2017
a636d34
sched http: don't read flags of default group until it is created
vankoven Mar 22, 2017
7a02d91
tests: sticky sessions stress tests
vankoven Mar 22, 2017
9de5f30
update docs
vankoven Mar 22, 2017
c5c80dd
remove #593 todo and fix rebase issue
vankoven Mar 22, 2017
a8c491c
fix code according code review
vankoven Mar 24, 2017
cb66fa8
unit tests: optimize sched tests for high loads
vankoven Mar 25, 2017
3b96b46
unit tests: don't brake kernel on test failure
vankoven Mar 25, 2017
2587f8e
sess: fix default value for sticky cookie key
vankoven Mar 25, 2017
326a21d
fix code according code review
vankoven Mar 27, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 32 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -420,7 +420,7 @@ re-forwarded to the server. However if it's not restored, then the server
connection is considered dead, and all outstanding requests are re-scheduled
to other servers and/or connections.

If a server connection fails intermittenly, then requests may sit in the
If a server connection fails intermittently, then requests may sit in the
connection's forwarding queue for some time. The following directives set
certain allowed limits before these requests are considered failed:
```
Expand Down Expand Up @@ -481,7 +481,7 @@ called `default`.

All server-related directives listed in [Servers](#Servers) section above
are applicable for definition for a server group. Also, a scheduler may be
speficied for a group.
specified for a group.

Below is an example of server group definition:
```
Expand Down Expand Up @@ -618,11 +618,41 @@ then the default match rule is added to route HTTP requests to the group
dropped.


#### Sticky Sessions

In addition to schedulers Tempesta can also use [Sticky Cookies](sticky-cookie)
for load distribution. In this method unique clients are pinned to the
upstream servers. Method can't be applied if client doesn't support cookies.
Not used in default configuration.

Client's first request to a server group will be forwarded to a server chosen
by the group scheduler. All the following requests to the server group will
be forwarded to the same server. If server goes down (for a maintenance or
due to networking errors) client receives `502` responses. When the server
is back online it will continue serving this client.

Session persistence is the highest priority for the method. So if the whole
primary server group is offline new sessions will be pinned to a server in the
backup group if applied. Backup server will continue serving the client even
when the primary group is back online. That means that switching from backup
server group back to the primary group ends only after all the current
sessions pinned to backup server group are expired.

`allow_failover` option allow Tempesta pin sessions to a new server if
the current pinned server went offline. Accident will be logged. Moving client
session from one server to another actually brakes session persistence, so
the backend application must support the feature.

Note, that method does not support setting different backup server groups for
the same primary groups in [HTTP Scheduler](http-scheduler).


### Sticky Cookie

**Sticky cookie** is a special HTTP cookie that is generated by Tempesta.
It allows for unique identification of each client or can be used as challenge
cookie for simple L7 DDoS mitigation when bots are unable to process cookies.
It is also used for a [load balancing](sticky-sessions).

When used, Tempesta sticky cookie is expected in HTTP requests.
Otherwise, Tempesta asks in an HTTP response that sticky cookie is present in
Expand Down
39 changes: 36 additions & 3 deletions etc/tempesta_fw.conf
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@
# SCHED_NAME is a name of a scheduler module that distributes the load
# among servers within a group. There are two schedulers available:
# - "round-robin" (default) - rotates all servers in the group in
# the round-robin manner, so requests are distributed uniformely across
# the round-robin manner, so requests are distributed uniformly across
# servers.
# - "hash" - chooses a server based on a URI/Host hash of a request.
# Requests are still distributed uniformely, but a request with the same
# Requests are still distributed uniformly, but a request with the same
# URI/Host is always sent to the same server.
#
# Note that there's also the HTTP scheduler. It dispatches requests among
Expand Down Expand Up @@ -140,7 +140,7 @@
# Syntax:
# server_queue_size 600;
#
# This is the maxumum number of requests that may be in the forwarding
# This is the maximum number of requests that may be in the forwarding
# queue of a server connection at any given time.
#
# Default:
Expand Down Expand Up @@ -494,6 +494,39 @@
# Example:
# sess_lifetime 900;

# TAG: sticky_sessions
#
# Load balancing method. Forward all requests from client to the the same
# upstream server. Applied to a server group.
#
# Syntax:
# sticky_sessions [allow_failover]
#
# Default:
# Sticky sessions are disabled;
#
# First request to a server group in client's HTTP session will be forwarded
# to upstream server chosen by scheduler applied to the server group. All the
# following requests to that group in this session will pass through scheduler
# and will be forwarded to the same server. Client will recieve 502 error
# if pinned upstream server went offline. Method can't be applied if client
# does not support cookies or Tempesta sticky cookie is disabled.
#
# Note: Session preservation is the priority for the method, so client will
# be served by backup server group even when primary group is back online.
#
# allow_failover - Pin session to a new upstream server and print the warning
# if the last pinned server went offline instead of sending 502 error to client.
#
# Examples:
# srv_group app {
# server 10.10.0.1:8080;
# server 10.10.0.2:8080;
# server [fc00::3]:8081 conns_n=1;
#
# sticky_sessions;
# }

#
# Frang configuration.
#
Expand Down
12 changes: 4 additions & 8 deletions tempesta_fw/http.c
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
#include "client.h"
#include "hash.h"
#include "http_msg.h"
#include "http_sess.h"
#include "log.h"
#include "procfs.h"
#include "server.h"
Expand Down Expand Up @@ -1599,6 +1600,9 @@ tfw_http_adjust_resp(TfwHttpResp *resp, TfwHttpReq *req)
return r;

r = tfw_http_add_hdr_via(hm);
if (r < 0)
return r;

if (resp->flags & TFW_HTTP_RESP_STALE) {
#define S_WARN_110 "Warning: 110 - Response is stale"
/* TODO: ajust for #215 */
Expand Down Expand Up @@ -1819,14 +1823,6 @@ tfw_http_req_cache_cb(TfwHttpReq *req, TfwHttpResp *resp)
* executed, to avoid unnecessary work in SoftIRQ and to speed up
* the cache operation. At the same time, cache hits are expected
* to prevail over cache misses, so this is not a frequent path.
*
* TODO #593: check whether req->sess->srv_conn is alive. If not,
* then get a new connection for req->sess->srv_conn->peer from
* an appropriate scheduler. That eliminates the long generic
* scheduling work flow. When the first request in a session is
* scheduled by the generic logic, TfwSession->srv_conn must be
* initialized to point at the appropriate TfwConn{}, so that
* all subsequent session hits are scheduled much faster.
*/
if (!(srv_conn = tfw_sched_get_srv_conn((TfwMsg *)req))) {
TFW_WARN("Unable to find a back end server\n");
Expand Down
29 changes: 3 additions & 26 deletions tempesta_fw/http.h
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,12 @@
#include "connection.h"
#include "gfsm.h"
#include "msg.h"
#include "server.h"
#include "str.h"
#include "vhost.h"

typedef struct tfw_http_sess_t TfwHttpSess;

/**
* HTTP Generic FSM states.
*
Expand Down Expand Up @@ -269,25 +272,6 @@ typedef struct {
/* It is stale, but pass with a warning */
#define TFW_HTTP_RESP_STALE 0x040000

/**
* HTTP session descriptor.
*
* @hmac - crypto hash from values of an HTTP request;
* @hentry - hash list entry for all sessions hash;
* @users - the session use counter;
* @ts - timestamp for the client's session;
* @expire - expiration time for the session;
* @srv_conn - upstream server connection servicing the session;
*/
typedef struct {
unsigned char hmac[SHA1_DIGEST_SIZE];
struct hlist_node hentry;
atomic_t users;
unsigned long ts;
unsigned long expires;
TfwSrvConn *srv_conn;
} TfwHttpSess;

/*
* The structure to hold data for an HTTP error response.
* An error response is sent later in an unlocked queue context.
Expand Down Expand Up @@ -488,11 +472,4 @@ int tfw_http_send_504(TfwHttpReq *req, const char *reason);
void *tfw_msg_setup(TfwHttpMsg *hm, size_t len);
void tfw_msg_add_data(void *handle, TfwMsg *msg, char *data, size_t len);

/*
* HTTP session routines.
*/
int tfw_http_sess_obtain(TfwHttpReq *req);
int tfw_http_sess_resp_process(TfwHttpResp *resp, TfwHttpReq *req);
void tfw_http_sess_put(TfwHttpSess *sess);

#endif /* __TFW_HTTP_H__ */
Loading