Monitoring solutions for Gatekeeper

[hzc12321]

Let's discuss about monitoring in Gatekeeper.

For any DDoS protection system, it's always important to view the detailed outcomes of traffic filtering. In the thesis, the measurement performed is "File Transfer Time" with fixed file size, with predefined attack type & volume.

However, in a real-world scenario, problems surfaced :

1. DDoS attack parameters are unknown, we have to passively handle all sorts of attacks that hit us. To find out what hit us, we need the raw details.
2. We can't effectively measure the file transfer time from the perspective of all legitimate senders. A more meaningful and accessible metric will be, for example, how much traffic of different flows are blocked, and how much are being let in.

For example, I want to know how much bps / pps of traffic are dropped / granted, preferably grouped by destination IP. If the raw details are available in the log, then I can setup some mechanism to parse and extract information to produce analytics.

I haven't found any documentation about Gatekeeper's log format and details. Therefore, what information can be found within the log the remains a question. Assuming that everything can be found in the log, if anyone has ever made a monitoring dashboard or reporting utilities to gain insights from the logs and is happy to share it, that would be a great help to complement Gatekeeper's ecosystem.

[AltraMayor]

Gatekeeper has aggregate information about the traffic, as you can see in the example log entry below.

GK/6 2024-12-11 18:24:26 NOTICE Basic measurements [tot_pkts_num = 391822, tot_pkts_size = 210544477, pkts_num_granted = 378547, pkts_size_granted = 203208695, pkts_num_request = 9582, pkts_size_request = 1078775, pkts_num_declined = 3663, pkts_size_declined = 4972031, tot_pkts_num_dropped = 3663, tot_pkts_size_dropped = 4972031, tot_pkts_num_distributed = 30, tot_pkts_size_distributed = 2220, flow_table_occupancy = 55723/250000000=0.0%]

You can aggregate log entries like that and plot the data. Currently, there is limited information on how to do it at section Exporting logs to InfluxDB with gkle of the wiki page Tips for Deployments.

The data is not broken down by destination IP address. There is general interest in improving the data logged, but the problem is not straightforward. Too much disk I/O can disrupt the performance of Gatekeeper servers during attacks. In addition, no one is working on this problem because higher priorities are being addressed.

Regarding your comment on the measurement done in Cody's PhD thesis, the measurement of file transfer time was used to emphasize the clients' experiences rather than to inform the policy.