Commits on Mar 18, 2016

1. Add support for include_fields and drop_fields …

   include_fields - removes all the fields from the event except the configured fields
   drop_fields - removes only the configured fields
   
   Both actions receive fields as argument that can be of type map or a value. For example: "proc", "proc.cpu.total_p".
   In case a map is passed as an argument, all the fields inside the map are considered.
   Each event MUST contain the fields ["@timestamp","type"] and they cannot be removed from the event.
   
   When multiple filtering rules are defined, we start with a copy of the event and apply the filtering rules one by to the
   event.
      event -> filter rule 1 -> event 1 -> filter rule 2 -> event 2 ...
      where event is the initial event, event1 is the event resulted after applying "filter rule1" and it's considered
      input for the
      "filter rule 2" and so on.
   
   Problem encountered:
   
   The generic filtering expects to receive an event with known types, meaning that it contains MapStr and other
   primitive types and not structures. In case the event contains an unknown structure for libbeat (it's defined
   inside the Beat), then we do a marshal & unmarshal to conver it to a MapStr.
   
   Configuration:
   
   If include_fields=None or not defined, then all the fields are exported
   If include_fields=[], then only the mandatory fields are exported
   If drop_fields=[], then all the fields are exported
   
   The default values are:
   include_fields=None
   drop_fields=[]

   

   monicasarbu committed Mar 18, 2016
   Configuration menu

   • View commit details
   • Copy full SHA for 09ca0f4
   • Browse repository at this point

   Copy the full SHA

   09ca0f4 View commit details

   Browse the repository at this point in the history