Skip to content

Latest commit

 

History

History
199 lines (162 loc) · 6.86 KB

README.md

File metadata and controls

199 lines (162 loc) · 6.86 KB
Raw

beaver Cookbook

Install beaver - the python daemon that munches on logs and sends their contents to logstash.

Requirements

Ohai and Chef:

This cookbook makes use of node['platform_family'] to simplify platform selection logic. This attribute was introduced in Ohai v0.6.12.

Platform:

The following platform families are supported:

  • Debian derivatives (Debian, Ubuntu, etc)
  • RHEL derivatives (RHEL, CentOS, etc)

Cookbooks

  • poise-python (to use the python_package LWRP)

Attributes

beaver::default

Key Type Description Default
['beaver']['version'] String Version to install via pip 29
['beaver']['log_path'] String Log path /var/log
['beaver']['log_file'] String Log file beaver.log
['beaver']['generate_keypair'] Boolean Whether to generate and expose keypair or not false
['beaver']['ssh_key_file'] String Basename of the keyfiles to generate logger
['beaver']['config_path'] String Configuration path /etc/beaver
['beaver']['config_file'] String Configuration file beaver.conf
['beaver']['user'] String User to run service as root
['beaver']['group'] String Group to run service as root
['beaver']['configuration'] Hash Key/Value configuration pairs { 'respawn_delay' => 3, 'max_failure' => 7 }
['beaver']['files'] Array Array containing hashes like { 'path' => '/var/log/syslog', 'type' => 'syslogs', 'tags' => 'sys, syslog' } for files to watch []
['beaver']['input_type']['tcp/ampq/etc'] Hash Key/Value input_types
['beaver']['output'] Hash Key/Value
['poise-python']['beaver']['provider'] String Python provider to install with. See poise-python for details. system
['poise-python']['beaver']['version'] String Python version to install. Blank means any. Beaver only supports 2 as of v36. '2'

Resources/Providers

Managing log files

This cookbook includes an LWRP for managing log files consumed by Beaver. It does so by dropping configuration snippets for each log file into Beaver's conf.d directory.

Actions

  • :create: Creates a config file and restarts Beaver to load it. (This is the default action)
  • :delete: Removes the config file.

Parameters

Parameter Type Description Default
name String Should be a name for the log file. e.g. 'syslog'
path String The path to the log file being monitored. (Required)
cookbook String Which cookbook contains the config file template, beaver-tail.conf.erb. beaver
format String What Logstash format should be used to send the log data. json_event
type String What Logstash type to associate with the log data. file
tags Array The Logstash tags to associate with the log data as an array of strings. []
add_field Array The Logstash field(s) to associate with the log data. An array of strings in the form ['fieldname1', 'fieldvalue1']. []
exclude String Which log files to exclude. Useful if using a file glob in the path parameter. The value must be a valid Python regex string.

Examples

# Monitor /var/log/syslog
beaver_tail "syslog" do
  path "/var/log/syslog"
  type "syslog"
  format "json_event"
end

# Follow all logs in /var/log except those with `messages` or `secure` in the name.
beaver_tail "system logs" do
  path "/var/log/*log"
  type "syslog"
  tags: ["sys"]
  exclude "(messages|secure)"
end

# Stop monitoring syslog
beaver_tail "syslog" do
  action :delete
end

Usage

beaver::default

Just include beaver in your node's run_list:

{
  "name":"my_node",
  "run_list": [
    "recipe[beaver]"
  ]
}

And configure beaver either in role or on the node. If you set ['beaver']['ssh']['generate_keypair'] to true the cokbook will create a public-key pair in the config_path and will expose the public key on the node. This key can then be searched by e.g. the logstash host and grant password-free access to tunnel e.g. redis (or anything else) through it.

Configuration example (role)

{
  "name": "logstash-client",
  "description": "",
  "json_class": "Chef::Role",
  "chef_type": "role",
  "default_attributes": {
    "beaver": {
      "generate_keypair": true,
      "configuration": {
        "transport": "redis",
        "redis_url": "redis://localhost:6379/0",
        "redis_namespace": "logstash:beaver",
        "ssh_key_file": "remote_key",
        "ssh_tunnel": "logging@logs.example.net",
        "ssh_tunnel_port": 6379,
        "ssh_remote_host": "127.0.0.1",
        "ssh_remote_port": "6379"
      }
    }
  },
  "run_list": [
    "recipe[beaver]"
  ],
}

Input Type Configuration (Hash)

  node.set['beaver']['input_type']['amqp'] = {
  'name' => "logstash_queue",
  'type' => "direct",
  'host' => "10.0.0.1",
  'exchange' => "logstash-exchange",
  'key' => "logstash-key",
  'exclusive' => false,
  'durable' => false,
  'auto_delete' => false,
}

Output Type Configuration (Hash)

node.set['beaver']['output'] = {
    'output' => 'stdout { debug => true }'
}

Contributing

  1. Fork the repository on Github
  2. Create a named feature branch (like add_component_x)
  3. Write you change
  4. Write tests for your change (if applicable)
  5. Run the tests, ensuring they all pass
  6. Submit a Pull Request using Github

Authors

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.