Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pagerduty output support #164

Merged
merged 2 commits into from
Dec 24, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ Currently available outputs are :
* [**GCP PubSub**](https://cloud.google.com/pubsub)
* [**Google Chat**](https://workspace.google.com/products/chat/)
* [**Apache Kafka**](https://kafka.apache.org/)
* [**PagerDuty**](https://pagerduty.com/)

## Usage

Expand Down Expand Up @@ -266,6 +267,13 @@ kafka:
topic: "" # Name of the topic, if not empty, Kafka output is enabled
# partition: 0 # Partition number of the topic.
# minimumpriority: "debug" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)

pagerduty:
# apikey: # Pagerduty API Key, if not empty, Pagerduty output is enabled
service: "" # Service to create an incident (mandatory)
assignee: "" # A list of comma separated users to assign. Cannot be provided if pagerduty.escalationpolicy is already specified.
escalationpolicy: "" # Escalation policy to assign. Cannot be provided if pagerduty.escalationpolicy is already specified
# minimumpriority: "debug" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
```

Usage :
Expand Down Expand Up @@ -384,6 +392,11 @@ The *env vars* "match" field names in *yaml file with this structure (**take car
* **KAFKA_TOPIC**: The name of the Kafka topic
* **KAFKA_PARTITION**: The number of the Kafka partition
* **KAFKA_MINIMUMPRIORITY**: minimum priority of event for using this output, order is `emergency|alert|critical|error|warning|notice|informational|debug or "" (default)`
* **PAGERDUTY_APIKEY**: Pagerduty API Key, if not empty, Pagerduty output is *enabled*
* **PAGERDUTY_SERVICE**: Service to create an incident (mandatory)
* **PAGERDUTY_ASSIGNEE**: A list of comma separated users to assign. Cannot be provided if `PAGERDUTY_ESCALATION_POLICY` is already specified. If not empty, Pagerduty is *enabled*
* **PAGERDUTY_ESCALATION_POLICY**: Escalation policy to assign. Cannot be provided if `PAGERDUTY_ASSIGNEE` is already specified.If not empty, Pagerduty is *enabled*
* **PAGERDUTY_MINIMUMPRIORITY**: minimum priority of event for using this output, order is `emergency|alert|critical|error|warning|notice|informational|debug or "" (default)`

#### Slack/Rocketchat/Mattermost/Googlechat Message Formatting

Expand Down
6 changes: 6 additions & 0 deletions config.go
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,11 @@ func getConfig() *types.Configuration {
v.SetDefault("Kafka.Topic", "")
v.SetDefault("Kafka.Partition", 0)
v.SetDefault("Kafka.MinimumPriority", "")
v.SetDefault("Pagerduty.APIKey", "")
v.SetDefault("Pagerduty.Service", "")
v.SetDefault("Pagerduty.Assignee", []string{})
v.SetDefault("Pagerduty.EscalationPolicy", "")
v.SetDefault("Pagerduty.MinimumPriority", "")

v.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))
v.AutomaticEnv()
Expand Down Expand Up @@ -192,6 +197,7 @@ func getConfig() *types.Configuration {
c.GCP.PubSub.MinimumPriority = checkPriority(c.GCP.PubSub.MinimumPriority)
c.Googlechat.MinimumPriority = checkPriority(c.Googlechat.MinimumPriority)
c.Kafka.MinimumPriority = checkPriority(c.Kafka.MinimumPriority)
c.Pagerduty.MinimumPriority = checkPriority(c.Pagerduty.MinimumPriority)

c.Slack.MessageFormatTemplate = getMessageFormatTemplate("Slack", c.Slack.MessageFormat)
c.Rocketchat.MessageFormatTemplate = getMessageFormatTemplate("Rocketchat", c.Rocketchat.MessageFormat)
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ require (
cloud.google.com/go/pubsub v1.8.3
github.com/Azure/azure-event-hubs-go/v3 v3.3.3
github.com/DataDog/datadog-go v4.2.0+incompatible
github.com/PagerDuty/go-pagerduty v1.3.0
github.com/aws/aws-sdk-go v1.35.30
github.com/emersion/go-sasl v0.0.0-20200509203442-7bfe0ed36a21
github.com/emersion/go-smtp v0.14.0
Expand Down
4 changes: 4 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,8 @@ github.com/DataDog/datadog-go v4.2.0+incompatible h1:Q73jzyKHwyA04Gf4SSukRF+KR4w
github.com/DataDog/datadog-go v4.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ=
github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible/go.mod h1:r7JcOSlj0wfOMncg0iLm8Leh48TZaKVeNIfJntJ2wa0=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/PagerDuty/go-pagerduty v1.3.0 h1:2vWajLxpmGeP8pmsyZ0MjFneHa8ASJrztJRSe3FNOzg=
github.com/PagerDuty/go-pagerduty v1.3.0/go.mod h1:W5hSIIPrzSgAkNBDiuymWN5g9yQVzimL7BUBL44f3RY=
github.com/Shopify/sarama v1.19.0/go.mod h1:FVkBWblsNy7DGZRfXLU0O9RCGt5g3g3yEuWXgklEdEo=
github.com/Shopify/toxiproxy v2.1.4+incompatible/go.mod h1:OXgGpZ6Cli1/URJOF1DMxUHB2q5Ap20/P/eIdh4G0pI=
github.com/VividCortex/gohistogram v1.0.0/go.mod h1:Pf5mBqqDxYaXu3hDrrU+w6nw50o/4+TcAqDqk/vUH7g=
Expand Down Expand Up @@ -247,6 +249,8 @@ github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.2 h1:X2ev0eStA3AbceY54o37/0PQ/UWqKEiiO2dKL5OPaFM=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
Expand Down
4 changes: 4 additions & 0 deletions handlers.go
Original file line number Diff line number Diff line change
Expand Up @@ -219,4 +219,8 @@ func forwardEvent(falcopayload types.FalcoPayload) {
if config.Kafka.HostPort != "" && (priorityMap[strings.ToLower(falcopayload.Priority)] >= priorityMap[strings.ToLower(config.Kafka.MinimumPriority)] || falcopayload.Rule == TestRule) {
go kafkaClient.KafkaProduce(falcopayload)
}

if config.Pagerduty.APIKey != "" && config.Pagerduty.Service != "" && (priorityMap[strings.ToLower(falcopayload.Priority)] >= priorityMap[strings.ToLower(config.Pagerduty.MinimumPriority)] || falcopayload.Rule == TestRule) {
go pagerdutyClient.PagerdutyCreateIncident(falcopayload)
}
}
12 changes: 12 additions & 0 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ var (
gcpClient *outputs.Client
googleChatClient *outputs.Client
kafkaClient *outputs.Client
pagerdutyClient *outputs.Client

statsdClient, dogstatsdClient *statsd.Client
config *types.Configuration
Expand Down Expand Up @@ -315,6 +316,17 @@ func init() {
}
}

if config.Pagerduty.APIKey != "" && config.Pagerduty.Service != "" {
var err error
pagerdutyClient, err = outputs.NewPagerdutyClient(config, stats, promStats, statsdClient, dogstatsdClient)
if err != nil {
config.Pagerduty.APIKey = ""
config.Pagerduty.Service = ""
KeisukeYamashita marked this conversation as resolved.
Show resolved Hide resolved
} else {
enabledOutputsText += "Pagerduty "
}
}

log.Printf("%v\n", enabledOutputsText)
}

Expand Down
2 changes: 2 additions & 0 deletions outputs/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import (

"cloud.google.com/go/pubsub"
"github.com/DataDog/datadog-go/statsd"
"github.com/PagerDuty/go-pagerduty"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/segmentio/kafka-go"

Expand Down Expand Up @@ -53,6 +54,7 @@ type Client struct {
DogstatsdClient *statsd.Client
GCPTopicClient *pubsub.Topic
KafkaProducer *kafka.Conn
PagerdutyClient *pagerduty.Client
}

// NewClient returns a new output.Client for accessing the different API.
Expand Down
74 changes: 74 additions & 0 deletions outputs/pagerduty.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
package outputs

import (
"fmt"
"log"

"github.com/DataDog/datadog-go/statsd"
"github.com/PagerDuty/go-pagerduty"
"github.com/falcosecurity/falcosidekick/types"
)

// NewPagerdutyClient returns a new output.Client for accessing the Pagerduty API.
func NewPagerdutyClient(config *types.Configuration, stats *types.Statistics, promStats *types.PromStatistics, statsdClient, dogstatsdClient *statsd.Client) (*Client, error) {
if len(config.Pagerduty.Assignee) > 0 && config.Pagerduty.EscalationPolicy != "" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer we check this at creation of client.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return nil, fmt.Errorf("assignee and escalation policy cannot be both configured")
}

return &Client{
OutputType: "PagerDuty",
Config: config,
Stats: stats,
PromStats: promStats,
StatsdClient: statsdClient,
DogstatsdClient: dogstatsdClient,
PagerdutyClient: pagerduty.NewClient(config.Pagerduty.APIKey),
}, nil
}

// PagerdutyCreateIncident posts incident to Pagerduty
func (c *Client) PagerdutyCreateIncident(falcopayload types.FalcoPayload) {
c.Stats.Pagerduty.Add(Total, 1)

opts := &pagerduty.CreateIncidentOptions{
Type: "incident",
Title: falcopayload.Output,
Service: &pagerduty.APIReference{
ID: c.Config.Pagerduty.Service,
Type: "service_reference",
},
}

if len(c.Config.Pagerduty.Assignee) > 0 {
assignments := make([]pagerduty.Assignee, len(c.Config.Pagerduty.Assignee))
for i, a := range c.Config.Pagerduty.Assignee {
assignments[i] = pagerduty.Assignee{
Assignee: pagerduty.APIObject{
ID: a,
Type: "user_reference",
},
}
}
opts.Assignments = assignments
}

if policy := c.Config.Pagerduty.EscalationPolicy; policy != "" {
opts.EscalationPolicy = &pagerduty.APIReference{
ID: policy,
Type: "escalation_policy_reference",
}
}

if _, err := c.PagerdutyClient.CreateIncident("falcosidekick", opts); err != nil {
go c.CountMetric(Outputs, 1, []string{"output:pagerduty", "status:error"})
c.Stats.Pagerduty.Add(Error, 1)
c.PromStats.Outputs.With(map[string]string{"destination": "pagerduty", "status": Error}).Inc()
log.Printf("[ERROR] : PagerDuty - %v\n", err)
return
}

go c.CountMetric(Outputs, 1, []string{"output:pagerduty", "status:ok"})
c.Stats.Pagerduty.Add(OK, 1)
c.PromStats.Outputs.With(map[string]string{"destination": "pagerduty", "status": OK}).Inc()
log.Printf("[INFO] : Pagerduty - Create Incident OK\n")
}
1 change: 1 addition & 0 deletions stats.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ func getInitStats() *types.Statistics {
GCPPubSub: getOutputNewMap("gcppubsub"),
GoogleChat: getOutputNewMap("googlechat"),
Kafka: getOutputNewMap("kafka"),
Pagerduty: getOutputNewMap("pagerduty"),
}
stats.Falco.Add(outputs.Emergency, 0)
stats.Falco.Add(outputs.Alert, 0)
Expand Down
10 changes: 10 additions & 0 deletions types/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ type Configuration struct {
GCP gcpOutputConfig
Googlechat GooglechatConfig
Kafka kafkaConfig
Pagerduty pagerdutyConfig
}

// SlackOutputConfig represents parameters for Slack
Expand Down Expand Up @@ -239,6 +240,14 @@ type kafkaConfig struct {
MinimumPriority string
}

type pagerdutyConfig struct {
APIKey string
Service string
Assignee []string
EscalationPolicy string
MinimumPriority string
}

// Statistics is a struct to store stastics
type Statistics struct {
Requests *expvar.Map
Expand Down Expand Up @@ -270,6 +279,7 @@ type Statistics struct {
GCPPubSub *expvar.Map
GoogleChat *expvar.Map
Kafka *expvar.Map
Pagerduty *expvar.Map
}

// PromStatistics is a struct to store prometheus metrics
Expand Down