Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial Support for v3 Webhook Payloads #427

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions examples/webhooks/webhook_server.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@ package main

import (
"fmt"
"log"
"net/http"

log "github.com/sirupsen/logrus"
ChezCrawford marked this conversation as resolved.
Show resolved Hide resolved

"github.com/PagerDuty/go-pagerduty/webhookv3"
)

Expand Down Expand Up @@ -35,5 +36,18 @@ func handler(w http.ResponseWriter, r *http.Request) {
return
}

fmt.Fprintf(w, "received signed webhook")
log.Infof("Received signed webhook")

payload, err := webhookv3.ReadWebhookPayload(r)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
log.Errorf("%v", err)
return
}

event := payload.Event
dataType, _ := event.GetEventDataValue("type")
log.Infof("Event: %v, Event Type: %v, EventData Type: %v", event.ID, event.EventType, dataType)

fmt.Fprint(w, "OK\n")
}
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,6 @@ require (
github.com/mitchellh/cli v1.0.0
github.com/mitchellh/go-homedir v1.1.0
github.com/sirupsen/logrus v1.4.2
github.com/stretchr/testify v1.2.2
gopkg.in/yaml.v2 v2.2.2
)
87 changes: 87 additions & 0 deletions webhookv3/webhook_event.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
package webhookv3

import (
"encoding/json"
"fmt"
"strconv"

"github.com/PagerDuty/go-pagerduty"
)

// OutboundEventData is the unmarshalled data portion of the OutboundEvent.
type OutboundEventData struct {
Object map[string]interface{}
// The raw json data for use in structured unmarshalling.
RawData json.RawMessage
}

// OutboundEvent represents the event that is delivered in a V3 Webhook Payload.
// See https://developer.pagerduty.com/docs/ZG9jOjExMDI5NTkw-v3-overview#webhook-payload for more details.
type OutboundEvent struct {
ChezCrawford marked this conversation as resolved.
Show resolved Hide resolved
ID string `json:"id"`
EventType string `json:"event_type"`
ResourceType string `json:"resource_type"`
OccurredAt string `json:"occurred_at"`
Agent *pagerduty.APIReference `json:"agent"`
Data *OutboundEventData `json:"data"`
}

// WebhookPayload represents the full object delivered as a result of V3 Webhook Subscriptions.
// See https://developer.pagerduty.com/docs/ZG9jOjExMDI5NTkw-v3-overview#webhook-payload for more details.
type WebhookPayload struct {
Event OutboundEvent `json:"event"`
}

// UnmarshalJSON is a custom unmarshaller used to produce OutboundEventData
// for further processing.
func (e *OutboundEventData) UnmarshalJSON(data []byte) error {
if err := json.Unmarshal(data, &e.Object); err != nil {
return err
}

if err := e.RawData.UnmarshalJSON(data); err != nil {
return err
}

return nil
}

// GetEventDataValue returns a value from the e.Data object using the keys as a path
// or returns an error if the path does not point to a field.
//
// For example, `e.GetEventDataValue("type")` would return the `event.data.type` from a Webhook Payload.
// If the event type is `"incident"`, e.GetEventDataValue("priority", "id") would return the priority id.
// See the tests for additional examples.
func (e OutboundEvent) GetEventDataValue(keys ...string) (string, error) {
Copy link
Collaborator

@theckman theckman Mar 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd still personally like to see this become func GetEventDataValue(e OutboundEvent, keys ...string) (string, error) since it's treating the public fields of the OutboundEvent as input, and is not modifying internal state of the value.

return getDataValue(e.Data.Object, keys)
}

func getDataValue(d map[string]interface{}, keys []string) (string, error) {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One case I am still considering how we should handle is when this method is called using a set of keys that does not terminate in a string value. For example if the object was:

{
      "title": "A little bump in the road",
      "service": {
        "html_url": "https://acme.pagerduty.com/services/PF9KMXH",
        "id": "PF9KMXH",
        "self": "https://api.pagerduty.com/services/PF9KMXH",
        "summary": "API Service",
        "type": "service_reference"
      }
}

and this function was called with GetEventDataValue("service") you would get the stringified version of the service object which isn't super useful.

My gut feeling is to return an error for this case as well.

key := keys[0]
node := d[key]

for k := 1; k < len(keys); k++ {
key = keys[k]

switch n := node.(type) {
case []interface{}:
intKey, err := strconv.Atoi(key)
if err != nil {
return "", fmt.Errorf("cannot identify array element with key '%s'", key)
}
node = n[intKey]
continue
case map[string]interface{}:
node = n[key]
continue
default:
break
}
}

if node == nil {
return "", fmt.Errorf("JSON does not have field '%s'", key)
}

return fmt.Sprintf("%v", node), nil
}
58 changes: 58 additions & 0 deletions webhookv3/webhook_event_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
package webhookv3

import (
"encoding/json"
"testing"

"github.com/stretchr/testify/assert"
)

func TestWebhookPayload_UnmarshallJSON(t *testing.T) {
var wp WebhookPayload

data := `{ "event": { "id": "5ac64822-4adc-4fda-ade0-410becf0de4f", "event_type": "incident.priority_updated", "resource_type": "incident", "occurred_at": "2020-10-02T18:45:22.169Z", "agent": { "html_url": "https://acme.pagerduty.com/users/PLH1HKV", "id": "PLH1HKV", "self": "https://api.pagerduty.com/users/PLH1HKV", "summary": "Tenex Engineer", "type": "user_reference" }, "client": { "name": "PagerDuty" }, "data": { "id": "PGR0VU2", "type": "incident", "self": "https://api.pagerduty.com/incidents/PGR0VU2", "html_url": "https://acme.pagerduty.com/incidents/PGR0VU2", "number": 2, "status": "triggered", "title": "A little bump in the road", "service": { "html_url": "https://acme.pagerduty.com/services/PF9KMXH", "id": "PF9KMXH", "self": "https://api.pagerduty.com/services/PF9KMXH", "summary": "API Service", "type": "service_reference" }, "assignees": [ { "html_url": "https://acme.pagerduty.com/users/PTUXL6G", "id": "PTUXL6G", "self": "https://api.pagerduty.com/users/PTUXL6G", "summary": "User 123", "type": "user_reference" } ], "escalation_policy": { "html_url": "https://acme.pagerduty.com/escalation_policies/PUS0KTE", "id": "PUS0KTE", "self": "https://api.pagerduty.com/escalation_policies/PUS0KTE", "summary": "Default", "type": "escalation_policy_reference" }, "teams": [ { "html_url": "https://acme.pagerduty.com/teams/PFCVPS0", "id": "PFCVPS0", "self": "https://api.pagerduty.com/teams/PFCVPS0", "summary": "Engineering", "type": "team_reference" } ], "priority": { "html_url": "https://acme.pagerduty.com/account/incident_priorities", "id": "PSO75BM", "self": "https://api.pagerduty.com/priorities/PSO75BM", "summary": "P1", "type": "priority_reference" }, "urgency": "high", "conference_bridge": { "conference_number": "+1 1234123412,,987654321#", "conference_url": "https://example.com" }, "resolve_reason": null } } }`

err := json.Unmarshal([]byte(data), &wp)
assert.NoError(t, err)

oe := wp.Event

assert.Equal(t, "5ac64822-4adc-4fda-ade0-410becf0de4f", oe.ID)
assert.Equal(t, "incident.priority_updated", oe.EventType)
assert.Equal(t, "incident", oe.ResourceType)
assert.Equal(t, "2020-10-02T18:45:22.169Z", oe.OccurredAt)

assert.Equal(t, "PLH1HKV", oe.Agent.ID)
assert.Equal(t, "user_reference", oe.Agent.Type)
}

func TestWebhookEvent_GetEventDataValue(t *testing.T) {
var wp WebhookPayload

data := `{ "event": { "id": "5ac64822-4adc-4fda-ade0-410becf0de4f", "event_type": "incident.priority_updated", "resource_type": "incident", "occurred_at": "2020-10-02T18:45:22.169Z", "agent": { "html_url": "https://acme.pagerduty.com/users/PLH1HKV", "id": "PLH1HKV", "self": "https://api.pagerduty.com/users/PLH1HKV", "summary": "Tenex Engineer", "type": "user_reference" }, "client": { "name": "PagerDuty" }, "data": { "id": "PGR0VU2", "type": "incident", "self": "https://api.pagerduty.com/incidents/PGR0VU2", "html_url": "https://acme.pagerduty.com/incidents/PGR0VU2", "number": 2, "status": "triggered", "title": "A little bump in the road", "service": { "html_url": "https://acme.pagerduty.com/services/PF9KMXH", "id": "PF9KMXH", "self": "https://api.pagerduty.com/services/PF9KMXH", "summary": "API Service", "type": "service_reference" }, "assignees": [ { "html_url": "https://acme.pagerduty.com/users/PTUXL6G", "id": "PTUXL6G", "self": "https://api.pagerduty.com/users/PTUXL6G", "summary": "User 123", "type": "user_reference" } ], "escalation_policy": { "html_url": "https://acme.pagerduty.com/escalation_policies/PUS0KTE", "id": "PUS0KTE", "self": "https://api.pagerduty.com/escalation_policies/PUS0KTE", "summary": "Default", "type": "escalation_policy_reference" }, "teams": [ { "html_url": "https://acme.pagerduty.com/teams/PFCVPS0", "id": "PFCVPS0", "self": "https://api.pagerduty.com/teams/PFCVPS0", "summary": "Engineering", "type": "team_reference" } ], "priority": { "html_url": "https://acme.pagerduty.com/account/incident_priorities", "id": "PSO75BM", "self": "https://api.pagerduty.com/priorities/PSO75BM", "summary": "P1", "type": "priority_reference" }, "urgency": "high", "conference_bridge": { "conference_number": "+1 1234123412,,987654321#", "conference_url": "https://example.com" }, "resolve_reason": null } } }`

err := json.Unmarshal([]byte(data), &wp)
assert.NoError(t, err)

oe := wp.Event

value, _ := oe.GetEventDataValue("type")
assert.Equal(t, "incident", value)

value, _ = oe.GetEventDataValue("title")
assert.Equal(t, "A little bump in the road", value)

value, _ = oe.GetEventDataValue("service", "summary")
assert.Equal(t, "API Service", value)

value, err = oe.GetEventDataValue("not_a_field")
assert.Equal(t, "", value)
assert.Error(t, err)

value, _ = oe.GetEventDataValue("assignees", "0", "summary")
assert.Equal(t, "User 123", value)

value, err = oe.GetEventDataValue("assignees", "not_an_integer")
assert.Equal(t, "", value)
assert.Error(t, err)
}
22 changes: 22 additions & 0 deletions webhookv3/webhookv3.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
Expand Down Expand Up @@ -105,3 +106,24 @@ func calculateSignature(payload []byte, secret string) []byte {
mac.Write(payload)
return mac.Sum(nil)
}

func ReadWebhookPayload(r *http.Request) (*WebhookPayload, error) {
orb := r.Body

b, err := ioutil.ReadAll(io.LimitReader(r.Body, webhookBodyReaderLimit))
if err != nil {
return nil, fmt.Errorf("failed to read response body: %w", err)
}

defer func() { _ = orb.Close() }()
r.Body = ioutil.NopCloser(bytes.NewReader(b))

if len(b) == 0 {
return nil, ErrMalformedBody
}
Comment on lines +111 to +123
Copy link
Collaborator

@theckman theckman Feb 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per the io.Reader interface contract, you should try processing the data in b before you handle the io.ReadAll error. I think this block of code should be something like:

	orb := r.Body
	defer func() { _ = orb.Close() }()

	b, err := ioutil.ReadAll(io.LimitReader(r.Body, webhookBodyReaderLimit))
	r.Body = ioutil.NopCloser(bytes.NewReader(b))

	if len(b) == 0 {
		if err != nil {
			return nil, fmt.Errorf("failed to read response body: %w", err)
		}

		return nil, ErrMalformedBody
	}

This version does a bit more to ensure the *http.Request.Body will contain the request body we received, even if we only got a partial read of the data. That way consumers can read that partial body themselves and inspect it.

The one case this does not handle is when the io.LimitReader is violated. In that case they will get the .Body that we read so far, and the remaining bytes will be lost meaning the caller couldn't choose to fully read it even if they wanted to. That could be accomplished by using io.MultiReader(), but would require some reworking of my snippet above.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per the io.Reader interface contract, you should try processing the data in b before you handle the io.ReadAll error.
@theckman : Where are you seeing that? I don't see it mentioned in io.ReadAll or in the example provided there...

(IIRC, we got this code from the comments you provided in the previous PR.)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those snippets were things I probably hammered out on my mobile phone while responding back to the comment(s). In hindsight, I should have done a better job at communicating that I hacked them together pretty rapidly as an example. I'm sorry about that.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries but I would still like to get clarity on what is the "perfect" reusable HTTP body reader.

This is one of the reasons I was pushing for more focused functions that verify signatures and construct events in #326. I would like to focus my effort on providing solid support for V3 Webhooks rather than the details of how folks implement that functionality.


var wp WebhookPayload
err = json.Unmarshal(b, &wp)

return &wp, err
}