Audit Event data structure

[yquansah]

As Flipt exploring the implementation of sending audit events to various sinks that can be configured on the server, we are exploring what data an audit event should contain. At the least, the audit event would include the resource being acted on, and the action taken. The other details of what should be in the payload is up for debate, as we would like to know and source certain ideas from the community

Example structure:

{
    "type": "flag",
    "action": "created",
    "payload": {
        "name": "this flag",
        "key": "this flag"
    }
}

The payload in the above would be the payload per request that comes in from a client, so these will be arbitrary per request and resource being acted on.

[GeorgeMac]

Perhaps the event structure should contain at-least three core components:

• Version
• Metadata
• Payload

Version

This would track a version for the structure of the event. We could do just straight numberic, semver or something similar to that of k8s resource versions (e.g. v1 v1beta1 and so on).
Either way, this should just support us breaking this contract in the future if needs be with a version increment.

Metadata

Place for describing what changed and how and other routing related information for consumers.
There should be some well-defined standard properties here.
The two you have given I think fit here nicely. Other kinds could be something like producer.

However, some space for additional arbitrary extras could be helpful too. Maybe someone will want to thread some other context through the events at some point.

Payload

Details specific to the combination of type and action. These would be well understood with respect to each unique comination of type and action. Exactly as you described above.

This is a slightly adjustment of what you have above with these ideas:

{
  "version": "v0.1",
  "metadata": {
    "type": "flag"
    "action": "created"
    "producer": "flipt-instance-0987654321"
  },
  "payload": {
    "name": "this flag"
    "key": "this flag"
  }
}

[yquansah]

@GeorgeMac I like this versioning concept due to the fact that the schema definitely could change for audit events.

[yquansah]

With the versioning, where would we house the different schemas for the different versions of the audit event payload? I am assuming this would mean that the user would supply what version they want of the audit event through configuration, and we have a mapping of version to schema in code?

[markphelps]

I also think we should capture what changed in the case of 'update' type events. so for example if the event was type: flag and action: updated we could include the following the payload:

{
  "payload": {
    "flag": {
      "key": "12355",
      "description": "so clean",
      "variants": [
      ],
    },
    "changed": {
      "description": {
        "from": "so fresh",
      }
    }
  }
}

I also think we should put the type as one of the keys in payload, or nest the object itself, that way we don't require as complicated parsing of the payload object. For example:

{
  "payload": {
    "flag": {
      "key": "12355",
      "description": "so clean",
      "variants": [
      ],
    }
}

[yquansah]

@markphelps For updated events, it might be a little tricky. It seems as though we Update the flag and set the fields without knowing the previous state of the flag: https://github.com/flipt-io/flipt/blob/main/internal/storage/sql/common/flag.go#L327-L332

[yquansah]

I wonder if there is potential to do pre-work in a gRPC interceptor or something for update requests to fetch the previous flag before we actually go and update it.

[markphelps]

yeah good point. i guess as a first pass we could leave out the changed attributes, then determine if there's a way to get them later on?

[yquansah]

@markphelps Yeah. Maybe a later version. The way that first came to mind was a gRPC interceptor, but definitely could leave that discussion up for a later time, and come up with the best solution.

[markphelps]

Closing as completed, see #1469