Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: refinery_rules.md #802

Merged
merged 7 commits into from
Aug 1, 2023
Merged

docs: refinery_rules.md #802

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
622958a
updates to example config and datatypes
Jul 19, 2023
6ddb611
Move Terra's suggestions to the templates
mjingle Jul 20, 2023
fc3eb28
Remove manual change to markdown file
mjingle Jul 20, 2023
14be065
Remove Mary's auto formatting chaos
mjingle Jul 20, 2023
007f1eb
Rebase and regen
kentquirk Aug 1, 2023
263b5b7
Tweak and regen
kentquirk Aug 1, 2023
5b56087
Merge branch 'main' into terra.rules_doc_update
kentquirk Aug 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Honeycomb Refinery Configuration Documentation

This is the documentation for the configuration file for Honeycomb's Refinery.
It was automatically generated on 2023-07-17 at 17:49:09 UTC.
It was automatically generated on 2023-08-01 at 15:26:34 UTC.

## The Config file

Expand Down Expand Up @@ -158,7 +158,6 @@ AcceptOnlyListedKeys is a boolean flag that causes events arriving with API keys
If `true`, then only traffic using the keys listed in `ReceiveKeys` is accepted.
Events arriving with API keys not in the `ReceiveKeys` list will be rejected with an HTTP `401` error.
If `false`, then all traffic is accepted and `ReceiveKeys` is ignored.
Must be specified if `ReceiveKeys` is specified.

- Eligible for live reload.
- Type: `bool`
Expand Down Expand Up @@ -742,9 +741,11 @@ The number of traces in the cache should be many multiples (100x to 1000x) of th
AvailableMemory is the amount of system memory available to the Refinery process.

This value will typically be set through an environment variable controlled by the container or deploy script.
If set, then this must be a memory size and `Collections.maxAlloc` must not be defined to avoid unclear configuration.
If this value is zero or not set, then `MaxMemory` cannot be used to calculate the maximum allocation and `MaxAlloc` will be used instead.
If set, then this must be a memory size.
64-bit values are supported.
Sizes with standard unit suffixes (`MB`, `GiB`, etc.) and Kubernetes units (`M`, `Gi`, etc.) are also supported.
If set, `Collections.MaxAlloc` must not be defined.

- Eligible for live reload.
- Type: `memorysize`
Expand All @@ -770,10 +771,11 @@ Useful values for this setting are generally in the range of 70-90.

MaxAlloc is the maximum number of bytes that should be allocated by the Collector.

If set, then this must be a memory size and `Collections.AvailableMemory` must not be defined to avoid unclear configuration.
If set, then this must be a memory size.
64-bit values are supported.
Sizes with standard unit suffixes (`MB`, `GiB`, etc) and Kubernetes units (`M`, `Gi`, etc) are also supported.
See `MaxMemory` for more details.
If set, `Collections.AvailableMemory` must not be defined.

- Eligible for live reload.
- Type: `memorysize`
Expand Down
1 change: 1 addition & 0 deletions config/metadata/rulesMeta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -579,6 +579,7 @@ groups:
case-sensitive.
- name: Operator
type: string
valuetype: choice
validations:
- type: choice
choices:
Expand Down
4 changes: 2 additions & 2 deletions refinery_config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,6 @@ This list only applies to span traffic - other Honeycomb API actions will be pro
If `true`, then only traffic using the keys listed in `ReceiveKeys` is accepted.
Events arriving with API keys not in the `ReceiveKeys` list will be rejected with an HTTP `401` error.
If `false`, then all traffic is accepted and `ReceiveKeys` is ignored.
Must be specified if `ReceiveKeys` is specified.

- Eligible for live reload.
- Type: `bool`
Expand Down Expand Up @@ -731,6 +730,7 @@ If this value is zero or not set, then `MaxMemory` cannot be used to calculate t
If set, then this must be a memory size.
64-bit values are supported.
Sizes with standard unit suffixes (`MB`, `GiB`, etc.) and Kubernetes units (`M`, `Gi`, etc.) are also supported.
If set, `Collections.MaxAlloc` must not be defined.

- Eligible for live reload.
- Type: `memorysize`
Expand All @@ -746,7 +746,6 @@ If nonzero, then it must be an integer value between 1 and 100, representing the
If set to a non-zero value, then once per tick (see `SendTicker`) the collector will compare total allocated bytes to this calculated value.
If allocation is too high, then traces will be ejected from the cache early to reduce memory.
Useful values for this setting are generally in the range of 70-90.
If this value is `0`, then `MaxAlloc` will be used.

- Eligible for live reload.
- Type: `percentage`
Expand All @@ -761,6 +760,7 @@ If set, then this must be a memory size.
64-bit values are supported.
Sizes with standard unit suffixes (`MB`, `GiB`, etc) and Kubernetes units (`M`, `Gi`, etc) are also supported.
See `MaxMemory` for more details.
If set, `Collections.AvailableMemory` must not be defined.

- Eligible for live reload.
- Type: `memorysize`
Expand Down
10 changes: 6 additions & 4 deletions refinery_rules.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ The Refinery `rules` file is a YAML file.

## Example

Here is a simple example of a `rules` file:
Below is a simple example of a `rules` file.
For a complete example, [visit the Refinery GitHub repository](https://github.com/honeycombio/refinery/blob/main/rules_complete.yaml).

```yaml
RulesVersion: 2
Expand Down Expand Up @@ -48,7 +49,7 @@ It is not influenced by the contents of the trace other than the trace ID.

The sample rate to use.
It indicates a ratio, where one sample trace is kept for every N traces seen.
For example, a `SampleRate` of `30` will keep 1 out of every 30 traces.
For example, a `SampleRate` of `30` will keep 1 out of every 30 traces.
The choice on whether to keep any specific trace is random, so the rate is approximate.
The sample rate is calculated from the trace ID, so all spans with the same trace ID will be sampled or not sampled together.

Expand All @@ -67,7 +68,7 @@ This sampler uses the `AvgSampleRate` algorithm from that package.

The sample rate to use.
It indicates a ratio, where one sample trace is kept for every N traces seen.
For example, a `SampleRate` of `30` will keep 1 out of every 30 traces.
For example, a `SampleRate` of `30` will keep 1 out of every 30 traces.
The choice on whether to keep any specific trace is random, so the rate is approximate.
The sample rate is calculated from the trace ID, so all spans with the same trace ID will be sampled or not sampled together.

Expand Down Expand Up @@ -130,7 +131,7 @@ Every key will be represented at least once in any given window and more frequen

The sample rate to use.
It indicates a ratio, where one sample trace is kept for every N traces seen.
For example, a `SampleRate` of `30` will keep 1 out of every 30 traces.
For example, a `SampleRate` of `30` will keep 1 out of every 30 traces.
The choice on whether to keep any specific trace is random, so the rate is approximate.
The sample rate is calculated from the trace ID, so all spans with the same trace ID will be sampled or not sampled together.

Expand Down Expand Up @@ -487,6 +488,7 @@ The comparison operator to use.
String comparisons are case-sensitive.

- Type: `string`
- Options: `=`, `!=`, `>`, `<`, `>=`, `<=`, `starts-with`, `contains`, `does-not-contain`, `exists`, `not-exists`

### `Value`

Expand Down
13 changes: 8 additions & 5 deletions rules.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# Honeycomb Refinery Rules Documentation

This is the documentation for the rules configuration for Honeycomb's Refinery.
It was automatically generated on 2023-07-11 at 16:32:35 UTC.
It was automatically generated on 2023-08-01 at 15:26:34 UTC.

## The Rules file

The rules file is a YAML file.
Here is a simple example of a rules file:
Below is a simple example of a `rules` file.
View a [complete example](https://github.com/honeycombio/refinery/blob/main/rules_complete.yaml).

```yaml
RulesVersion: 2
Expand Down Expand Up @@ -66,7 +67,7 @@ It is not influenced by the contents of the trace other than the trace ID.

The sample rate to use.
It indicates a ratio, where one sample trace is kept for every N traces seen.
For example, a `SampleRate` of `30` will keep 1 out of every 30 traces.
For example, a `SampleRate` of `30` will keep 1 out of every 30 traces.
The choice on whether to keep any specific trace is random, so the rate is approximate.
The sample rate is calculated from the trace ID, so all spans with the same trace ID will be sampled or not sampled together.

Expand All @@ -88,7 +89,7 @@ This sampler uses the `AvgSampleRate` algorithm from that package.

The sample rate to use.
It indicates a ratio, where one sample trace is kept for every N traces seen.
For example, a `SampleRate` of `30` will keep 1 out of every 30 traces.
For example, a `SampleRate` of `30` will keep 1 out of every 30 traces.
The choice on whether to keep any specific trace is random, so the rate is approximate.
The sample rate is calculated from the trace ID, so all spans with the same trace ID will be sampled or not sampled together.

Expand Down Expand Up @@ -154,7 +155,7 @@ Every key will be represented at least once in any given window and more frequen

The sample rate to use.
It indicates a ratio, where one sample trace is kept for every N traces seen.
For example, a `SampleRate` of `30` will keep 1 out of every 30 traces.
For example, a `SampleRate` of `30` will keep 1 out of every 30 traces.
The choice on whether to keep any specific trace is random, so the rate is approximate.
The sample rate is calculated from the trace ID, so all spans with the same trace ID will be sampled or not sampled together.

Expand Down Expand Up @@ -527,6 +528,8 @@ String comparisons are case-sensitive.

Type: `string`

- Options: `=`, `!=`, `>`, `<`, `>=`, `<=`, `starts-with`, `contains`, `does-not-contain`, `exists`, `not-exists`

### `Value`

The value to compare against.
Expand Down
13 changes: 6 additions & 7 deletions tools/convert/templates/configV2.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
## Honeycomb Refinery Configuration ##
######################################
#
# created {{ now }} from {{ .Input }} using a template generated on 2023-07-11 at 17:52:53 UTC
# created {{ now }} from {{ .Input }} using a template generated on 2023-08-01 at 15:26:32 UTC

# This file contains a configuration for the Honeycomb Refinery. It is in YAML
# format, organized into named groups, each of which contains a set of
Expand Down Expand Up @@ -138,7 +138,6 @@ AccessKeys:
## accepted. Events arriving with API keys not in the `ReceiveKeys` list
## will be rejected with an HTTP `401` error.
## If `false`, then all traffic is accepted and `ReceiveKeys` is ignored.
## Must be specified if `ReceiveKeys` is specified.
##
## Eligible for live reload.
{{ conditional .Data "AcceptOnlyListedKeys" "nostar APIKeys" }}
Expand Down Expand Up @@ -751,8 +750,9 @@ Collection:
## not set, then `MaxMemory` cannot be used to calculate the maximum
## allocation and `MaxAlloc` will be used instead. If set, then this must
## be a memory size. 64-bit values are supported. Sizes with standard
## unit suffixes (`MB`, `GiB`, etc) and Kubernetes units (`M`, `Gi`, etc)
## are also supported.
## unit suffixes (`MB`, `GiB`, etc.) and Kubernetes units (`M`, `Gi`,
## etc.) are also supported. If set, `Collections.MaxAlloc` must not be
## defined.
##
## Eligible for live reload.
{{ memorysize .Data "AvailableMemory" "AvailableMemory" "4Gb" }}
Expand All @@ -766,8 +766,7 @@ Collection:
## per tick (see `SendTicker`) the collector will compare total allocated
## bytes to this calculated value. If allocation is too high, then traces
## will be ejected from the cache early to reduce memory. Useful values
## for this setting are generally in the range of 70-90. If this value is
## `0`, then `MaxAlloc` will be used.
## for this setting are generally in the range of 70-90.
##
## default: 75
## Eligible for live reload.
Expand All @@ -779,7 +778,7 @@ Collection:
## If set, then this must be a memory size. 64-bit values are supported.
## Sizes with standard unit suffixes (`MB`, `GiB`, etc) and Kubernetes
## units (`M`, `Gi`, etc) are also supported. See `MaxMemory` for more
## details.
## details. If set, `Collections.AvailableMemory` must not be defined.
##
## Eligible for live reload.
{{ memorysize .Data "MaxAlloc" "InMemCollector.MaxAlloc" "" }}
Expand Down
8 changes: 4 additions & 4 deletions tools/convert/templates/rules_docrepo.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ It was automatically generated {{ now }}.
## The Rules file

The rules file is a YAML file.
Here is a simple example of a rules file:
Below is a simple example of a `rules` file.
View a [complete example](https://github.com/honeycombio/refinery/blob/main/rules_complete.yaml).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kentquirk Not sure if the templates can support markdown format for links or not... Is this possible or what should be used instead?


```yaml
RulesVersion: 2
Expand Down Expand Up @@ -108,8 +109,7 @@ Example: `{{ $field.Example }}`

{{- end }}
{{ if eq $field.ValueType "choice" -}}
{{ printf "Options: %s" (join $field.Choices " ") }}

{{- end }}
- {{ printf "Options: `%s`" (join $field.Choices "`, `") }}
{{ end }}
{{- println -}}
{{- end -}}
5 changes: 3 additions & 2 deletions tools/convert/templates/rules_docsite.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ The Refinery `rules` file is a YAML file.

## Example

Here is a simple example of a `rules` file:
Below is a simple example of a `rules` file.
For a complete example, [visit the Refinery GitHub repository](https://github.com/honeycombio/refinery/blob/main/rules_complete.yaml).

```yaml
RulesVersion: 2
Expand Down Expand Up @@ -80,7 +81,7 @@ The remainder of this page describes the samplers that can be used within the `S
{{- println -}}
{{- end -}}
{{- if eq $field.ValueType "choice" -}}
- {{ printf "Options: %s" (join $field.Choices " ") }}
- {{ printf "Options: `%s`" (join $field.Choices "`, `") }}
{{- println -}}
{{- end }}
{{- println -}}
Expand Down