Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libbeat: add support for defining unit and metric_type field metadata in ES templates #25419

Merged
merged 5 commits into from
May 1, 2021

Conversation

axw
Copy link
Member

@axw axw commented Apr 29, 2021

What does this PR do?

Add optional unit and metric_type properties to fields in fields.yml, which are used to set field metadata in Elasticsearch templates. These are allowed only for numeric field types, and we check the values against those defined in the Elasticsearch docs at https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-field-meta.html

Why is it important?

We would like to add unit/metric_type field metadata in fields defined by APM Server.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

axw added 3 commits April 29, 2021 14:07
Add Unit and MetricType properties to Field, which may
be set to "standard" unit and metric type metadata values
to aid in interpretation of the data.
@axw axw requested review from jsoriano and exekias April 29, 2021 06:11
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Apr 29, 2021
@axw axw added the Team:Elastic-Agent Label for the Agent team label Apr 29, 2021
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Apr 29, 2021
@axw axw changed the title Libbeat field unitmetrictype libbeat: add support for defining unit and metric_type field metadata in ES templates Apr 29, 2021
return fmt.Errorf("no format expected for field %s, found: %s", f.Name, f.Format)
}
allowedFormatters = []string{"date_range"}
case "boolean", "binary", "ip", "alias", "array":
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have moved "histogram" up to the numeric types

@axw axw marked this pull request as ready for review April 29, 2021 06:13
@elasticmachine
Copy link
Collaborator

Pinging @elastic/agent (Team:Agent)

@elasticmachine
Copy link
Collaborator

elasticmachine commented Apr 29, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #25419 updated

  • Start Time: 2021-04-30T07:34:33.680+0000

  • Duration: 134 min 46 sec

  • Commit: 06f2733

Test stats 🧪

Test Results
Failed 0
Passed 47167
Skipped 5246
Total 52413

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 47167
Skipped 5246
Total 52413

Copy link
Contributor

@exekias exekias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for tackling this! I left a comment about ES compatibility, otherwise LGTM

libbeat/template/processor.go Outdated Show resolved Hide resolved
libbeat/template/processor.go Outdated Show resolved Hide resolved
@axw axw requested a review from exekias April 30, 2021 07:34
@exekias
Copy link
Contributor

exekias commented Apr 30, 2021

For the record, we added this also to the package spec, I'm glad the we are using the same field names, so things are compatible between beats and packages 🎉 elastic/package-spec#42

@axw axw merged commit 23c546e into elastic:master May 1, 2021
@axw axw deleted the libbeat-field-unitmetrictype branch May 1, 2021 07:43
mergify bot pushed a commit that referenced this pull request May 1, 2021
… in ES templates (#25419)

* libbeat/mapping: add Unit and MetricType to Field

Add Unit and MetricType properties to Field, which may
be set to "standard" unit and metric type metadata values
to aid in interpretation of the data.

* libbeat/template: add meta.{metric_type,unit}

(cherry picked from commit 23c546e)
axw added a commit that referenced this pull request May 3, 2021
… in ES templates (#25419) (#25479)

* libbeat/mapping: add Unit and MetricType to Field

Add Unit and MetricType properties to Field, which may
be set to "standard" unit and metric type metadata values
to aid in interpretation of the data.

* libbeat/template: add meta.{metric_type,unit}

(cherry picked from commit 23c546e)

Co-authored-by: Andrew Wilkins <axw@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v7.14.0 Automated backport with mergify enhancement Team:Elastic-Agent Label for the Agent team v7.14.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants