Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cgroup.cpuacct percentages #25057

Merged

Conversation

fearful-symmetry
Copy link
Contributor

What does this PR do?

This PR addresses #23391 and adds percentage values to the cpuacct cgroup. Although that change is fairly straightforward, I felt the need to do some refactoring to make it nice--right now, the process metricset tends to divide work between the metricset itself and libbeat in a way that doesn't seem consistent, as libbeat does almost all the heavy lifting for regular process metrics, and then for cgroup data we do all the work in the metricset.

In order to emulate how process calculates regular cpu percentages, I just had libbeat manage cgroup data the way in manages all the other process data, so we just need to add the bits that deals with cgroup logic, and everything else is handled by the existing process process code. As a result of this, most of the cgroup logic has now been moved to libbeat, since I felt it made organizational sense to not flip-flop between doing the MapStr manipulation between two different libraries.

Why is it important?

People are asking for the enhancement in #23391

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

  • Pull down, build
  • Run system/process on a host with cgroups, and cgroup processes running.
  • Look for the percentages in system.process.cgroups.cpuacct, make sure the values are there, and that they make sense.

Related issues

@fearful-symmetry fearful-symmetry requested a review from a team April 13, 2021 22:54
@fearful-symmetry fearful-symmetry self-assigned this Apr 13, 2021
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Apr 13, 2021
@fearful-symmetry fearful-symmetry added the Team:Integrations Label for the Integrations team label Apr 13, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Apr 13, 2021
@fearful-symmetry fearful-symmetry changed the title Initial commit of cgroup cpuacct percentages Add cgroup.cpuacct percentages Apr 13, 2021
@fearful-symmetry fearful-symmetry removed the request for review from a team April 13, 2021 22:57
@fearful-symmetry fearful-symmetry requested a review from a team April 13, 2021 23:11
@elasticmachine
Copy link
Collaborator

elasticmachine commented Apr 14, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #25057 updated

  • Start Time: 2021-04-14T17:25:06.340+0000

  • Duration: 70 min 59 sec

  • Commit: dfc2cf1

Test stats 🧪

Test Results
Failed 0
Passed 46474
Skipped 5025
Total 51499

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 46474
Skipped 5025
Total 51499

libbeat/metric/system/process/cgroup.go Outdated Show resolved Hide resolved
libbeat/metric/system/process/process.go Outdated Show resolved Hide resolved
libbeat/metric/system/process/process.go Outdated Show resolved Hide resolved
@fearful-symmetry
Copy link
Contributor Author

@jsoriano some good suggestions, thanks!

@fearful-symmetry fearful-symmetry requested review from jsoriano and a team April 14, 2021 16:28
Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A possible leftover and some nitpicking, for the rest it LGTM 👍

libbeat/metric/system/process/process.go Outdated Show resolved Hide resolved
libbeat/metric/system/process/process.go Outdated Show resolved Hide resolved
libbeat/metric/system/process/process.go Outdated Show resolved Hide resolved
libbeat/metric/system/process/process.go Outdated Show resolved Hide resolved
@fearful-symmetry fearful-symmetry merged commit b2f22fa into elastic:master Apr 14, 2021
fearful-symmetry added a commit to fearful-symmetry/beats that referenced this pull request Apr 14, 2021
* initial commit of cgroup cpuacct percentages

* add in percpu flag

* fix up pct function

* formatting, name cleanup

* remove old code

(cherry picked from commit b2f22fa)
v1v added a commit to v1v/beats that referenced this pull request Apr 15, 2021
* upstream/master:
  packer cache support for the 7.x and 7.latestMinor branches (elastic#25091)
  Remove EventFetcher and EventsFetcher interface (elastic#25093)
  Update go-structform to 0.0.8 (elastic#25051)
  Update copy_fields.asciidoc (elastic#25053)
  [elastic-agent] ensure container is backwards compatible (elastic#25092)
  Add --fleet-server-service-token. Rename --fleet-server to --fleet-server-es. (elastic#25083)
  Add cgroup.cpuacct percentages (elastic#25057)
  Add tests for truncated and symlinked files in filestream input (elastic#24425)
  Fix panic when Hearbeat monitor initialization fails twice (elastic#25073)
  [Filebeat][httpjson] Change append transform to initiate new fields as a slice (elastic#25074)
  Osquerybeat: Result values type translation (elastic#25012)
  Update Osquerybeat spec to get it downloading from the correct artifactory path (elastic#25076)
  Fix changelog (elastic#25079)
  Strip Azure EventHub connection string in debug logs (elastic#25066)
  Change googlecloud to gcp in field names (elastic#25038)
  Bump stack version to 7.12.0 for testing (elastic#24957)
  packer-cache: cache the existing docker images on ARM and some more (elastic#25068)
  Disable logstash TestFetch flaky test (elastic#25044)
fearful-symmetry added a commit that referenced this pull request Apr 19, 2021
* Add cgroup.cpuacct percentages (#25057)

* initial commit of cgroup cpuacct percentages

* add in percpu flag

* fix up pct function

* formatting, name cleanup

* remove old code

(cherry picked from commit b2f22fa)

* bad merge

* revert logging
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Team:Integrations Label for the Integrations team v7.13.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants