Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metricbeat] Add new page statistics to system/memory metricset #15492

Merged
merged 11 commits into from
Jan 14, 2020

Conversation

fearful-symmetry
Copy link
Contributor

@fearful-symmetry fearful-symmetry commented Jan 13, 2020

See #13744

This adds a new set of page scan and reclaim metrics to the system/memory metricset, as requested in the original issue. I'm a tad hesitant do dump the entire /proc/vmstat metrics in here, since this metricset is relatively clean and has a a fairly consistent format that I'd rather not dirty with a vmstat.* group.

These are fairly high-level metrics, so this adds a new page_stats group for them.

Metrics added:

  • pgscan metrics, measuring pages scanned
  • pgsteal metrics, measuring pages reclaimed
  • pgfree metric, measuring pages freed by the system
  • two efficiency metrics, measuring the efficiency of page reclaim.

With regards to efficiency, I was hesitant to to copy the vmeff metric from sar, as the pgscan and pgsteal metrics we get are much more detailed than what sar is collecting, so I instead copied the vmeff metric into groups, that measure the same thing (reclaim efficiency) at a more detailed level.

Test Plan

  • Start metricbeat with the system/memory metricset enabled on a linux system
  • Look for this field in the data:
           "page_stats": {
                "direct_efficiency": {
                    "pct": 0.9976
                },
                "kswapd_efficiency": {
                    "pct": 0.6213
                },
                "pgfree": {
                    "pages": 4382105954
                },
                "pgscan_direct": {
                    "pages": 485820
                },
                "pgscan_kswapd": {
                    "pages": 77390925
                },
                "pgsteal_direct": {
                    "pages": 484631
                },
                "pgsteal_kswapd": {
                    "pages": 48081976
                }
            },

Copy link
Contributor

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I believe this level knowledge deserves a short paragraph in README.

@unixsurfer
Copy link

@fearful-symmetry Thanks a lot for your work! It looks great, I only had on comment about the description for the efficiency field.

@fearful-symmetry
Copy link
Contributor Author

Updated data.json and field descriptions.

@fearful-symmetry fearful-symmetry merged commit 0fd2250 into elastic:master Jan 14, 2020
fearful-symmetry added a commit to fearful-symmetry/beats that referenced this pull request Jan 14, 2020
…tic#15492)

* add new page statistics to system/memory

* update field descriptions

* make update

* update data.json

* update system tests

* update changelog

* fix conflicts

* try to fix system python tests

(cherry picked from commit 0fd2250)
fearful-symmetry added a commit that referenced this pull request Jan 14, 2020
…etricset (#15527)

* [Metricbeat] Add new page statistics to system/memory metricset (#15492)

* add new page statistics to system/memory

* update field descriptions

* make update

* update data.json

* update system tests

* update changelog

* fix conflicts

* try to fix system python tests

(cherry picked from commit 0fd2250)

* update fields

* fix readme
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants