Add support to set gc percentage #25394

urso · 2021-04-28T15:47:57Z

Bug

What does this PR do?

Introduce new setting to Beats: gc_percentage
The new setting sets GOGC, which configures the next gc limit. The
default value is 100. Setting the percentage less to 100 makes the Beats
trigger collection more often, due to a less next gc limit.

In Agent we add per process type gc limit environment variables:
APMSERVER_GOGC, FILEBEAT_GOGC, and METRICBEAT_GOGC.
These can be used to fine tune gc percentage limit per sub process type

Why is it important?

Allow users to set gc percentage.

Checklist

My code follows the style guidelines of this project
~~- [ ] I have commented my code, particularly in hard-to-understand areas~~
~~- [ ] I have made corresponding changes to the documentation~~
~~- [ ] I have made corresponding change to the default configuration files~~
~~- [ ] I have added tests that prove my fix is effective or that my feature works~~
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Set gc_percentage in the configuration file or via -E. The Beat will print a log message (info level) on startup.

Introduce new setting to Beats: gc_percentage The new setting sets GOGC, which configures the next gc limit. The default value is 100. Setting the percentage less to 100 makes the Beats trigger collection more often, due to a less next gc limit. In Agent we add per process type gc limit environment variables: APMSERVER_GOGC, FILEBEAT_GOGC, and METRICBEAT_GOGC. These can be used to fine tune gc percentage limit per sub process type

elasticmachine · 2021-04-28T15:48:10Z

Pinging @elastic/agent (Team:Agent)

urso · 2021-04-28T15:48:58Z

x-pack/elastic-agent/spec/apm-server.yml

+  "-E", "management.enabled=true",
+  "-E", "management.mode=x-pack-fleet",
+  "-E", "apm-server.data_streams.enabled=true"
+  "-E", "gc_percent=${APMSERVER_GOGC:100}"


This works because libbeat merges all -E flags into the configuration. The environment variable with default will be evaluated by libbeat.

blakerouse

Looks good.

blakerouse · 2021-04-28T15:52:56Z

libbeat/cmd/instance/beat.go

 		runtime.GOMAXPROCS(maxProcs)
 	}
+	if gcPercent := b.Config.GCPercent; gcPercent > 0 {
+		logp.Info("Set gc percentage to: %v", gcPercent)


Is it worth logging if the value is 100? Being that its the default.

We only log if it is configured. Might be helpful to see all message when we have to compare configs with logs output in support cases.

For the logs that are only logged once on startup (assuming this is correct) we should rather log too much then too little. Makes debugging easier.

elasticmachine · 2021-04-28T15:55:41Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #25394 updated
Start Time: 2021-05-01T20:38:42.808+0000
Duration: 133 min 39 sec
Commit: a379684

Test stats 🧪

Test	Results
Failed	0
Passed	47167
Skipped	5246
Total	52413

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	47167
Skipped	5246
Total	52413

ph · 2021-04-28T18:07:47Z

@urso you need to run make update in the elastic agent directory because you had updated the spec files.

urso · 2021-04-28T22:16:32Z

/test

urso · 2021-04-29T10:30:36Z

/test

ruflin · 2021-04-29T13:25:11Z

x-pack/elastic-agent/spec/metricbeat.yml

+  "-E", "management.mode=x-pack-fleet",
+  "-E", "management.enabled=true",
+  "-E", "logging.level=debug",
+  "-E", "gc_percent=${METRICBEAT_GOGC:100}"


FB / MB are enough for now, but I assume we will need to apply it to every Golang process / beat we run?

Yes, if we find this works well, we will extend on this and document the env vars.

ruflin · 2021-04-29T13:26:00Z

libbeat/cmd/instance/beat.go

 		runtime.GOMAXPROCS(maxProcs)
 	}
+	if gcPercent := b.Config.GCPercent; gcPercent > 0 {
+		logp.Info("Set gc percentage to: %v", gcPercent)


For the logs that are only logged once on startup (assuming this is correct) we should rather log too much then too little. Makes debugging easier.

urso · 2021-04-30T10:44:25Z

/test

mergify · 2021-05-01T07:46:37Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b libbeat-set-gc-percentage upstream/libbeat-set-gc-percentage
git merge upstream/master
git push upstream libbeat-set-gc-percentage

(cherry picked from commit b21b87d)

(cherry picked from commit b21b87d) # Conflicts: # x-pack/elastic-agent/pkg/agent/program/supported.go

Co-authored-by: Steffen Siering <steffen.siering@elastic.co>

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Apr 28, 2021

urso added the Team:Elastic-Agent Label for the Agent team label Apr 28, 2021

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Apr 28, 2021

urso commented Apr 28, 2021

View reviewed changes

urso added 2 commits April 28, 2021 17:51

fix typo

bb2cb0c

add changelog

afa2fba

blakerouse approved these changes Apr 28, 2021

View reviewed changes

ph added the v7.13.0 label Apr 28, 2021

ph assigned urso Apr 28, 2021

make update

b48c2e8

urso added backport-v7.13.0 Automated backport with mergify backport-v7.14.0 Automated backport with mergify labels Apr 28, 2021

add APMSERVER_GOGC to legacy container startup

fc063c2

ruflin approved these changes Apr 29, 2021

View reviewed changes

Merge branch 'master' into libbeat-set-gc-percentage

216d443

Merge branch 'master' into libbeat-set-gc-percentage

a379684

urso merged commit b21b87d into elastic:master May 2, 2021

urso deleted the libbeat-set-gc-percentage branch May 2, 2021 11:47

mergify bot pushed a commit that referenced this pull request May 2, 2021

Add support to set gc percentage (#25394)

4db6e76

(cherry picked from commit b21b87d)

mergify bot mentioned this pull request May 2, 2021

[7.x](backport #25394) Add support to set gc percentage #25487

Merged

mergify bot pushed a commit that referenced this pull request May 2, 2021

Add support to set gc percentage (#25394)

8bfe8fa

(cherry picked from commit b21b87d) # Conflicts: # x-pack/elastic-agent/pkg/agent/program/supported.go

mergify bot mentioned this pull request May 2, 2021

[7.13](backport #25394) Add support to set gc percentage #25488

Merged

urso pushed a commit that referenced this pull request May 3, 2021

[7.x](backport #25394) Add support to set gc percentage (#25487)

c0a4fce

Co-authored-by: Steffen Siering <steffen.siering@elastic.co>

urso pushed a commit that referenced this pull request May 3, 2021

[7.13](backport #25394) Add support to set gc percentage (#25488)

b4980c7

Co-authored-by: Steffen Siering <steffen.siering@elastic.co>

This was referenced May 3, 2021

[7.13] Update to elastic/beats@8551a8144fb4 elastic/apm-server#5218

Merged

[7.x] Update to elastic/beats@8d792d1ed1f9 elastic/apm-server#5219

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support to set gc percentage #25394

Add support to set gc percentage #25394

urso commented Apr 28, 2021 •

edited

Loading

elasticmachine commented Apr 28, 2021

urso Apr 28, 2021

blakerouse left a comment

blakerouse Apr 28, 2021

urso Apr 28, 2021

ruflin Apr 29, 2021

elasticmachine commented Apr 28, 2021 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

Trends 🧪

Test stats 🧪

ph commented Apr 28, 2021

urso commented Apr 28, 2021

urso commented Apr 29, 2021

ruflin Apr 29, 2021

urso Apr 29, 2021

ruflin Apr 29, 2021

urso commented Apr 30, 2021

mergify bot commented May 1, 2021

Add support to set gc percentage #25394

Add support to set gc percentage #25394

Conversation

urso commented Apr 28, 2021 • edited Loading

What does this PR do?

Why is it important?

Checklist

How to test this PR locally

elasticmachine commented Apr 28, 2021

urso Apr 28, 2021

Choose a reason for hiding this comment

blakerouse left a comment

Choose a reason for hiding this comment

blakerouse Apr 28, 2021

Choose a reason for hiding this comment

urso Apr 28, 2021

Choose a reason for hiding this comment

ruflin Apr 29, 2021

Choose a reason for hiding this comment

elasticmachine commented Apr 28, 2021 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

💚 Flaky test report

Test stats 🧪

ph commented Apr 28, 2021

urso commented Apr 28, 2021

urso commented Apr 29, 2021

ruflin Apr 29, 2021

Choose a reason for hiding this comment

urso Apr 29, 2021

Choose a reason for hiding this comment

ruflin Apr 29, 2021

Choose a reason for hiding this comment

urso commented Apr 30, 2021

mergify bot commented May 1, 2021

urso commented Apr 28, 2021 •

edited

Loading

elasticmachine commented Apr 28, 2021 •

edited by jenkins-beats-ci bot

Loading