[stack monitoring][apm] use cgroup values for memory

[simitt]

from #79050

Stack Monitoring should use these for calculating more accurate CPU and memory usage when running Beats and APM Server in containers.

While testing 7.13 on cloud I realized that the changes added in #90873 only covered using cgroup values for CPU, but not for memory usage.
Unfortunately I missed this during the PR review.

The collected cgroup values are stored in beats_stats.metrics.beat.cgroup.memory.*. I suggest we show the beats_stats.metrics.beat.cgroup.memory.mem.usage.bytes and GC Next (unchanged) for cgroup data.

@igoristic can you also add the memory changes for 7.13? At the moment it is a bit surprising to see CPU cgroup vs. memory system usage.

cc @elastic/apm-server @ruflin

[elasticmachine]

Pinging @elastic/stack-monitoring (Team:Monitoring)

[igoristic]

@simitt
Do we need to add periods and quota in the calculation like we did for CPU?

[simitt]

There are no beats_stats.metrics.beat.cgroup.memory.stats.periods and beats_stats.metrics.beat.cgroup.memory.cfs.quota.us reported.

I suggest we

• show beats_stats.metrics.beat.cgroup.memory.mem.usage.bytes
• show beats_stats.metrics.beat.cgroup.memory.mem.limit.bytes (this will just be a straight line if a limit is set)
• keep showing the GC Next information
• remove the Allocated Memory and Process Total (as they are based on system metrics).

@axw are ok with this or did you have something different in mind?

[axw]

Sounds good to me.

[ravikesarwani]

remove the Allocated Memory and Process Total (as they are based on system metrics).

Can we please make sure the changes will work for both container and non container environments?
Also, it should work for APM and beats (filebeat, metricbeat ...) in general as well.

[simitt]

The changes should only affect container environments, and only affect the apm stack montioring.