metrics don't emit max_memory

[tgross]

We emit the allocated memory for tasks but not the max_memory for oversubscription, which means you can't use the metrics endpoint to track how close you are to OOM when oversubscribed.

cc @angrycub

[danishprakash]

@tgross is this something that a new contributor can pick up? would like to work on this.

[tgross]

Hi @danishprakash! It should be reasonably approachable. We emit the metrics for allocated memory in setGaugeForAllocationStats so it should be similar to that. Thanks!

[danishprakash]

@tgross thanks for the response. I ran into some issues while running make test which looks like something to do with my system. (I followed the instructions over at contributing though). Anyway, I've pushed the changes here, basically just returning MemoryMaxMB as part of ComparableResources struct and then exporting it as max_memory gauge.

[tgross]

@danishprakash just wanted to follow-up on this one... the branch you have up looks reasonable to me. If you're interested in opening a PR we could try to get that merged in. I realize it's been a while though, so if you don't have time I'd can pick it up.

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.