Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for EVE memory settings. #4164

Conversation

OhmSpectator
Copy link
Member

This commit introduces a detailed documentation file that explains the memory settings for EVE OS. It covers the following topics:

  • Overview of cgroups and their memory limits in EVE OS.
  • Detailed explanation of soft and hard memory limits.
  • Hierarchical structure of cgroups in EVE.
  • Specific memory settings for EVE services and user applications.
  • Recommendations for adjusting memory settings.
  • Explanation of VMM overhead and how to override it.
  • Handling of Out-Of-Memory (OOM) situations.

The documentation aims to provide clear guidelines for configuring memory settings to optimize the performance and stability of EVE OS.

@OhmSpectator OhmSpectator force-pushed the feature/add-memory-settings-documentation branch 3 times, most recently from e1c6738 to 4e827db Compare August 26, 2024 21:36
@OhmSpectator OhmSpectator self-assigned this Aug 26, 2024
This commit introduces a detailed documentation file that explains the
memory settings for EVE OS. It covers the following topics:

- Overview of cgroups and their memory limits in EVE OS.
- Detailed explanation of soft and hard memory limits.
- Hierarchical structure of cgroups in EVE.
- Specific memory settings for EVE services and user applications.
- Recommendations for adjusting memory settings.
- Explanation of VMM overhead and how to override it.
- Handling of Out-Of-Memory (OOM) situations.

The documentation aims to provide clear guidelines for configuring
memory settings to optimize the performance and stability of EVE OS.

Signed-off-by: Nikolay Martyanov <nikolay@zededa.com>
@OhmSpectator OhmSpectator force-pushed the feature/add-memory-settings-documentation branch from 4e827db to fc7ac44 Compare August 26, 2024 22:22
Copy link
Contributor

@deitch deitch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By all means, yes! More docs!

@deitch
Copy link
Contributor

deitch commented Aug 27, 2024

I don't see any need to wait for all of the long eden tests to run when this is just doc changes. @OhmSpectator go ahead and merge whenever you want.

@deitch
Copy link
Contributor

deitch commented Aug 27, 2024

Just FYI, #4161 makes the usage of cgroupsv1 explicit with cmdline parameter. That doesn't change anything here, but it prevents us from accidentally falling into v2.

@OhmSpectator
Copy link
Member Author

Just FYI, #4161 makes the usage of cgroupsv1 explicit with cmdline parameter. That doesn't change anything here, but it prevents us from accidentally falling into v2.

Yeah, I already saw it. Actually, I want to move to v2 at some point as it provides more controls. But I have no idea what has to be changed for that.

@deitch
Copy link
Contributor

deitch commented Aug 27, 2024

Actually, I want to move to v2 at some point as it provides more controls

I think @rouming was looking at that, but I could be wrong.

@OhmSpectator
Copy link
Member Author

I don't see any need to wait for all of the long eden tests to run when this is just doc changes. @OhmSpectator go ahead and merge whenever you want.

Someone may want to add more information to the document or do a test read. I've spent quite a bit of time on it, and I might have missed some obvious errors or typos because my eyes are a bit blurred and I can't see the obvious anymore.

@deitch
Copy link
Contributor

deitch commented Aug 27, 2024

Up to you. This hardly will break anything if you missed something, it is a really big step above what we have today. I would say merge it, always can open another quick small docs PR if needed.

@OhmSpectator OhmSpectator merged commit 077cdcf into lf-edge:master Aug 27, 2024
33 of 35 checks passed
@OhmSpectator
Copy link
Member Author

Ok, let it be =D

@milan-zededa
Copy link
Contributor

Don't forget to link your doc from https://github.com/lf-edge/eve/blob/master/docs/mkdocs/mkdocs.yml


Cgroups are a Linux kernel feature that limits, accounts for, and isolates the
resource usage of a collection of processes. Cgroups are used to control the
memory, CPU, and I/O resources that a process can use.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also explain how cgroup hierarchy works since we are using it? Or maybe just add a link to some official Linux documentation?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will add

An indication that the EVE services require more memory can be the situation
when the EVE services are frequently hitting memory limits and the OOM killer is
fired with constraint `CONSTRAINT_NONE` and the `oom_memcg` value set to one of
the cgroups of the EVE services:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably forgot to list cgroups after the colon or the sentence should end with a fullstop.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ooops, yep... Will create a new PR soon

with constraint `CONSTRAINT_MEMCG` and the `oom_memcg` value set `eve` cgroup:

```text
oom-kill:constraint=CONSTRAINT_MEMCG,...,oom_memcg=/eve/,...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you already describe this somewhere, but where can the user see this oom-kill message, in dmesg and also in device logs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, dmesg and the logs on the controller (device index). Will add it.

#### GRUB override file

These settings can be changed using the `set_global` command in the
GRUB override file.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this changeable after the device is onboarded?
For example, if user notices that dom0 needs more memory, is he able to change the setting remotely without re-installing the device?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it can be set permanently by mounting the config partition and fixing the grub config.

Mount the configuration directory:

eve config mount /mnt

Add the desired kernel argument to the GRUB configuration:

echo 'set_global dom0_extra_args "<your_kernel_argument>"' >> /mnt/grub.cfg

Unmount the configuration directory:

eve config unmount

Reboot the device:

reboot

Will add it.

Copy link
Contributor

@milan-zededa milan-zededa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed documentation. I have added just few comments/questions.

@OhmSpectator
Copy link
Member Author

@milan-zededa here are the fixes: #4169

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants