Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle cgroups v2 in OsProbe #77128

Merged

Conversation

pugnascotia
Copy link
Contributor

@pugnascotia pugnascotia commented Sep 1, 2021

Closes #76812. Closes #77126.

OsProbe was only capable of handle cgroup data in the v1 format.
However, Debian 11 uses cgroups v2 by default, and Elasticsearch isn't
capable of reporting any cgroup information. Therefore, add support for
the v2 layout.

Note that we have to open access to all of /sys/fs/cgroup because with
cgroups v2, the files we need are in an unpredictably location.

Closes elastic#76812.

`OsProbe` was only capable of handle cgroup data in the v1 format.
However, Debian 11 uses cgroups v2 by default, and Elasticsearch isn't
capable of reporting any cgroup information. Therefore, add support for
the v2 layout.
@elasticmachine elasticmachine added the Team:Core/Infra Meta label for core/infra team label Sep 1, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

permission java.io.FilePermission "/sys/fs/cgroup/cpuacct/-", "read";
permission java.io.FilePermission "/sys/fs/cgroup/memory", "read";
permission java.io.FilePermission "/sys/fs/cgroup/memory/-", "read";
permission java.io.FilePermission "/sys/fs/cgroup/-", "read";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This recursive file permission grant looks good to me.

@pugnascotia pugnascotia added the :Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts label Sep 1, 2021
@elasticmachine elasticmachine added the Team:Delivery Meta label for Delivery team label Sep 1, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-delivery (Team:Delivery)

@elasticsearchmachine
Copy link
Collaborator

Hi @pugnascotia, I've created a changelog YAML for you.

@pugnascotia
Copy link
Contributor Author

@elasticmachine update branch

@pugnascotia
Copy link
Contributor Author

@elasticmachine update branch

@pugnascotia
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/part-2

@pugnascotia pugnascotia merged commit d238374 into elastic:master Sep 3, 2021
@pugnascotia pugnascotia deleted the cgroups-v2-take2-reapply-take2 branch September 3, 2021 14:49
wjp719 added a commit to wjp719/elasticsearch that referenced this pull request Sep 4, 2021
* master: (128 commits)
  Mute DieWithDignityIT (elastic#77283)
  Fix randomization in MlNodeShutdownIT (elastic#77281)
  Add target_node_name for REPLACE shutdown type (elastic#77151)
  [DOCS] Adds information about version compatibility headers (elastic#77096)
  Fix template equals when mappings are wrapped (elastic#77008)
  Fix TextFieldMapper Retaining a Reference to its Builder (elastic#77251)
  Move die with dignity to be a test module (elastic#77136)
  Update task names for rest compatiblity (elastic#75267)
  [ML] adjusting bwc serialization for elastic#77256 (elastic#77257)
  Move `index.hidden` from Static to Dynamic settings (elastic#77218)
  Handle cgroups v2 in `OsProbe` (elastic#77128)
  Choose postings format from FieldMapper instead of MappedFieldType (elastic#77234)
  Add segment sorter for data streams (elastic#75195)
  Update skip after backport (elastic#77212)
  [ML] adding new defer_definition_decompression parameter to put trained model API (elastic#77189)
  [ML] Fix bug in inference stats persister for when feature reset is called
  Only check replicas in cancelling existing recoveries. (elastic#60564)
  Format `AbstractFilteringTestCase` (elastic#77217)
  [DOCS] Fixes line breaks. (elastic#77248)
  Convert 'routing' values in REST API tests to strings
  ...

# Conflicts:
#	server/src/main/java/org/elasticsearch/cluster/metadata/DataStream.java
pugnascotia added a commit to pugnascotia/elasticsearch that referenced this pull request Sep 6, 2021
Closes elastic#76812. Closes elastic#77126.

OsProbe was only capable of handle cgroup data in the v1 format.
However, Debian 11 uses cgroups v2 by default, and Elasticsearch isn't
capable of reporting any cgroup information. Therefore, add support for
the v2 layout.

Note that we have to open access to all of /sys/fs/cgroup because with
cgroups v2, the files we need are in an unpredictably location.
elasticsearchmachine pushed a commit that referenced this pull request Sep 6, 2021
* Handle cgroups v2 in `OsProbe` (#77128)

Closes #76812. Closes #77126.

OsProbe was only capable of handle cgroup data in the v1 format.
However, Debian 11 uses cgroups v2 by default, and Elasticsearch isn't
capable of reporting any cgroup information. Therefore, add support for
the v2 layout.

Note that we have to open access to all of /sys/fs/cgroup because with
cgroups v2, the files we need are in an unpredictably location.

* Handle a max memory value of 'max' (#77289)

* Handle a max memory value of 'max'

* Update docs/changelog/77289.yaml

* Delete 77289.yaml

* Fixes to backport

* Fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Core/Infra/Core Core issues without another label :Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts >enhancement Team:Core/Infra Meta label for core/infra team Team:Delivery Meta label for Delivery team v7.16.0 v8.0.0-alpha2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Elasticsearch can't start on my laptop and on some ci nodes cgroups stats not available on Debian 11
6 participants