Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Linux cgrpoup v2, issue-4885 #5068

Merged
merged 34 commits into from
Apr 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
87261f3
Add support for Linux cgrpoup v2, issue-4885
Mar 26, 2024
f32c9ad
Removed unnecessary using, ordered using, removed blank lines
Mar 27, 2024
da44f5f
Changed path for /sys/fs/cgroup/cpu/cpu.shares
Mar 27, 2024
3f75b40
Experimental attribute for GuaranteedPodCpuUnits added, removed blank…
Mar 27, 2024
87e64b8
Exprerimental attributes added
Mar 27, 2024
cda49b5
Removed the experimental parameter, agreed to reuse cpuGuaranteedRequest
Mar 28, 2024
a09bfc8
Acceptance test to support cgroupv2
Apr 4, 2024
0bc23ba
Removed unnessecary using
Apr 4, 2024
e80f7fb
Update src/Libraries/Microsoft.Extensions.Diagnostics.ResourceMonitor…
danmoseley Apr 4, 2024
de0ca6b
Added Exists method for HardcodedValueFileSystem
Apr 5, 2024
0a4d70e
Merge branch 'issue-4885' of https://github.com/nezdali/extensions in…
Apr 5, 2024
89ee8fe
Adjusted the Preprocessor symbol
Apr 5, 2024
c124d5a
Moved cgroup version detection to a separate class
Apr 5, 2024
cca3d60
Summary added for ResourceMonitoringLinuxCgroupVersion class
Apr 5, 2024
1f9c963
Made cgroup detection class internal
Apr 8, 2024
742d0a4
Additional tests to cover cgroupv2 parser
Apr 8, 2024
c188df2
New method to get directory names with pattern and multiple test fixes
Apr 11, 2024
af86bc6
GetDirectoryNames method regex implementation
Apr 11, 2024
9ac1be5
refactored GetDirectoryNames for OSFIleSystem
Apr 11, 2024
5baf6ba
Added test to OSFileSystemTests.cs for GetDirectoryNames
Apr 12, 2024
d950ad3
Fix in OSFileSystemTests.cs
Apr 12, 2024
91b0a4b
Excluded from code coverage logic that identifies version of Cgroups
Apr 12, 2024
d24f868
Removed unnecessary 'using', fix for GetCgroupRequestCpu to return cp…
Apr 12, 2024
881f54b
Merge branch 'issue-4885' of https://github.com/nezdali/extensions in…
Apr 12, 2024
e4c69ae
Analyzer error fix
Apr 12, 2024
46cc5bf
Test fix
Apr 12, 2024
507f2ed
Removed one test
Apr 12, 2024
4f6eb8b
updated config path to fixtures to make osfilesystem test pass
Apr 12, 2024
05a0994
Added ExcludeFromCodeCoverage for GetCgroupType based on nested setting
Apr 12, 2024
5484960
Additional test for GetMemoryUsageInBytesFromSlices
Apr 12, 2024
00f46f5
Tests for GetCgroupCpuUsageInNanoseconds and for GetAvailableMemoryIn…
Apr 12, 2024
d099770
Update src/Libraries/Microsoft.Extensions.Diagnostics.ResourceMonitor…
nezdali Apr 15, 2024
031caa7
Renamed LinuxUtilizationParser to LinuxUtilizationParserCgroupV1 and …
Apr 16, 2024
d191d28
Merge branch 'issue-4885' of https://github.com/nezdali/extensions in…
Apr 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,18 @@ namespace Microsoft.Extensions.Diagnostics.ResourceMonitoring.Linux;
/// </summary>
internal interface IFileSystem
{
/// <summary>
/// Checks for file existence.
/// </summary>
/// <returns> True/False.</returns>
bool Exists(FileInfo fileInfo);

/// <summary>
/// Get directory names on the filesystem based on the provided pattern.
/// </summary>
/// <returns>string.</returns>
string[] GetDirectoryNames(string directory, string pattern);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though this is an internal class I'd caution against using mutable arrays as return values. This could lead to some gnarly hard to track bugs.
Could this be changed to something like IList<string> or IEnumerable<string>?

Copy link
Contributor Author

@nezdali nezdali Apr 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, will do the changes in another PR since this one is merged already.


/// <summary>
/// Reads content from the file.
/// </summary>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

namespace Microsoft.Extensions.Diagnostics.ResourceMonitoring.Linux;

/// <summary>
/// An interface to be implemented by a parser that reads files and extracts resource utilization data from them.
/// For both versions of cgroup controllers, the parser reads files from the /sys/fs/cgroup directory.
/// </summary>
internal interface ILinuxUtilizationParser
{
/// <summary>
/// Reads file /sys/fs/cgroup/memory.max, which is used to check the maximum amount of memory that can be used by a cgroup.
/// It is part of the cgroup v2 memory controller.
/// </summary>
/// <returns>maybeMemory.</returns>
ulong GetAvailableMemoryInBytes();

/// <summary>
/// Reads the file /sys/fs/cgroup/cpu.stat, which is part of the cgroup v2 CPU controller.
/// It provides statistics about the CPU usage of a cgroup.
/// </summary>
/// <returns>nanoseconds.</returns>
long GetCgroupCpuUsageInNanoseconds();
xakep139 marked this conversation as resolved.
Show resolved Hide resolved

/// <summary>
/// Reads the file /sys/fs/cgroup/cpu.max, which is part of the cgroup v2 CPU controller.
/// It is used to set the maximum amount of CPU time that can be used by a cgroup.
/// The file contains two fields, separated by a space.
/// The first field is the quota, which specifies the maximum amount of CPU time (in microseconds) that can be used by the cgroup during one period.
/// The second value is the period, which specifies the length of a period in microseconds.
/// </summary>
/// <returns>cpuUnits.</returns>
float GetCgroupLimitedCpus();

/// <summary>
/// Reads the file /proc/stat, which provides information about the system’s memory usage.
/// It contains information about the total amount of installed memory, the amount of free and used memory, and the amount of memory used by the kernel and buffers/cache.
/// </summary>
/// <returns>memory.</returns>
ulong GetHostAvailableMemory();

/// <summary>
/// Reads the file /sys/fs/cgroup/cpuset.cpus.effective, which is part of the cgroup v2 cpuset controller.
/// It shows the effective cpus that the cgroup can use.
/// </summary>
/// <returns>cpuCount.</returns>
float GetHostCpuCount();

/// <summary>
/// Reads the file /sys/fs/cgroup/cpu.stat, which is part of the cgroup v2 CPU controller.
/// It provides statistics about the CPU usage of a cgroup.
/// The file contains several fields, including usage_usec, which shows the total CPU time (in microseconds).
/// </summary>
/// <returns>total / (double)_userHz * NanosecondsInSecond.</returns>
long GetHostCpuUsageInNanoseconds();

/// <summary>
/// Reads the file /sys/fs/cgroup/memory.current, which is a file that contains the current memory usage of a cgroup in bytes.
/// </summary>
/// <returns>memoryUsage.</returns>
ulong GetMemoryUsageInBytes();

/// <summary>
/// Reads the file /sys/fs/cgroup/cpu.weight. And calculates the Pod CPU Request in millicores.
/// </summary>
/// <returns>cpuPodRequest.</returns>
float GetCgroupRequestCpu();
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,14 @@
namespace Microsoft.Extensions.Diagnostics.ResourceMonitoring.Linux;

/// <remarks>
/// Parses Linux files to retrieve resource utilization data.
/// Parses Linux cgroup v1 files to retrieve resource utilization data.
/// This class is not thread safe.
/// When the same instance is called by multiple threads it may return corrupted data.
/// </remarks>
internal sealed class LinuxUtilizationParser
internal sealed class LinuxUtilizationParserCgroupV1 : ILinuxUtilizationParser
{
private const float CpuShares = 1024;

/// <remarks>
/// File contains the amount of CPU time (in microseconds) available to the group during each accounting period.
/// </remarks>
Expand Down Expand Up @@ -76,11 +78,16 @@ internal sealed class LinuxUtilizationParser
/// </remarks>
private static readonly FileInfo _cpuacctUsage = new("/sys/fs/cgroup/cpuacct/cpuacct.usage");

/// <summary>
/// CPU weights, also known as shares in cgroup v1, is used for resource allocation.
/// </summary>
private static readonly FileInfo _cpuPodWeight = new("/sys/fs/cgroup/cpu/cpu.shares");

private readonly IFileSystem _fileSystem;
private readonly long _userHz;
private readonly BufferWriter<char> _buffer = new();

public LinuxUtilizationParser(IFileSystem fileSystem, IUserHz userHz)
public LinuxUtilizationParserCgroupV1(IFileSystem fileSystem, IUserHz userHz)
{
_fileSystem = fileSystem;
_userHz = userHz.Value;
Expand Down Expand Up @@ -160,6 +167,16 @@ public float GetCgroupLimitedCpus()
return GetHostCpuCount();
}

public float GetCgroupRequestCpu()
{
if (TryGetCgroupRequestCpu(_fileSystem, out var cpuUnits))
{
return cpuUnits;
}

return GetHostCpuCount();
}

public ulong GetAvailableMemoryInBytes()
{
const long UnsetCgroupMemoryLimit = 9_223_372_036_854_771_712;
Expand Down Expand Up @@ -425,4 +442,35 @@ private bool TryGetCpuUnitsFromCgroups(IFileSystem fileSystem, out float cpuUnit
cpuUnits = (float)quota / period;
return true;
}

/// <summary>
/// In cgroup v1 the CPU shares is used to determine the CPU allocation.
/// in cgroup v2 the CPU weight is used to determine the CPU allocation.
/// To calculete CPU request in cgroup v2 we need to read the CPU weight and convert it to CPU shares.
/// But for cgroup v1 we can read the CPU shares directly from the file.
/// 1024 equals 1 CPU core.
/// In cgroup v1 on some systems the location of the CPU shares file is different.
/// </summary>
private bool TryGetCgroupRequestCpu(IFileSystem fileSystem, out float cpuUnits)
{
if (!_fileSystem.Exists(_cpuPodWeight))
{
cpuUnits = 0;
return false;
}

fileSystem.ReadFirstLine(_cpuPodWeight, _buffer);
var cpuPodWeightBuffer = _buffer.WrittenSpan;
_ = GetNextNumber(cpuPodWeightBuffer, out var cpuPodWeight);

if (cpuPodWeightBuffer.IsEmpty || (cpuPodWeightBuffer.Length == 2 && cpuPodWeightBuffer[0] == '-' && cpuPodWeightBuffer[1] == '1'))
{
Throw.InvalidOperationException($"Could not parse '{_cpuPodWeight}' content. Expected to find CPU weight but got '{new string(cpuPodWeightBuffer)}' instead.");
}

_buffer.Reset();
var result = cpuPodWeight / CpuShares;
cpuUnits = result;
return true;
}
}
Loading