[macOS] psutil.Process(pid).cpu_percent calculation result is inaccurate

[RLYRLY]

Summary

• OS:macOS 13.6.5 (22G621)
• Architecture: ARM64
• Psutil version: 5.9.8
• Python version: 5.7.7
• Type: core, CPU

Description

By using psutil.Process(pid).cpu_percent(interval=1) to get the cpu utilization of a process, and then using the cpu utilization obtained by the top command or the htop command to verify the results of psutil, it is found that the results calculated by psutil are completely inconsistent.

example:

htop command: 4.1%

proc = psutil.Process(pid)
cpu_usage = proc.cpu_percent(interval=1)
print(f"Process '{proc.name()}' (PID: {proc.pid}) CPU utilization: {cpu_usage}%")

psutil:0.1%

[giampaolo]

How are you comparing the 2? If you want to compare with top/htop you should do it in a loop, like this:

p = psutil.Process(PID)
while 1:
    print(p.cpu_percent(interval=None)
    time.sleep(1)

[RLYRLY]

@giampaolo First of all, thank you for your help. I used loops for benchmarking and there is no change. Under arm64, the cpu utilization of the psutil calculation process is very different from that of htop calculation. In addition, I use other mac (inter chips), and the calculation results are basically the same as htop.

[RLYRLY]

@giampaolo Hello, please help me answer this question if you have time, it is very important to me.

[giampaolo]

Sorry but I have no idea. I don't have the hardware to test this against. Perhaps on arm64 process CPU metrics work differently and require using a different API. It would not be a first on macOS.

[RLYRLY]

@giampaolo Understood, thank you, if there is any progress in the future, please try to synchronize it. Let me investigate the specific reasons here.

[zhang25121]

I ran into the same problem on a 32 core window machine, and I found that on powerful machines, the CPU cores didn't always work

[dbwiddis]

I suspect the issue is a conversion based on the clock frequency somewhere.

x64 uses 1 tick per millisecond.
aarch64 uses 3 ticks per 125 milliseconds. You can find the conversion in the timebase_frequency field of any CPU in the IORegistry.

Here's a Java/JNA-based calculation of the offset: https://github.com/oshi/oshi/blob/099a4a1153effdfe4547c25052d2df4961e4d7eb/oshi-core/src/main/java/oshi/software/os/mac/MacOSProcess.java#L62-L88

If you multiply the psutil value by 41.6666... (125/3) do you get the expected result?

See also my answer at https://stackoverflow.com/a/76151406/1161484 with more detail including (untested) C code that might be appropriate here.

[dbwiddis]

As a follow-up, I see the existing code uses the CLK_TCK constant for division, which apparently doesn't work on arm64. The linked SO post in my previous comment also includes a few other native command calls available on arm64 mac which might be able to be used at compile time rather than my Java run-time hack.

In any case, I'm pretty sure the answer is as simple as "on Apple Silicon, multiply by 125/3" and finding the right way to define that in the code is something I'll leave to the C experts.

See also same bug/fix on my Java-based repo:

• Per-process Cpu Usage is incorrectly calculated on Apple Silicon oshi/oshi#2346
• Convert per-process CPU ticks on Apple Silicon to ms oshi/oshi#2361