[core] process commands are reported as "ray::IDLE" instead of the program name

[clarng]

What happened + What you expected to happen

when running top I notice the workers are named ray::IDLE even when they are executing:

e.g.

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

168935 ray 35 15 12.0g 2.4g 72072 R 86.1 7.6 0:21.64 ray::IDLE
168378 ray 35 15 11.8g 1.6g 619068 R 79.8 5.2 0:29.01 ray::IDLE
168377 ray 35 15 11.6g 1.5g 619488 R 79.1 4.8 0:27.96 ray::IDLE
168380 ray 35 15 11.7g 1.6g 618816 R 76.2 5.1 0:29.20 ray::IDLE
168376 ray 35 15 11.8g 1.6g 618976 R 72.5 5.2 0:29.03 ray::IDLE
168379 ray 35 15 11.7g 1.5g 618724 R 70.9 5.0 0:28.61 ray::IDLE

I expect them to be named after the actual process command when the worker is moved from the pool to the task

Versions / Dependencies

master

Reproduction script

ds = ray.data.range(10000000, parallelism=100)

def transform_batch(list) -> list:
return list

ds = ds.map_batches(transform_batch).fully_executed()

Issue Severity

Medium: It is a significant difficulty but I can work around it.

[clarng]

cc @scv119

[clarng]

seems related to dataset as the command works for trainer

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

168218 ray 35 15 12.1g 6.1g 5.9g S 99.0 19.7 14:19.02 ray::RayTrainWo
167599 ray 20 0 1725404 246528 27684 R 28.3 0.8 7:58.01 python

[rickyyx]

I think we keep workers around after finishing the tasks, so they could show up IDLE while not running the task (or running between tasks)? What if you block in the transform_batch - do they still show up as idle?