[core] make is_gpu, is_actor, root_detached_id fields late bind to workers. #47212

rynewang · 2024-08-19T21:34:33Z

When starting a worker for a task, we have:

early bind: runtime env, is_gpu, is_actor
late bind: job_id, root_detached_id

Now, move the latter 3 fields from "early bind" into "late bind". The only early bind thing is now runtime_env.

This helps in worker reuse: the prestarted workers can be used as actors or gpu tasks, as long as they don't have runtime env.

Refactoring: move the "does this worker fit for this task" code into worker.cc. Simplify PopWorker a bit.

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

rynewang · 2024-08-21T00:45:36Z

python/ray/tests/test_state_api.py

-    # work, otherwise it should have created the 2 prestarted task-only
-    # workers prior to https://github.com/ray-project/ray/pull/33623
-    assert result.total == 6
+    assert result.total == 4


big win: saved 2 workers from starting

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

jjyao · 2024-08-26T03:35:06Z

src/ray/common/task/task_spec.h

-                 bool is_gpu,
-                 bool is_root_detached_actor);
+  /// \param serialized_runtime_env The JSON-serialized runtime env for this worker.
+  explicit WorkerCacheKey(std::string serialized_runtime_env);


I think we can completely remove WorkerCacheKey?

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

jjyao

LGTM

There are test failures

src/ray/common/task/task_spec.cc

src/ray/raylet/worker.cc

src/ray/raylet/worker.h

src/ray/raylet/worker_pool.cc

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>

…ields

Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>

jjyao · 2024-09-11T18:39:53Z

python/ray/serve/tests/test_metrics.py

    wait_for_condition(
-        lambda: len(get_metric_dictionaries("serve_deployment_request_counter")) == 2,
+        lambda: len(get_metric_dictionaries("serve_deployment_request_counter_total"))


now the counter metric name ends with _total

jjyao · 2024-09-11T18:41:12Z

python/ray/serve/tests/test_metrics.py

-    num_requests = get_metric_dictionaries("serve_deployment_request_counter")
-    assert len(num_requests) == 2
-    expected_output = {"route": "/f", "deployment": "f", "application": "app1"}
-    verify_metrics(num_requests[0], expected_output)


The test assumes the order of the metrics it gets (f then g) but this is not something guaranteed so it's flaky. This PR changes to compare the whole metrics in a set.

edoakes

serve test changes LGTM

…rkers. (ray-project#47212) Signed-off-by: Ruiyang Wang <rywang014@gmail.com> Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com> Co-authored-by: Jiajun Yao <jeromeyjj@gmail.com> Signed-off-by: ujjawal-khare <ujjawal.khare@dream11.com>

rynewang added 4 commits August 19, 2024 14:31

make 3 fields late bind

d18e310

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

add tests

aecfaea

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

more tests

f4773bd

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

refactor the Fits func to a non virtual one

17a1916

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

rynewang commented Aug 21, 2024

View reviewed changes

rynewang and others added 3 commits August 20, 2024 22:02

fix unreachable

c3eb61b

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

Merge branch 'master' into late-bind-3fields

4330b81

refactor: move the unfit func to worker pool

c04a3b9

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

rynewang mentioned this pull request Aug 22, 2024

[core] Decouple create worker vs pop worker request. #47268

Draft

jjyao added the go add ONLY when ready to merge, run all tests label Aug 26, 2024

jjyao reviewed Aug 26, 2024

View reviewed changes

remove WorkerCacheKey

f729d90

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

jjyao reviewed Aug 27, 2024

View reviewed changes

rynewang and others added 9 commits September 4, 2024 13:27

address cmts

c2696bf

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

fix cpp tests

915a337

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

fix pytset

a88a29f

Signed-off-by: Ruiyang Wang <rywang014@gmail.com>

Merge branch 'master' into late-bind-3fields

3880559

Merge branch 'master' into late-bind-3fields

9ecb5eb

up

db0884b

Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>

Merge branch 'master' of github.com:ray-project/ray into late-bind-3f…

6472f25

…ields

up

6ab3e2e

Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>

test

2ce1fbc

Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>

jjyao reviewed Sep 11, 2024

View reviewed changes

edoakes reviewed Sep 11, 2024

View reviewed changes

jjyao approved these changes Sep 11, 2024

View reviewed changes

jjyao merged commit 7910a5e into ray-project:master Sep 11, 2024
4 of 5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[core] make is_gpu, is_actor, root_detached_id fields late bind to workers. #47212

[core] make is_gpu, is_actor, root_detached_id fields late bind to workers. #47212

rynewang commented Aug 19, 2024

rynewang Aug 21, 2024

jjyao Aug 26, 2024

jjyao left a comment

jjyao Sep 11, 2024

jjyao Sep 11, 2024

edoakes left a comment

[core] make is_gpu, is_actor, root_detached_id fields late bind to workers. #47212

[core] make is_gpu, is_actor, root_detached_id fields late bind to workers. #47212

Conversation

rynewang commented Aug 19, 2024

rynewang Aug 21, 2024

Choose a reason for hiding this comment

jjyao Aug 26, 2024

Choose a reason for hiding this comment

jjyao left a comment

Choose a reason for hiding this comment

jjyao Sep 11, 2024

Choose a reason for hiding this comment

jjyao Sep 11, 2024

Choose a reason for hiding this comment

edoakes left a comment

Choose a reason for hiding this comment