Dask Distributor #1379
Merged
Dask Distributor #1379
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Nov 5, 2024
This stack of pull requests is managed by Graphite. Learn more about stacking. Join @dongreenberg and the rest of your teammates on |
dongreenberg
force-pushed
the
dask_distributor
branch
from
46592e3 to
e2a300b
Compare
dongreenberg
force-pushed
the
pytorch_distributor
branch
from
8f38d00 to
6bc6892
Compare
This was referenced Nov 6, 2024
dongreenberg
force-pushed
the
pytorch_distributor
branch
2 times, most recently
from
2536203 to
72eb158
Compare
dongreenberg
force-pushed
the
dask_distributor
branch
from
e2a300b to
81b2296
Compare
dongreenberg
force-pushed
the
pytorch_distributor
branch
from
72eb158 to
fcbff27
Compare
dongreenberg
force-pushed
the
dask_distributor
branch
from
81b2296 to
5790f90
Compare
dongreenberg
force-pushed
the
pytorch_distributor
branch
from
fcbff27 to
2e42546
Compare
dongreenberg
force-pushed
the
dask_distributor
branch
from
5790f90 to
a388440
Compare
dongreenberg
force-pushed
the
pytorch_distributor
branch
2 times, most recently
from
d196d6a to
79438ee
Compare
carolineechen
force-pushed
the
pytorch_distributor
branch
from
79438ee to
9d30bdf
Compare
dongreenberg
force-pushed
the
pytorch_distributor
branch
from
9d30bdf to
42ddf5e
Compare
dongreenberg
force-pushed
the
dask_distributor
branch
2 times, most recently
from
198c085 to
1ad9750
Compare
| @@ -141,6 +142,10 @@ def address(self, addr): | |||
| self.ips = self.ips or [None] | |||
| self.ips[0] = addr | |||
|
|
|||
| @property | |||
| def internal_ips(self): | |||
| return self.ips | |||
There was a problem hiding this comment.
self.ips refers to external ips right?
There was a problem hiding this comment.
Whatever Donny has done here works great with Dask, but yes -- I think it works because it's never used anywhere lol
There was a problem hiding this comment.
No it's because for a static cluster internal_ips and external_ips are the same, and we need the cluster to have an internal_ips property just like on_demand_cluster, or the dask (and ray) start sequence for each needs to be different (we need to use an internal ip to connect from the worker nodes to the head node).
dongreenberg
force-pushed
the
pytorch_distributor
branch
from
42ddf5e to
7fafce3
Compare
dongreenberg
force-pushed
the
dask_distributor
branch
from
1ad9750 to
f969f23
Compare
dongreenberg
changed the base branch from
pytorch_distributor
to
graphite-base/1379
dongreenberg
force-pushed
the
dask_distributor
branch
from
f969f23 to
04a85e3
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.