Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ray] refine mars on ray usability #2580

Merged
merged 7 commits into from
Nov 18, 2021

Conversation

chaokunyang
Copy link
Contributor

@chaokunyang chaokunyang commented Nov 16, 2021

What do these changes do?

Refine mars on ray usability and documents.

Example1

import mars
import mars.dataframe as md
import mars.tensor as mt
session = mars.new_ray_session(cluster_name='abc', worker_num=2)
mt.random.RandomState(0).rand(1000_0000, 5, chunk_size=100_0000).sum().execute()
df = md.DataFrame(
    mt.random.rand(1000_0000, 4, chunk_size=500_0000),
    columns=list('abcd'))
print(df.sum().execute())

Example2

Create a mars cluster:

import mars.dataframe as md
import mars.tensor as mt
from mars.deploy.oscar import new_cluster_in_ray, new_ray_session
cluster = new_cluster_in_ray(cluster_name='test', worker_num=2)

then connect it in the client:

session = new_ray_session(address = cluster.address)

Related issue number

#2112

Check code requirements

  • tests added / passed (if needed)
  • Ensure all linting tests pass, see here for how to run them

@chaokunyang chaokunyang force-pushed the refine_mars_on_ray_usability branch from ed7e494 to 2d8f2ce Compare November 16, 2021 11:10
@qinxuye qinxuye added mod: ray integration to be backported Indicate that the PR need to be backported to stable branch type: enhancement request labels Nov 17, 2021
@qinxuye qinxuye added this to the v0.9.0a1 milestone Nov 17, 2021
@chaokunyang chaokunyang force-pushed the refine_mars_on_ray_usability branch 2 times, most recently from ce84f1e to a199d5d Compare November 17, 2021 13:51
@chaokunyang chaokunyang force-pushed the refine_mars_on_ray_usability branch from e7fc90c to f5365b0 Compare November 17, 2021 14:02
@chaokunyang
Copy link
Contributor Author

image

Copy link
Member

@wjsi wjsi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@qinxuye qinxuye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qinxuye qinxuye merged commit 35ffc16 into mars-project:master Nov 18, 2021
wjsi pushed a commit to wjsi/mars that referenced this pull request Dec 7, 2021
wjsi pushed a commit to wjsi/mars that referenced this pull request Dec 7, 2021
hekaisheng pushed a commit that referenced this pull request Dec 8, 2021
Co-authored-by: chaokunyang <chaokun.yck@antfin.com>
@wjsi wjsi added backported already PR has been backported and removed to be backported Indicate that the PR need to be backported to stable branch labels Dec 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants