Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store complete meta on worker and update supervisor meta via fetching from workers #2912

Merged
merged 2 commits into from
Apr 17, 2022

Conversation

qinxuye
Copy link
Collaborator

@qinxuye qinxuye commented Apr 12, 2022

What do these changes do?

This PR support storing complete meta on worker and update supervisor meta via fetching from workers.

Related issue number

Fixes #2908
#2791

Check code requirements

  • tests added / passed (if needed)
  • Ensure all linting tests pass, see here for how to run them

@qinxuye qinxuye force-pushed the enh/subtask_chunk_meta branch 7 times, most recently from 395251e to e56db8d Compare April 15, 2022 17:00
@qinxuye qinxuye marked this pull request as ready for review April 15, 2022 17:45
@qinxuye qinxuye requested review from wjsi and hekaisheng as code owners April 15, 2022 17:45
mars/tests/core.py Outdated Show resolved Hide resolved
Copy link
Contributor

@hekaisheng hekaisheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@wjsi wjsi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wjsi wjsi merged commit 1f18d80 into mars-project:master Apr 17, 2022
@qinxuye qinxuye deleted the enh/subtask_chunk_meta branch April 17, 2022 13:53
fyrestone pushed a commit to fyrestone/mars that referenced this pull request Apr 18, 2022
@fyrestone fyrestone mentioned this pull request Apr 18, 2022
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable worker meta to store large chunk meta like dtypes
3 participants