Include tileable property in detail api #2493

RandomY-2 · 2021-09-30T20:36:27Z

What do these changes do?

Tileable properties are included in the detail api, and we can later display these properties on web.

include tileable property in detail api

wjsi · 2021-10-01T02:44:43Z

mars/services/task/supervisor/processor.py

@@ -599,11 +599,27 @@ def get_tileable_details(self):
            else:
                status = SubtaskStatus.running

+            # get properties for tileable
+            all_propertes = {'slot': getattr(tileable, slot, None) for slot in tileable.__slots__}


I wonder if this list comprehension expression will work as it has only one key, slot. Please also try adding tests for your code.

changed all_properties

wjsi · 2021-10-04T11:32:25Z

mars/services/task/supervisor/processor.py

+
+                if isinstance(property_value, int) or isinstance(property_value, float) \
+                    or isinstance(property_value, str):
+                    displayed_properties[property_key] = property_value


Too complicated. two lines will cover all the needs.

props = {slot: getattr(tileable, slot, None) for slot in tileable.__slots__} props = {k: v for k, v in props.items() if k != 'key' and isinstance(v, (int, float, str)}

wjsi · 2021-10-04T11:34:31Z

mars/services/task/tests/test_service.py

@@ -384,6 +384,41 @@ def _get_fields(details, field, wrapper=None):
    details = await task_api.get_tileable_details(task_id)
    assert details[r7.key]['status'] == SubtaskStatus.errored.value

+    def a():


Is it possible to test with test cases generated previously?

Mars test

wjsi · 2021-10-05T02:24:54Z

mars/services/task/tests/test_service.py

@@ -384,6 +384,20 @@ def _get_fields(details, field, wrapper=None):
    details = await task_api.get_tileable_details(task_id)
    assert details[r7.key]['status'] == SubtaskStatus.errored.value

+    def is_valid_type(value):
+        return isinstance(value, int) or isinstance(value, float) or isinstance(value, str)


Multiple isinstances can be simplified with a single call on tuple of types, for inst., isinstance(value, (int, float, str)). Hence this utility function may not be needed.

simplify test

wjsi

LGTM

qinxuye

LGTM

Merge branch merge_github_2524 of git@gitlab.alipay-inc.com:ray-project/mars.git into master https://code.alipay.com/ray-project/mars/pull_requests/58?tab=diff Signed-off-by: 捕牛 <hejialing.hjl@antgroup.com> * [Ray] Support reconstructing worker (mars-project#2413) * Make cmdline support third party modules (mars-project#2454) Co-authored-by: hanguang <zhusiyuan.zsy@alibaba-inc.com> * Support visualizing subtask graphs on Mars Web (mars-project#2426) * Fix timeout error when waiting for a submitted task (mars-project#2457) * Print the error message when error happens in `TaskProcessor` (mars-project#2458) * Add nightly builds for docker images (mars-project#2456) * Fix misuse of `name` parameter in DataFrame align (mars-project#2469) * Fix hang when start sub pool fails (mars-project#2468) * Refine and unify subtask detail APIs (mars-project#2465) * Fix coverage for Azure pipeline (mars-project#2474) * Split tileable information and subtask graph into two tabs (mars-project#2480) * Support specified vineyard socket and skip the launching vineyardd process (mars-project#2481) * Basic reschedule subtask (mars-project#2467) * Compatible with scikit-learn 1.0 (mars-project#2486) Co-authored-by: hekaisheng <kaisheng.hks@alibaba-inc.com> * Fix wrong translation in cluster deployment. (mars-project#2489) * Fix bug that failed to execute query when there are multiple arguments (mars-project#2490) * Include tileable property in detail api (mars-project#2493) * Fix version of statsmodels to pass CI (mars-project#2497) * Implements `glm.LogisticRegression` (mars-project#2466) * Implements bagging sampling (mars-project#2496) * Refine MarsDMatrix & support more parameters for XGB classifier and regressor (mars-project#2498) * Fix output of df.groupby(as_index=False).size() (mars-project#2507) * Add preliminary implementations for ufunc methods (mars-project#2510) * Add doc for reading csv in oss (mars-project#2514) * [Ray] Fix serializing lambdas in web (mars-project#2512) * Add `make_regression` support for learn module (mars-project#2515) * Fix reduction result on empty series (mars-project#2520) * Fix df.loc when df is empty (mars-project#2524) * fix start subpool * fix test_kill_and_wait_timeout * fix autoscale timeout * fix ray larger clsuter fixture * Update ci ray package to 1.2.2 * remove python3.6 3.8 .39 ut and upgrade ray 3.7 image * echo python path * fix json decode error * fix bundle release timeout * fix remove placement group timeout * fix no_restart * fix ci * fix autoscale

RandomY-2 added 3 commits September 29, 2021 20:52

include tileable property in detail api

9d3828c

changed test

f95ee72

Merge pull request #3 from RandomY-2/mars-test

144a0ec

include tileable property in detail api

RandomY-2 requested review from qinxuye and wjsi as code owners September 30, 2021 20:36

RandomY-2 added 2 commits September 30, 2021 16:27

rerun

02b2e36

rerun

6fd00f9

wjsi reviewed Oct 1, 2021

View reviewed changes

RandomY-2 added 4 commits October 3, 2021 16:40

changed all_properties

50f8ac5

Merge pull request #4 from RandomY-2/mars-test

de830fa

changed all_properties

removed console log

c69ba0e

rerun

26900ec

wjsi reviewed Oct 4, 2021

View reviewed changes

RandomY-2 added 3 commits October 4, 2021 14:36

simplify property field

0a69aac

change test;

205ef58

Merge pull request #5 from RandomY-2/mars-test

de02ab2

Mars test

wjsi reviewed Oct 5, 2021

View reviewed changes

qinxuye added mod: web to be backported Indicate that the PR need to be backported to stable branch type: feature New feature labels Oct 5, 2021

qinxuye added this to the v0.8.0b2 milestone Oct 5, 2021

RandomY-2 added 2 commits October 5, 2021 21:13

simplify test

fa0a7bd

Merge pull request #6 from RandomY-2/mars-test

81960a2

simplify test

wjsi approved these changes Oct 6, 2021

View reviewed changes

qinxuye approved these changes Oct 6, 2021

View reviewed changes

qinxuye merged commit f862dac into mars-project:master Oct 6, 2021

RandomY-2 deleted the mars branch October 7, 2021 04:01

wjsi pushed a commit to wjsi/mars that referenced this pull request Oct 9, 2021

Include tileable property in detail api (mars-project#2493)

340ef50

wjsi mentioned this pull request Oct 9, 2021

[BACKPORT] Include tileable property in detail api (#2493) #2499

Merged

qinxuye pushed a commit that referenced this pull request Oct 9, 2021

[BACKPORT] Include tileable property in detail api (#2493) (#2499)

1ac7a1f

qinxuye added backported already PR has been backported and removed to be backported Indicate that the PR need to be backported to stable branch labels Oct 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Include tileable property in detail api #2493

Include tileable property in detail api #2493

RandomY-2 commented Sep 30, 2021

wjsi Oct 1, 2021 •

edited

Loading

wjsi Oct 4, 2021 •

edited

Loading

wjsi Oct 4, 2021

wjsi Oct 5, 2021 •

edited

Loading

wjsi left a comment

qinxuye left a comment

Include tileable property in detail api #2493

Include tileable property in detail api #2493

Conversation

RandomY-2 commented Sep 30, 2021

What do these changes do?

wjsi Oct 1, 2021 • edited Loading

Choose a reason for hiding this comment

wjsi Oct 4, 2021 • edited Loading

Choose a reason for hiding this comment

wjsi Oct 4, 2021

Choose a reason for hiding this comment

wjsi Oct 5, 2021 • edited Loading

Choose a reason for hiding this comment

wjsi left a comment

Choose a reason for hiding this comment

qinxuye left a comment

Choose a reason for hiding this comment

wjsi Oct 1, 2021 •

edited

Loading

wjsi Oct 4, 2021 •

edited

Loading

wjsi Oct 5, 2021 •

edited

Loading