Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic profiling support for supervisor #2586

Merged
merged 22 commits into from
Dec 2, 2021

Conversation

fyrestone
Copy link
Contributor

What do these changes do?

  • Add a enable_profiling extra config, default is False. e.g. a.execute(show_progress=False, extra_config={"enable_profiling": True})
  • Fix extra_config for web session.
  • Sample profile result for supervisor.
WARNING  mars.deploy.oscar.session:session.py:861 Profile task PO1e7wSe4DFpxzoFk5IKYiey execution result:
{
    "supervisor": {
        "general": {
            "optimize": 0.002045869827270508,
            "incref_fetch_tileables": 0.003548145294189453,
            "stage_g9ovAV2iVQQS20b2ym8lbrMt": {
                "tile": 0.02925896644592285,
                "gen_subtask_graph": 0.05370688438415527,
                "run": 0.19794082641601562,
                "total": 0.29035496711730957
            },
            "total": 0.299562931060791
        },
        "serialization": {
            "serialize": 0.014928340911865234,
            "deserialize": 0.0011813640594482422,
            "total": 0.016109704971313477
        }
    }
}

Related issue number

N/A

Check code requirements

  • tests added / passed (if needed)
  • Ensure all linting tests pass, see here for how to run them

@fyrestone fyrestone changed the title [WIP] Basic Profiling support for supervisor Basic Profiling support for supervisor Dec 1, 2021
Copy link
Member

@wjsi wjsi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wjsi wjsi changed the title Basic Profiling support for supervisor Add basic profiling support for supervisor Dec 2, 2021
Copy link
Contributor

@hekaisheng hekaisheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qinxuye qinxuye added this to the v0.9.0a1 milestone Dec 2, 2021
Copy link
Collaborator

@qinxuye qinxuye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qinxuye qinxuye merged commit ea8dc9f into mars-project:master Dec 2, 2021
@zhongchun zhongchun mentioned this pull request Mar 2, 2022
2 tasks
chaokunyang pushed a commit to chaokunyang/mars that referenced this pull request May 31, 2022
chaokunyang pushed a commit to chaokunyang/mars that referenced this pull request May 31, 2022
…isor (mars-project#2586)

Merge branch cp_2586 of git@gitlab.alipay-inc.com:ray-project/mars.git into master
https://code.alipay.com/ray-project/mars/pull_requests/131?tab=comment#note_18200047

Signed-off-by: 捕牛 <hejialing.hjl@antgroup.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants