Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run Task related refactoring #7696

Merged
merged 2 commits into from
May 1, 2020
Merged

Run Task related refactoring #7696

merged 2 commits into from
May 1, 2020

Conversation

RomanNikitenko
Copy link
Contributor

@RomanNikitenko RomanNikitenko commented Apr 29, 2020

What it does

1. Change the order of search task configuration at running
At the moment at running a task the order of search is:

  • search a config among provided/detected tasks
  • search a config among configured tasks
  • search a config among customized tasks

It works and sure it founds the corresponding config for running.
The problem is: it does extra requests when it tries to find a configured task configuration.

Let's say user runs some configured task:

I think we can change the order and do search among configured tasks first. It will allow to avoid extra requests: it just check if a task exists in map - we don't read tasks.json file at this step, so it can not slow down a search.
So:

  • if we are looking for a configured task, it finds the corresponding config and do not search among provided tasks
  • if we are looking for a provided task, it just check if a task exists in map, then it tries to find among provided tasks

2. Separate running a compound task
At the moment for any task config we do extra logic related to running a compound task.

But on the next step we need it if a task is compound only.

My propose is: separate this logic in another method and execute it if it's really required.

How to test

It's just refactoring, so we should avoid a regression.

  1. Try to run any configured task
  2. Try to run any detected task
  3. Try to run a compound task

For compound tasks testing I used in tasks.json file configurations like:

"tasks": [
    {
      "type": "shell",
      "label": "1111",
      "command": "sleep 2 && echo 111111",
      "dependsOn": "2222"
    },
    {
      "type": "shell",
      "label": "2222",
      "command": "sleep 2 && echo 222222",
      "dependsOn": "3333"
    },
    {
      "type": "shell",
      "label": "3333",
      "command": "sleep 2 && echo 33333"
    }
  ]

Review checklist

Reminder for reviewers

Signed-off-by: Roman Nikitenko <rnikiten@redhat.com>
@RomanNikitenko
Copy link
Contributor Author

@elaihau
could you take a look, is it important to keep the current order of search or we can change it
thanks in advance!

Copy link
Contributor

@elaihau elaihau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I smoke tested

  • configured tasks
  • detected tasks
  • customized detected tasks
  • task dependencies
    and they worked properly.

Thank you Roman !

@RomanNikitenko RomanNikitenko added the tasks issues related to the task system label Apr 30, 2020
@akosyakov
Copy link
Member

I think changes looks good. Not sure about the motivation. Is it performance optimisation? Usually we test them by looking at numbers before and after to understand whether refactoring is worth it.

@RomanNikitenko
Copy link
Contributor Author

RomanNikitenko commented Apr 30, 2020

Not sure about the motivation. Is it performance optimisation?

yes, the PR allows to:

  • avoid execution compound task related logic for ordinary tasks. For example, why we first get all tasks for a workspace and on the next step check if it's really required.
  • skip extra steps at running configured task (get all providers, fetch and cache all provided tasks)

Copy link
Member

@akosyakov akosyakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have not measured anything, but changes sound reasonable.

Signed-off-by: Roman Nikitenko <rnikiten@redhat.com>
@RomanNikitenko
Copy link
Contributor Author

RomanNikitenko commented May 1, 2020

Not sure about the motivation. Is it performance optimisation? Usually we test them by looking at numbers before and after to understand whether refactoring is worth it.

@akosyakov
I did some measure using console.time()-console.timeEnd() for simple (without dependsOn) configured shell and detected tsc tasks. I used repo from How to test section of the PR #7676 to have a lot of tasks.

I had 3 points for master branch:

Please see details:
measure_way

I had only 2 points for my branch because get workspace tasks part is not involved in running simple (without dependsOn) task process after applying my changes.

@RomanNikitenko
Copy link
Contributor Author

RomanNikitenko commented May 1, 2020

Results, gifs
You can skip this comment and go to the next one with screenshots

For master branch
Detected task:
master_detected2

Configured task:
master_configured

For my branch
Detected task:
my_changes_detected

Configured task:
my_changes_configured

@RomanNikitenko
Copy link
Contributor Author

RomanNikitenko commented May 1, 2020

The difference is:

Detected task, master
master_detected

Detected task, my branch
my_changes_detected

Configured, master
master_configured

Configured, my branch
my_changes_configured

@RomanNikitenko
Copy link
Contributor Author

RomanNikitenko commented May 1, 2020

So, you can see that:

For a configured task get config step reduced from ~300 ms to 0.035 ms because my changes allows to skip extra steps at running configured task (get all providers, fetch and cache all provided tasks) and just take the corresponding config from map.

For a detected task get config step after my changes we have the same time ~ 60 ms - so after my changes we don't have some regression for getting a configuration of detected tasks

For both configured and detected simple task time between starting point and this point was reduced from 700-1000 ms to 50-60 ms because my changes allows to avoid execution compound task related logic at running a simple task

According to the screenshots above you can see that we have huge time for getting workspace tasks - about 600-800 ms. So we should consider if we can reduce this time and improve our code.

p.s. I think that measurement depends on number of detected and configured tasks in a workspace, but it should reflect our problems in general.

@akosyakov @elaihau
wdyt?
is it good way to detect problem places?
should we have an issue to investigate get workspace tasks delay as possible place for improvement?
please let me know if you can see something wrong in my measurement

@RomanNikitenko
Copy link
Contributor Author

I need to have these changes merged to continue investigate and detect delay at running tasks, so I'm merging this PR.

But we can continue to discuss here the result of the measurement.

@RomanNikitenko RomanNikitenko merged commit dcb9c9c into master May 1, 2020
@RomanNikitenko RomanNikitenko deleted the tasksRefactoring branch May 1, 2020 18:13
@tsmaeder
Copy link
Contributor

tsmaeder commented May 4, 2020

@RomanNikitenko if I understand the reasoning correctly, the idea is that if we have a detected task, we will look for a customization for that task anyway, so it doesn't matter whether we find the customized task or the contributed task...it will result in the same task being executed, right?

@RomanNikitenko
Copy link
Contributor Author

@tsmaeder

There were two improvements:

  • separation compound task related logic
  • prevention extra steps at running a configured task

First one has influence on all simple tasks: configured, detected and customized.
Second one reduces search time for a configured task because:

skip extra steps at running configured task (get all providers, fetch and cache all provided tasks)

Regarding

the idea is that if we have a detected task, we will look for a customization for that task anyway, so it doesn't matter whether we find the customized task or the contributed task...it will result in the same task being executed, right?

Sorry, I don't have a ready answer...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tasks issues related to the task system
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants