Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

Fix evolution tuner RuntimeError and IndexError #2695

Merged
merged 15 commits into from
Sep 18, 2020
Merged

Fix evolution tuner RuntimeError and IndexError #2695

merged 15 commits into from
Sep 18, 2020

Conversation

colorjam
Copy link
Contributor

@colorjam colorjam commented Jul 17, 2020

The current version of evolution tuner has several problems:

  1. trialConcurrency should be larger than population_size, otherwise, the experiment will raise RuntimeError('The population is empty').

    Add callback function

  2. Do not deal with trial failure. If the number of failed trails larger than population_size, the experiment will raise RuntimeError('The population is empty').

    Resume the failed trial.

  3. The end time of each trial may not be the same, so, the population length maybe 1. It can not make a comparison between two randomly selected individuals.

for item in self.space:
is_rand[item] = True

config = json2parameter(self.searchspace_json, is_rand, self.random_state)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to generate duplicate parameters?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the first generation is random generated.


def trial_end(self, parameter_id, success, **kwargs):
"""
To deal with trial failure. If a trial fails, readd the trial parameters into population.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"readd"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I modify the docstring for clarity, please review on the latest version, thanks~


total_config = config
if not self.population:
self.population.pop(1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if self.population only has one element?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks~ I have fixed it, please review on the latest version.

@QuanluZhang
Copy link
Contributor

@colorjam UT is failed might because trail_end is not handled in UT. @ultmaster FYI

@QuanluZhang QuanluZhang mentioned this pull request Sep 16, 2020
79 tasks
@QuanluZhang
Copy link
Contributor

@colorjam please double check whether the generated configs are the same as the previous implementation when there is no failed trial.

@colorjam
Copy link
Contributor Author

@QuanluZhang Thanks, I have checked it. I use the previous implementation to generate individuals.

@QuanluZhang QuanluZhang merged commit 171400e into microsoft:master Sep 18, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants