Add Optuna utilities to generate and run studies on single problem instance and multiple problem instances #227
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- add a "high" value for each hyperparameter (can still be tuned in a specific study) - when their are used as subbrick as only class, use subbrick_cls to avoid having a useless subbrick hyperparameter always set to ParamsConstraintBuilder
We rewrite the rcpsp optuna example on multiple instances and the coloring optuna example with timed pruning by making use of these new generic functions.
The dummy class will raise an error upon instantiation. It allows the import of any module importing TimedPercentilePruner without error but monly a warning.
nhuet
changed the title
- check_satisfy: wether or not checking that the solution does actually satisfy the problem - prop_startup_instances: proportion of instances used to startup before allowing pruning. - randomize_instances: whether randomizing instances order when running a trial. - report_cumulated_fitness: whether reporting cumulated fitness instead of individual fitness for each problem instance.
…tisfying solutions
It prevents meddling with best trial when pruning: if we are maximizing negative fitnesses, the cumulative fitness is decreasing with number of problem instances and thus should not be returned as the trial value when pruned (since we are maximizing and this will always be greater than if we had not pruned).
nhuet
force-pushed
the
optuna-multiple-instances
branch
from
418f51b to
89b8311
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We provide generic functions to help create and run optuna studies
generic_optuna_experiment_monoproblem(): this takes into account the latest examples added to examples/, and allow the possibility to report/prune according to elapsed time (meaningful when dealing with several unrelated solvers) or internal optimization step numbergeneric_optuna_experiment_multiproblem(): as the example provided in optuna doc for WilcoxonPruner, we report intermediate values and potentially prune after each instance resolutionThese generic functions set default values for study name, journal log, ... and one should write its own script in order to have more control on all details.