Invalid parameters in study.best_params after hyperparameter optimization

[neonine2]

There are parameters that are sampled in the hyperparameter optimization process, but are either not actual hyperparameters or that the sampled value is not valid (see the two cases below). This makes it difficult to directly pass the study.best_params to a training session, as the user will need to manually modify these parameters which may cause inconsistency issues down the road when the master code changes.

Case 1:
the sampling function used for hyperparameter optimization assigns a particular string to represent an actual value, I'm assuming this is done to make suggest_categorical work.
For example in hyperparams_opt.py,

net_arch = trial.suggest_categorical("net_arch", ["small", "medium"])
net_arch = {
        "small": [dict(pi=[64, 64], vf=[64, 64])],
        "medium": [dict(pi=[256, 256], vf=[256, 256])],
    }[net_arch]

Here, "small" is an alias for that particular network architecture and shows up in study.best_params, but it cannot be passed as a hyperparameter value directly totrain.py since it is only an alias.

Case 2:
Similarly, there are sampled parameters that are not actually hyperparameters but used to modify hyperparameters.
For example again in hyperparams_opt.py,

subsample_steps = trial.suggest_categorical("subsample_steps", [1, 2, 4, 8])
gradient_steps = max(train_freq // subsample_steps, 1)

Here, subsample_steps will show up as a parameter in study.best_params, but it is not an actual hyperparameter, it is only used to modify gradient_steps.

It would be great if these issues can be resolved. Thank you.

[araffin]

Hello,
yes, i'm aware of this issue, a duplicate of #121

the solution would be a script for post processing the study and we would welcome a PR that solves this issue ;)