Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support intermediate value plots for constrained optimization #676

Conversation

nabenabe0928
Copy link
Collaborator

@nabenabe0928 nabenabe0928 commented Oct 31, 2023

Contributor License Agreement

This repository (optuna-dashboard) and Goptuna share common code.
This pull request may therefore be ported to Goptuna.
Make sure that you understand the consequences concerning licenses and check the box below if you accept the term before creating this pull request.

  • I agree this patch may be ported to Goptuna by other Goptuna contributors.

Reference Issues/PRs

This PR intends to support PR#4851 in Optuna.
PR#4851 supported the gray coloring for infeasible solutions in the intermediate value plot.
Hence, I introduced the same feature to OptunaDashBoard.

Here is an example.

  1. Run the following code:
import optuna
import matplotlib.pyplot as plt
from optuna.trial import Trial, FrozenTrial
from optuna.study import create_study
from optuna.visualization import plot_intermediate_values
from typing import Sequence

def objective_with_constraints(trial: Trial) -> float:
        trial.set_user_attr("constraint", [trial.number % 2])

        trial.report(trial.number, step=0)
        trial.report(trial.number + 1, step=1)
        return 0.0

def constraints(trial: FrozenTrial) -> Sequence[float]:
    return trial.user_attrs["constraint"]

study = create_study(sampler=optuna.samplers.NSGAIIISampler(constraints_func=constraints), storage="sqlite:///test.db")
study.optimize(objective_with_constraints, n_trials=10)
  1. Then check the visualization:
$ npm run build:dev
$ optuna-dashboard sqlite:///$HOME/test.db
  1. The appearance of the dashboard:

image

What does this implement/fix? Explain your changes.

  1. Color the infeasible trials with #CCCCCC,
  2. Make the infeasible lines dashed (for color blind), and
  3. Add the label saying (infeasible) if the trials are infeasible and complete.

@codecov
Copy link

codecov bot commented Oct 31, 2023

Codecov Report

Merging #676 (2403246) into main (29a5833) will not change coverage.
Report is 5 commits behind head on main.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #676   +/-   ##
=======================================
  Coverage   62.88%   62.88%           
=======================================
  Files          35       35           
  Lines        2250     2250           
=======================================
  Hits         1415     1415           
  Misses        835      835           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@not522
Copy link
Member

not522 commented Nov 1, 2023

@HideakiImamura Could you review this PR?

Copy link
Member

@HideakiImamura HideakiImamura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. I have a minor comment. PTAL.

optuna_dashboard/ts/components/GraphIntermediateValues.tsx Outdated Show resolved Hide resolved
? `trial #${trial.number}`
: `trial #${trial.number} (running)`,
name,
...(!isFeasible && { line: { color: "#CCCCCC", dash: "dash" } }),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a comment. I think that making dashed line of infeasible trials is very good.

@HideakiImamura
Copy link
Member

I confirmed that the change of this PR produces the correct plot as follows. I used this code for verification.

Screenshot 2023-11-02 at 12 09 57

@nabenabe0928 nabenabe0928 force-pushed the feat/add-intermediate-plot-for-constrained-opt branch from 71fa843 to 9315b61 Compare November 2, 2023 04:02
@nabenabe0928 nabenabe0928 force-pushed the feat/add-intermediate-plot-for-constrained-opt branch from 9315b61 to 634f959 Compare November 2, 2023 04:02
Copy link
Member

@HideakiImamura HideakiImamura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@HideakiImamura HideakiImamura removed their assignment Nov 2, 2023
@keisuke-umezawa keisuke-umezawa merged commit 7a6b1af into optuna:main Nov 5, 2023
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants