Update mlos_core API to support with multi-factor optimization #730

motus · 2024-05-01T23:55:56Z

Pass multi-column DataFrame instead of Sequence to BaseOptimizer.register() and other methods that deal with scores
Update mlos_bench MlosCoreOptimizer to support the new mlos_core API
Update unit tests to work with the new API
Add unit tests for end-to-end multi-target optimization

Merge after ~~#726~~

… when loading the results

…lk_register()

…led trials into the optimizer

…eOptimizer

…to sergiym/opt/multiobjective

…tive_config

…se optimizer

…imizer

…/opt/mobj_core

another PR to reduce the size of #730

…/opt/mobj_core

motus · 2024-05-14T00:46:53Z

Multi-target optimization in FLAML and optional weights for optimization targets implemented in the next PR #738

mlos_bench/mlos_bench/optimizers/mlos_core_optimizer.py

bpkroth

Let's back out the change to submitting all non-objective scores as context for now. Thanks!

…separate PR

…into sergiym/opt/mobj_core

mlos_bench/mlos_bench/optimizers/mlos_core_optimizer.py

bpkroth

Minor comment request and I think so test failures, but LGTM

… trials

…t multi-target optimization for FLAML. (#738) Summary of changes: * Pass optional weights for optimization targets in mlos_core * Implement weighted average for multi-objective optimization in FLAML * Add more unit tests for multi-objective optimization on mlos_core side Merge after ~#730~ --------- Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>

…t multi-target optimization for FLAML. (microsoft#738) Summary of changes: * Pass optional weights for optimization targets in mlos_core * Implement weighted average for multi-objective optimization in FLAML * Add more unit tests for multi-objective optimization on mlos_core side Merge after ~microsoft#730~ --------- Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>

motus added 30 commits March 22, 2024 14:55

load ALL metrics from the DB when feeding the optimizer

2abf2c7

disable incorrect pylint warning

63a4e15

fix unit tests to handle the new data types returned from teh storage…

93d4161

… when loading the results

register multi-dimensional results in optimizer.bulk_register()

4adf433

update unit tests to validate multi-dimensional input to Optimizer.bu…

9c16ba6

…lk_register()

fix the key/value SELECT statement in SQL Storage implementaion

7874fec

fix the input to grid search optimizer .bulk_register call

6cb1fb6

minor fix in GridSearchOptimizer

e5adef5

remove unused import

a79e555

bugfix: take the optimization direction into account when loading fai…

b48c698

…led trials into the optimizer

handle -Inf in try_parse_val(). check it in unit tests

3c29bed

move try_parse_val fix to a separate PR; undo the sign fix in MlosCor…

5015f75

…eOptimizer

start updating the configs to include more than one optimization target

6b2804d

Merge branch 'main' into sergiym/opt/multiobjective

dc84cfa

Merge branch 'main' into sergiym/opt/multiobjective

e99c591

Merge branch 'sergiym/opt/multiobjective' of github.com:motus/MLOS in…

323fba8

…to sergiym/opt/multiobjective

add a comment re: Row._tuple() pylint warning

838ca37

Merge branch 'sergiym/opt/multiobjective' into sergiym/opt/multiobjec…

623f012

…tive_config

fix the optimization_target JSON schema; handle it properly in the ba…

6661ac0

…se optimizer

use NotImplementedError instead of NotImplemented

dd44568

add type annotations

f1e989a

fix the json schema with the new optimizer_type format

5f70f02

fix teh optimization_target schema

9c20101

typo fixed

7e3d1d2

use common optimization_target definition in all JSON schemas

70790ce

implement multi-obje4ctive optimization all the way up to MlosCoreOpt…

3c513a3

…imizer

fix all pylint/myoy warnings

d466aaf

minor pylint fix

c60ec07

make some unit tests work

39f7f22

fix opt_bulk_register_test tests to use multiple scores

3a89ade

motus added 2 commits May 13, 2024 13:59

Merge branch 'main' of https://github.com/microsoft/MLOS into sergiym…

35ac77a

…/opt/mobj_core

code review fixes

3d5d1a4

jsfreischuetz mentioned this pull request May 13, 2024

Fix docstrings for mlos_core SMAC and FLAML optimizers (#736) jsfreischuetz/MLOS#2

Closed

motus added 2 commits May 13, 2024 14:13

Merge branch 'main' of https://github.com/microsoft/MLOS into sergiym…

51acda1

…/opt/mobj_core

add details to get_best_observations() docstring

65b4007

motus mentioned this pull request May 13, 2024

make optimizer a fixture for 1-hot encoding/decoding tests #737

Merged

minimize diff with master

a6c5a9d

motus added a commit that referenced this pull request May 13, 2024

make optimizer a fixture for 1-hot encoding/decoding tests (#737)

e1a0cdd

another PR to reduce the size of #730

motus added 3 commits May 13, 2024 14:59

Merge branch 'main' of https://github.com/microsoft/MLOS into sergiym…

36370d5

…/opt/mobj_core

use multi_objective_algorithm for SMAC

4c48459

remove unused import

571c232

motus mentioned this pull request May 14, 2024

Pass optional weights for optimization targets in mlos_core; implement multi-target optimization for FLAML. #738

Merged

motus changed the title ~~Update mlos_core API to support with multi-factor optimization and contexts~~ Update mlos_core API to support with multi-factor optimization May 15, 2024

Merge branch 'main' into sergiym/opt/mobj_core

79039ac

bpkroth reviewed May 20, 2024

View reviewed changes

mlos_bench/mlos_bench/optimizers/mlos_core_optimizer.py Outdated Show resolved Hide resolved

bpkroth reviewed May 20, 2024

View reviewed changes

mlos_bench/mlos_bench/optimizers/mlos_core_optimizer.py Outdated Show resolved Hide resolved

bpkroth requested changes May 20, 2024

View reviewed changes

motus added 3 commits May 20, 2024 13:53

rollback the optimization context change: will do it explicitly in a …

d25e4ee

…separate PR

Merge branch 'sergiym/opt/mobj_core' of https://github.com/motus/MLOS …

438b9af

…into sergiym/opt/mobj_core

do not pass optimization context (yet) in bulk_register

a0dd714

motus enabled auto-merge (squash) May 20, 2024 21:01

bpkroth reviewed May 20, 2024

View reviewed changes

mlos_bench/mlos_bench/optimizers/mlos_core_optimizer.py Show resolved Hide resolved

bpkroth approved these changes May 20, 2024

View reviewed changes

add a comment and link to open issue about the values used for failed…

74495fc

… trials

motus merged commit e2b819f into microsoft:main May 20, 2024
9 of 12 checks passed

motus deleted the sergiym/opt/mobj_core branch May 24, 2024 18:34

jsfreischuetz mentioned this pull request Jun 3, 2024

Pass optional weights for optimization targets in mlos_core; implemen… jsfreischuetz/MLOS#6

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update mlos_core API to support with multi-factor optimization #730

Update mlos_core API to support with multi-factor optimization #730

motus commented May 1, 2024 •

edited

Loading

motus commented May 14, 2024

bpkroth left a comment

bpkroth left a comment

Update mlos_core API to support with multi-factor optimization #730

Update mlos_core API to support with multi-factor optimization #730

Conversation

motus commented May 1, 2024 • edited Loading

motus commented May 14, 2024

bpkroth left a comment

Choose a reason for hiding this comment

bpkroth left a comment

Choose a reason for hiding this comment

motus commented May 1, 2024 •

edited

Loading