Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Latest version of FLAML changes optimization behavior #839

Open
bpkroth opened this issue Aug 12, 2024 · 0 comments
Open

Latest version of FLAML changes optimization behavior #839

bpkroth opened this issue Aug 12, 2024 · 0 comments

Comments

@bpkroth
Copy link
Contributor

bpkroth commented Aug 12, 2024

Recently a new version (v2.2.0) of FLAML was released and it introduces some new warnings and behavior:

INFO     flaml.tune.searcher.blendsearch:blendsearch.py:169 No low-cost partial config given to the search algorithm. For cost-frugal search, consider providing low-cost values for cost-related hps via 'low_cost_partial_config'. More info can be found at https://microsoft.github.io/FLAML/docs/FAQ#about-low_cost_partial_config-in-tune
WARNING  flaml.tune.searcher.suggestion:suggestion.py:555 You passed a `space` parameter to OptunaSearch that contained unresolved search space definitions. OptunaSearch should however be instantiated with fully configured search spaces only. To use Ray Tune's automatic search space conversion, pass the space definition as part of the `config` argument to `tune.run()` instead.
WARNING  flaml.tune.searcher.suggestion:suggestion.py:555 You passed a `space` parameter to OptunaSearch that contained unresolved search space definitions. OptunaSearch should however be instantiated with fully configured search spaces only. To use Ray Tune's automatic search space conversion, pass the space definition as part of the `config` argument to `tune.run()` instead.
FAILED mlos_bench/mlos_bench/tests/optimizers/toy_optimization_loop_test.py::test_flaml_optimization_loop - assert 63.77786073141286 == 60.15 ± 6.0e-01
  
  comparison failed
  Obtained: 63.77786073141286
  Expected: 60.15 ± 6.0e-01

We should investigate and fix these.

For now, I'm creating a PR to simply pin the version to avoid them until we have time to do that.

bpkroth added a commit to bpkroth/MLOS that referenced this issue Aug 12, 2024
bpkroth added a commit that referenced this issue Aug 14, 2024
See Also: #839

Also fixes `make dist-test` rules to workaround recently broken `pip`
version interpretation when using symlinked whl files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant