Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for GPFlow as a backend #104

Open
5 tasks
kiudee opened this issue Oct 21, 2021 · 0 comments
Open
5 tasks

Support for GPFlow as a backend #104

kiudee opened this issue Oct 21, 2021 · 0 comments
Labels
enhancement New feature or request Priority: Medium

Comments

@kiudee
Copy link
Owner

kiudee commented Oct 21, 2021

Currently, the library uses scikit-optimize/sklearn as a backend to do Gaussian process computations. These implementations are easy to use and served the library well so far.
A big problem is that the library is quite limited in functionality. One big use case of bayes-skopt is to handle very noisy target functions. In that regard it would be useful to be able to model heteroscedastic noise, which the Gaussian process in sklearn does not really support. It is possible to set the alpha parameter to a vector, which will incorporate the noise of the training data, but during prediction it is still noiseless.
This could be useful for acquisition functions which properly handle the observation noise, like noisy EI and knowledge gradient.

GPFlow offers a lot of the needed functionality out of the box. It is straightforward to construct heteroscedastic likelihoods. It also supports stochastic variational Gaussian processes, which allow GPs to scale to more than 10k observations.
Therefore, migrating to GPflow as a backend would be a good long-term goal. Sadly, it will require a major rewrite of the library, since many classes (Optimizer, acquisition functions etc.) are tightly coupled to the current GP implementation.

Roadmap

  • Get familiar with all relevant aspects of GPFlow.
  • Identify an interface for the GP component which would allow plugging in different backends.
  • Migrate the existing code to obey the new interface specification, reducing the coupling between the components.
  • Add the GPFlow GP implementation.
  • Add GPFlow specific features.
@kiudee kiudee added enhancement New feature or request Priority: Medium labels Oct 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Priority: Medium
Projects
None yet
Development

No branches or pull requests

1 participant