awesome-gaussian-regression

List of Gaussian regression papers and surveys

• Swiler, L., Gulian, M., Frankel, A., Safta, C., & Jakeman, J. (2020). A Survey of Constrained Gaussian Process Regression: Approaches and Implementation Challenges. arXiv preprint arXiv:2006.09319. https://arxiv.org/pdf/2006.09319
• Liu, K., Hu, X., Wei, Z., Li, Y., & Jiang, Y. (2019). Modified Gaussian process regression models for cyclic capacity prediction of lithium-ion batteries. IEEE Transactions on Transportation Electrification, 5(4), 1225-1236. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8853281
• Chen, Z., & Wang, B. (2018). How priors of initial hyperparameters affect Gaussian process regression models. Neurocomputing, 275, 1702-1710. https://www.sciencedirect.com/science/article/pii/S092523121731679X
  • The choice of prior distribution may play a vital role in the predictability of a GP model in a multiple starting point scenario. They consider different types of priors for the initial values of hyperparameters for some commonly used kernels. The important result is different priors for the initial hyperparameters have no significant impact on the performance of GPR prediction, once a kernel is chosen, despite that the estimates of the hyperparameters are very different to the true values in some cases.
• Kamath, A., Vargas-Hernández, R. A., Krems, R. V., Carrington Jr, T., & Manzhos, S. (2018). Neural networks vs Gaussian process regression for representing potential energy surfaces: A comparative study of fit quality and vibrational spectrum accuracy. The Journal of chemical physics, 148(24), 241702. https://aip.scitation.org/doi/full/10.1063/1.5003074
  • Show that the GP fitting error is lower, and the GP spectrum is more accurate. Also they find that the GP surface produces a relatively accurate spectrum when obtained based on as few as 313 points.
• Guo, M., & Hesthaven, J. S. (2018). Reduced order modeling for nonlinear structural analysis using gaussian process regression. Computer methods in applied mechanics and engineering, 341, 807-826. https://www.sciencedirect.com/science/article/pii/S0045782518303487
• Bu, Y., & Pan, J. (2015). Stellar atmospheric parameter estimation using Gaussian process regression. Monthly Notices of the Royal Astronomical Society, 447(1), 256-265. https://academic.oup.com/mnras/article/447/1/256/2907952
• Huang, W., Zhao, D., Sun, F., Liu, H., & Chang, E. (2015, June). Scalable gaussian process regression using deep neural networks. In Twenty-fourth international joint conference on artificial intelligence. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1032.9889&rep=rep1&type=pdf
• Camps-Valls, G., Verrelst, J., Munoz-Mari, J., Laparra, V., Mateo-Jiménez, F., & Gómez-Dans, J. (2016). A survey on Gaussian processes for earth-observation data analysis: A comprehensive investigation. IEEE Geoscience and Remote Sensing Magazine, 4(2), 58-78. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7487896
• Sun, A. Y., Wang, D., & Xu, X. (2014). Monthly streamflow forecasting using Gaussian process regression. Journal of Hydrology, 511, 72-81. https://www.sciencedirect.com/science/article/pii/S0022169414000298
• Vanhatalo, J., Riihimäki, J., Hartikainen, J., Jylänki, P., Tolvanen, V., & Vehtari, A. (2013). GPstuff: Bayesian modeling with Gaussian processes. Journal of Machine Learning Research, 14(Apr), 1175-1179. https://www.jmlr.org/papers/volume14/vanhatalo13a/vanhatalo13a.pdf
• Sarkka, S., Solin, A., & Hartikainen, J. (2013). Spatiotemporal learning via infinite-dimensional Bayesian filtering and smoothing: A look at Gaussian process regression through Kalman filtering. IEEE Signal Processing Magazine, 30(4), 51-61. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6530736
• Wilson, A. G., Knowles, D. A., & Ghahramani, Z. (2011). Gaussian process regression networks. arXiv preprint arXiv:1110.4411. https://arxiv.org/pdf/1110.4411
• Ranganathan, A., Yang, M. H., & Ho, J. (2010). Online sparse Gaussian process regression and its applications. IEEE Transactions on Image Processing, 20(2), 391-404.
• Hartikainen, J., & Särkkä, S. (2010, August). Kalman filtering and smoothing solutions to temporal Gaussian process regression models. In 2010 IEEE international workshop on machine learning for signal processing (pp. 379-384). IEEE.
• Chen, T., & Ren, J. (2009). Bagging for Gaussian process regression. Neurocomputing, 72(7-9), 1605-1610.
• Songthip T Ounpraseuth (2008) Gaussian Processes for Machine Learning, Journal of the American Statistical Association, 103:481, 429-429, DOI: 10.1198/jasa.2008.s219. https://doi.org/10.1198/jasa.2008.s219
• Yu, K., Tresp, V., & Schwaighofer, A. (2005, August). Learning Gaussian processes from multiple tasks. In Proceedings of the 22nd international conference on Machine learning (pp. 1012-1019). https://dl.acm.org/doi/pdf/10.1145/1102351.1102479
• Engel, Y., Mannor, S., & Meir, R. (2005, August). Reinforcement learning with Gaussian processes. In Proceedings of the 22nd international conference on Machine learning (pp. 201-208). https://dl.acm.org/doi/pdf/10.1145/1102351.1102377
• Rasmussen, C. E. (2003, February). Gaussian processes in machine learning. In Summer School on Machine Learning (pp. 63-71). Springer, Berlin, Heidelberg.
• Paciorek, C., & Schervish, M. (2003). Nonstationary covariance functions for Gaussian process regression. Advances in neural information processing systems, 16, 273-280.
• Seeger, M. (2004). Gaussian processes for machine learning. International journal of neural systems, 14(02), 69-106.
• MacKay, D. J. (1997). Gaussian processes-a replacement for supervised neural networks?.
• Williams, C. K., & Rasmussen, C. E. (1996). Gaussian processes for regression. In Advances in neural information processing systems (pp. 514-520). https://proceedings.neurips.cc/paper/1995/file/7cce53cf90577442771720a370c3c723-Paper.pdf

Books

• Gramacy, R. B. (2020). Surrogates: Gaussian Process Modeling, Design, and Optimization for the Applied Sciences. CRC Press.
• Kocijan, J. (2016). Modelling and control of dynamic systems using Gaussian process models. Springer International Publishing.
  • The main contribution of this book (for me) is in Chapter 4, which describes control methods based on GP models that were already published in the literature.
• Lifshits, M. (2012). Lectures on Gaussian processes. In Lectures on Gaussian Processes (pp. 1-117). Springer, Berlin, Heidelberg.
  • The book can be grouped into two blocks. The first chapters show essentials of the classical theory of Gaussian processes, while the later chapters present important issues, such as small deviations, expansions, and quantization of processes.
• Ibragimov, I. A., & Rozanov, Y. A. E. (2012). Gaussian random processes. Springer Science & Business Media.
• Shi, J. Q., & Choi, T. (2011). Gaussian process regression analysis for functional data. CRC Press.
• Ounpraseuth, S. T. (2008). Gaussian processes for machine learning.
• Dym, H., & McKean, H. P. (2008). Gaussian processes, function theory, and the inverse spectral problem. Courier Corporation.
• Williams, C. K., & Rasmussen, C. E. (2005). Gaussian processes for machine learning. Cambridge, MA: MIT press.
  • My personal favorite book on the topic.

Videos

Name	Lecturer	Year	Comments
A Primer on Gaussian Processes for Regression Analysis @ PyData	Chris Fonnesbeck	2019	A tutorial on how to use PyMC3 with good examples at PyData NYC 2019
Gaussian Process for Time Series Analysis	Juan Orduz	2019	An introduction of GP at PyData 2019 in Berlin. Also show good references during the lecture.
Machine Learning Lecture 26 "Gaussian Processes" -Cornell CS4780 SP17	Kilian Weinberger	2018	Nice lecture
ML Tutorial: Gaussian Processes	Richard Turner	2016	Nicely teached.
Gaussian Process Regression - Machine Learning Class 10-701	Alex Smola	2015
TimeSeriesAnalysis Using Gaussian Processes in Python & the Search for Earth 2.0	Dan Foreman-Mackey	2014	Cool talk about planets, but the analysis comes only at the last third of the presentation
Introduction to Gaussian process regression	Nando de Freitas	2013	Well detailed introduction to GP.
Regression with Gaussian processes	Nando de Freitas	2013	Regression
Introduction to Gaussian processes	Mauricio Alvarez Lopez	2020
Introduction to Deep GPs	-	2020
Gaussian Processes and Non-Gaussian Likelihoods	Alan Saul	2020
Deep Gaussian processes	Neil Lawrence	2020
Constraining Gaussian Processes by Variational Fourier Features	Arno Solin	2019

Tools

• De G. Matthews, A. G., Van Der Wilk, M., Nickson, T., Fujii, K., Boukouvalas, A., León-Villagrá, P., ... & Hensman, J. (2017). GPflow: A Gaussian process library using TensorFlow. The Journal of Machine Learning Research, 18(1), 1299-1304.
  • Library in Github: https://github.com/GPflow/GPflow
• Vanhatalo, J., Riihimäki, J., Hartikainen, J., Jylänki, P., Tolvanen, V., & Vehtari, A. (2013). GPstuff: Bayesian modeling with Gaussian processes. Journal of Machine Learning Research, 14(Apr), 1175-1179. https://www.jmlr.org/papers/volume14/vanhatalo13a/vanhatalo13a.pdf
  • Library in Github: https://github.com/gpstuff-dev/gpstuff
• Rasmussen, C. E., & Nickisch, H. (2010). Gaussian processes for machine learning (GPML) toolbox. The Journal of Machine Learning Research, 11, 3011-3015.
  • Library for Matlab: http://www.gaussianprocess.org/gpml/code/matlab/doc/

In python

• GPyTorch
  • https://gpytorch.ai/
• PyMC3
  • https://docs.pymc.io/Gaussian_Processes.html
• Scikit-learn: https://scikit-learn.org/stable/modules/gaussian_process.html
• George:
  • Documentation: https://george.readthedocs.io/en/latest/
• GPy: https://sheffieldml.github.io/GPy/
  • Documentation: https://gpy.readthedocs.io/en/devel/
• Deep Gaussian Processes
  • https://github.com/SheffieldML/PyDeepGP
• Gaussian Processes for Orientation Preference Maps
  • https://github.com/mackelab/gp-maps-python
• Neumann, M., Huang, S., Marthaler, D. E., & Kersting, K. (2015). pygps: A python library for gaussian process regression and classification. The Journal of Machine Learning Research, 16(1), 2611-2616.
• de Wolff, T., Cuevas, A., & Tobar, F. (2020). MOGPTK: The Multi-Output Gaussian Process Toolkit. arXiv preprint arXiv:2002.03471.
• Seikel, M., Clarkson, C., & Smith, M. (2013). Gapp: Gaussian processes in python. ascl, ascl-1303.
• In tensorflow: tfp.distributions.GaussianProcessRegressionModel
  • https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/GaussianProcessRegressionModel