2022-06-28-biggs22a.md

title	booktitle	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Non-Vacuous Generalisation Bounds for Shallow Neural Networks	Proceedings of the 39th International Conference on Machine Learning	We focus on a specific class of shallow neural networks with a single hidden layer, namely those with $L_2$-normalised data and either a sigmoid-shaped Gaussian error function (“erf”) activation or a Gaussian Error Linear Unit (GELU) activation. For these networks, we derive new generalisation bounds through the PAC-Bayesian theory; unlike most existing such bounds they apply to neural networks with deterministic rather than randomised parameters. Our bounds are empirically non-vacuous when the network is trained with vanilla stochastic gradient descent on MNIST and Fashion-MNIST.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	biggs22a	0	Non-Vacuous Generalisation Bounds for Shallow Neural Networks	1963	1981	1963-1981	1963	false	Biggs, Felix and Guedj, Benjamin	given family Felix Biggs given family Benjamin Guedj	2022-06-28		Proceedings of the 39th International Conference on Machine Learning	162	inproceedings	date-parts 2022 6 28	https://proceedings.mlr.press/v162/biggs22a/biggs22a.pdf	label link Other Files https://media.icml.cc/Conferences/ICML2022/supplementary/biggs22a-supp.zip