2022-06-28-andreis22a.md

title	booktitle	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Set Based Stochastic Subsampling	Proceedings of the 39th International Conference on Machine Learning	Deep models are designed to operate on huge volumes of high dimensional data such as images. In order to reduce the volume of data these models must process, we propose a set-based two-stage end-to-end neural subsampling model that is jointly optimized with an <em>arbitrary</em> downstream task network (e.g. classifier). In the first stage, we efficiently subsample <em>candidate elements</em> using conditionally independent Bernoulli random variables by capturing coarse grained global information using set encoding functions, followed by conditionally dependent autoregressive subsampling of the candidate elements using Categorical random variables by modeling pair-wise interactions using set attention networks in the second stage. We apply our method to feature and instance selection and show that it outperforms the relevant baselines under low subsampling rates on a variety of tasks including image classification, image reconstruction, function reconstruction and few-shot classification. Additionally, for nonparametric models such as Neural Processes that require to leverage the whole training data at inference time, we show that our method enhances the scalability of these models.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	andreis22a	0	Set Based Stochastic Subsampling	619	638	619-638	619	false	Andreis, Bruno and Lee, Seanie and Nguyen, A. Tuan and Lee, Juho and Yang, Eunho and Hwang, Sung Ju	given family Bruno Andreis given family Seanie Lee given family A. Tuan Nguyen given family Juho Lee given family Eunho Yang given family Sung Ju Hwang	2022-06-28		Proceedings of the 39th International Conference on Machine Learning	162	inproceedings	date-parts 2022 6 28	https://proceedings.mlr.press/v162/andreis22a/andreis22a.pdf