Neural Architecture Search

Tabel of contents

• 1. NAS introdction
• 2. NAS reviews
• 3. Papers

Introduction

The list includes the papers related to NAS I have read. All of them are grouped into three classes ( Search Space, Seach strategy and evaluation strategy) based on my subjectivity.

Papers

Reviews

Data	Title	Venue	Notes
2018	Neural Architecture Search: A Survey	JMLR	-
2008	Neuroevolution: From Architectures to Learning	Evolutionary Intelligence	-
1999	Evolving Artificial Neural Networks	IEEE	-

Search Space / Encoding strategies

Data	Title	Venue	Code	Notes
2018	Neural Architecture Search Over a Graph Search Space	-	-	This paper defined a search space on direct graph which is used to instruct the construction of networks.
2017	Genetic CNN	ICCV	Tensorflow	This paper proposed a binary string encoding strategy.
2017/18	Efficient Architecture Search by Network Transformation	AAAI	-	This paper Proposed a popular block based network definition method.
2009	A Hypercube-based Encoding for Evolving large-scale Neural Networks	IEEE Artificial Life	-	This method use a extended CPPNs to encode the ANN.
2006	Evolutionary Design of Neural Network Architectures Using a Descriptive Encoding Language	IEEE TEVC	-	This paper proposed a human-readable and writable encoding method.
2002	Evolving Neural Networks through Augmenting Topologies	EC	-	NEAT: An Awesome Approach to NeuroEvolution
1998	Network Generating Attribute grammar Encoding	IJCNN	-	Grammar Encoding
1996	A comparison between cellular encoding and direct encoding for genetic neural networks	-	-	This paper compared the celluar encoding and direct encoding methods
1996	Evolving Graphs and Networks with Edge encoding: preliminary report	The Genetic Programming Conference	-	Edge Encoding method.
1994	Neurogenetic learning: an integrated method of designing and training neural networks using genetic algorithms	Physica D	-	This paper inspired by L-System.
1994	Automatic definition of modular neural networks	Adaptive behavior	-	-
1990	Designing Neural Networks Using Genetic Algorithms with Graph Generation System	The Conference on Genetic Algorithms	-	This paper proposed a graph grammatical encoding and analysed the problems of the direct encoding method.

Search Strategies

Data	Title	Venue	Code	Notes
2018	Reinforced Evolutionary Neural Architecture Search	CVPR	mxnet	EA/RL
2018	Neural Architecture Optimization	NeurIPS	Pytorch Tensorflow	Gradient Based
2018	NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search	GECCO	-	This paper can be viewed as an extension of Genetic CNN in search strategy and slightly improving in encoding strategy.
2017	Large-Scale Evolution of Image Classifiers	ICML	-	Google's work. Trying to prove the possibility of evolving CNN automatically.
2017	SMASH: One-Shot Model Architecture Search through HyperNetworks	NeurIPS	-	Gradient Based
2017	Neural Architecture Search with Reinforcement Learning	ICLR 2017	-	RL
2017	Designing Neural Network Architectures using Reinforcement Learning	ICLR	-	RL

Evaluation Strategies

Data	Title	Venue	Code	Notes
2018	Progressive Neural Architecture Search	ECCV	Pytorch Tensorflow	This paper use a predictor to evaluate the acc of each network with few real training.

Extension

Data	Title	Venue	Code	Notes
2019	NAS-Bench-101: Towards Reproducible Neural Architecture Search	ICML	Code	A Benchmark of NAS.
2016	Net2Net: Accelerating Learning via Knowledge Transfer	ICLR 2016	-	This paper proposed a mehtod (similar to parameters sharing) to transfer the knowledge from previous network to a larger one.

Useful pages

1. LITERATURE ON NEURAL ARCHITECTURE SEARCH
2. Awesome NAS