'Keras-like' model creation

[wagenaartje]

Recently, I brought up an idea in this comment: #12 (comment). I think the way custom networks can be created still requires some knowledge about network architectures, which may be too complicated for some. I want to revamp model creation, making it more layered like synaptic, but with extra flexibility.

My current idea of network creation:

// Create a network
var model = new Model();
model.add(Dense(10)); // 10 inputs
model.add(LSTM(5, { activation: Activation.RELU }));
model.add(NARX(10));
model.add(Dense(3, { activation: Activation.TANH})); // 3 outputs

// To be discussed, adding custom (recurrent) connections
model.layers[0].connect(model.layers[2]);
model.layers[3].connect(model.layers[1]);

var network = model.compile();
network.train(...

Some basic points:

• A model should always start and end with a Dense() layer.
• Optional arguments can be specified in a dictionary, e.g. activation, bias, weight initialisation, connection type to next layer (e.g. ALL_TO_ALL, ONE_TO_ONE)
• After setting up the layers, the network must be compiled. This turns the network in array of nodes, make it more susceptible to genetic algorithms and optimization
• The training function will remain the same as now

Things to discuss:

• What is a good way to allow custom connections? Check out the code above about what my first idea is on how to do this

Layer types I want to embed:

• LSTM
• NARX
• Dense
• Random
• GRU
• Clock
• Softmax
• Convolution
• Pooling

[wagenaartje]

I've got a problem with two LSTM layers after each other. Feel free to help: https://stats.stackexchange.com/questions/279529/which-gate-gates-an-inter-lstm-connection