neuralnet.md

This is a C code for a simple neural network with the ability to learn XOR. The network has a sigmoid activation function, stochastic gradient descent, and a mean square error function. The code consists of several functions to create, train and use the neural network.

Here's a step-by-step breakdown of what the code does:

• The code includes four libraries: stdio.h, stdlib.h, math.h, and time.h.
• The code defines the sigmoid activation function and its derivative using the following functions:

double sigmoid(double x) { return 1 / (1 + exp(-x)); }
double dSigmoid(double x) { return x * (1 - x); }

• The code defines a function to initialize random weights for the neurons:

double init_weight() { return ((double)rand())/((double)RAND_MAX); }

• The code defines a function to shuffle the training data:

void shuffle(int *array, size_t n)
{
    if (n > 1)
    {
        size_t i;
        for (i = 0; i < n - 1; i++)
        {
            size_t j = i + rand() / (RAND_MAX / (n - i) + 1);
            int t = array[j];
            array[j] = array[i];
            array[i] = t;
        }
    }
}

• The code defines some constants and arrays for the neural network, such as the number of inputs, hidden nodes, outputs, training sets, and learning rate, as well as arrays for the hidden layer, output layer, hidden layer biases, output layer biases, hidden weights, and output weights:

#define numInputs 2
#define numHiddenNodes 2
#define numOutputs 1
#define numTrainingSets 4

const double lr = 0.1f;

double hiddenLayer[numHiddenNodes];
double outputLayer[numOutputs];

double hiddenLayerBias[numHiddenNodes];
double outputLayerBias[numOutputs];

double hiddenWeights[numInputs][numHiddenNodes];
double outputWeights[numHiddenNodes][numOutputs];

double training_inputs[numTrainingSets][numInputs] = {{0.0f,0.0f},
                                                      {1.0f,0.0f},
                                                      {0.0f,1.0f},
                                                      {1.0f,1.0f}};
double training_outputs[numTrainingSets][numOutputs] = {{0.0f},
                                                        {1.0f},
                                                        {1.0f},
                                                        {0.0f}};

• The code initializes the weights and biases for the neural network:

for (int i=0; i<numInputs; i++) {
    for (int j=0; j<numHiddenNodes; j++) {
        hiddenWeights[i][j] = init_weight();
    }
}
for (int i=0; i<numHiddenNodes; i++) {
    hiddenLayerBias[i] = init_weight();
    for (int j=0; j<numOutputs; j++) {
        outputWeights[i][j] = init_weight();
    }
}
for (int i=0; i<numOutputs; i++) {
    outputLayerBias[i] = init_weight();
}

• The code creates an array of integers to shuffle the order of the training set:

int trainingSetOrder[] = {0,1,2,3};

• The code trains the neural network using stochastic gradient descent:

for(int epochs=0; epochs < numberOfEpochs; epochs++) {
    // As per SGD, shuffle the order of the training set
    shuffle(trainingSetOrder,numTrainingSets);