![](\"images/ipsa_logo.png\"){kind=logo}
\n",
+ "\n",
+ "\n",
+ "> Version: **1.0**\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "# 1 . Implementation of a Neural Network\n",
+ "\n",
+ "In this exercise you will learn how to implement from scratch a deep neural network.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Set-up\n",
+ "\n",
+ "Firstly you will import all the packages used through the notebook. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "c:\\users\\efabr\\miniconda3\\lib\\site-packages\\numpy\\_distributor_init.py:32: UserWarning: loaded more than 1 DLL from .libs:\n",
+ "c:\\users\\efabr\\miniconda3\\lib\\site-packages\\numpy\\.libs\\libopenblas.QVLO2T66WEPI7JZ63PS3HMOHFEY472BC.gfortran-win_amd64.dll\n",
+ "c:\\users\\efabr\\miniconda3\\lib\\site-packages\\numpy\\.libs\\libopenblas.WCDJNK7YVMPZQ2ME2ZZHJJRJ3JIKNDB7.gfortran-win_amd64.dll\n",
+ " stacklevel=1)\n"
+ ]
+ }
+ ],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import h5py\n",
+ "\n",
+ "%matplotlib inline\n",
+ "\n",
+ "%load_ext autoreload\n",
+ "%autoreload 2\n",
+ "\n",
+ "np.random.seed(3)\n",
+ "\n",
+ "from utils import *"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Initialization\n",
+ "\n",
+ "Start by defining a function that allows to initialize the parameters of a deep neural network where the dimensions. The number of units in the different layers are passed as argument with `layer_dims`.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 42,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def initialization(layer_dims):\n",
+ "\n",
+ " np.random.seed(5)\n",
+ " parameters = {}\n",
+ " L = len(layer_dims) \n",
+ " \n",
+ " for l in range(1, L):\n",
+ " parameters['W' + str(l)] = np.random.randn(layer_dims[l],layer_dims[l-1])*0.01\n",
+ " parameters['b' + str(l)] = np.zeros((1,layer_dims[l]))\n",
+ " \n",
+ " return parameters"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 44,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "W1 = [[ 0.00441227 -0.0033087 0.02430771 -0.00252092 0.0010961 ]\n",
+ " [ 0.01582481 -0.00909232 -0.00591637 0.00187603 -0.0032987 ]\n",
+ " [-0.01192765 -0.00204877 -0.00358829 0.00603472 -0.01664789]\n",
+ " [-0.00700179 0.01151391 0.01857331 -0.0151118 0.00644848]]\n",
+ "b1 = [[0. 0. 0. 0.]]\n",
+ "W2 = [[-9.80607885e-03 -8.56853155e-03 -8.71879183e-03 -4.22507929e-03]\n",
+ " [ 9.96439827e-03 7.12421271e-03 5.91442432e-04 -3.63310878e-03]\n",
+ " [ 3.28884293e-05 -1.05930442e-03 7.93053319e-03 -6.31571630e-03]]\n",
+ "b2 = [[0. 0. 0.]]\n"
+ ]
+ }
+ ],
+ "source": [
+ "parameters = initialization([5,4,3])\n",
+ "print(\"W1 = \" + str(parameters[\"W1\"]))\n",
+ "print(\"b1 = \" + str(parameters[\"b1\"]))\n",
+ "print(\"W2 = \" + str(parameters[\"W2\"]))\n",
+ "print(\"b2 = \" + str(parameters[\"b2\"]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Forward propagation\n",
+ "\n",
+ "The forward propagation has been split in different steps. Firstly, the linear forward module computes the following equations:\n",
+ "\n",
+ "$$Z^{[l]} = W^{[l]}A^{[l-1]} +b^{[l]}\\tag{4}$$\n",
+ "\n",
+ "where $A^{[0]} = X$. \n",
+ "\n",
+ "Define a function to compute $Z^{[l]}$"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 45,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def linear_forward(A, W, b):\n",
+ " Z = W@A + b\n",
+ " cache = (A, W, b)\n",
+ " \n",
+ " return Z, cache"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 46,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Z = [[-0.67356113 0.67062057]]\n"
+ ]
+ }
+ ],
+ "source": [
+ "A, W, b = linear_forward_test()\n",
+ "\n",
+ "Z, linear_cache = linear_forward(A, W, b)\n",
+ "print(\"Z = \" + str(Z))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**Expected output**:\n",
+ "\n",
+ "| **Z** | \n", + "[[ -0.67356113 0.67062057]] | \n", + "
| **cost** | \n", + "0.2797765635793422 | \n", + "
| dW1 | \n", + "[[ 0.41010002 0.07807203 0.13798444 0.10502167]\n", + " [ 0. 0. 0. 0. ]\n", + " [ 0.05283652 0.01005865 0.01777766 0.0135308 ]] | \n", + "
| db1 | \n", + "[[-0.22007063]\n", + " [ 0. ]\n", + " [-0.02835349]] | \n", + "
| dA1 | \n", + "[[ 0.12913162 -0.44014127]\n", + " [-0.14175655 0.48317296]\n", + " [ 0.01663708 -0.05670698]] | \n", + "\n", + "
| W1 | \n", + "[[-0.59562069 -0.09991781 -2.14584584 1.82662008]\n", + " [-1.76569676 -0.80627147 0.51115557 -1.18258802]\n", + " [-1.0535704 -0.86128581 0.68284052 2.20374577]] | \n", + "
| b1 | \n", + "[[-0.04659241]\n", + " [-1.28888275]\n", + " [ 0.53405496]] | \n", + "
| W2 | \n", + "[[-0.55569196 0.0354055 1.32964895]] | \n", + "
| b2 | \n", + "[[-0.84610769]] | \n", + "