theory exercises solved (1 to 11)

Educational/Hands-On-Machine-Learning/CH12 Custom Models and Training with TensorFlow/Exercises/theory.ipynb

@@ -142,7 +142,186 @@
   {
    "cell_type": "markdown",
    "metadata": {},
-   "source": []
+   "source": [
+    "4. Can you name six other data structures available in TensorFlow, beyond regular tensors?\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Sets represented as regular tensors, but tensorflow provides special function to manipulate it such as tf.sets\n",
+    "\n",
+    "Queues\n",
+    "\n",
+    "Strings used in nlp, also represented as regular tensors, but tensorflow provides special function to manipulate it such as tf.strings\n",
+    "\n",
+    "RaggedTensors which can bee seen as array of arrays, but all arrays (tensors) inside are variant in length\n",
+    "\n",
+    "SparseTensors Tensors those elements mostly zeros\n",
+    "\n",
+    "TensorArray is list of tensors but all has the same shape, and it can be fixed size or dynamic.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "5. You can define a custom loss function by writing a function or by subclassing thetf.keras.losses.Loss class. When would you use each option?\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "when we want to create a loss function, we just implement it as regular python function.\n",
+    "\n",
+    "if our function accepts hyperparameter or any other state, then we should subclass `tf.keras.losses.Loss` and implement `__init__`, `call()` functions, and to the hyperparameter saved along with the model, we implement `get_config()` also.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "6.  Similarly, you can define a custom metric in a function or as a subclass of tf.keras.metrics.Metric. When would you use each option?\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "similarly, we can define it in a function if it is so simple without hyperparameters or caching records needed (ex: recall). otherwise sub-classing would be must.\n",
+    "\n",
+    "Moreover, if computing the metric over a whole epoch is not equivalent to computing the mean metric over all batches in that epoch (e.g., as for the precision and recall metrics), then you should subclass the keras.metrics.Metric class and implement the `__init__()`, `update_state()`, and `result()` methods to keep track of a running metric during each epoch. You should also implement the `reset_states()` method unless all it needs to do is reset all variables to 0.0. If you want the state to be saved along with the model, then you should implement the `get_config()` method as well.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "7.  When should you create a custom layer versus a custom model?\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The Custom Model created when it is not sequential and has multi inputs, multi outputs.\n",
+    "\n",
+    "The Custom Layer created when we need another behavior of the built-in layers, and we put it in the custom or sequential model easily.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "8.  What are some use cases that require writing your own custom training loop?\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "usually we will not write our custom training loop. except in debugging or using different optimizer ot different parts of the network.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "9.  Can custom Keras components contain arbitrary Python code, or must they be convertible to TF functions?\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If you absolutely need to include arbitrary Python code in a custom component, you can either wrap it in a tf.`py_function()` operation (but this will reduce performance and limit your model's portability) or set `dynamic=True `when creating the custom layer or model (or set `run_eagerly=True` when calling the model's `compile()` method).\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "10. What are the main rules to respect if you want a function to be convertible to a TF function?\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "a function may not be convertible to tf function ( cannot used by @tf.function at graph execution ) for this reasons:\n",
+    "if the function contains python prints or it modifying dynamic python things like list, dictionaries. or use global variables\n",
+    "\n",
+    "unsupported operations by tf\n",
+    "\n",
+    "dynamic flow control by python like using loops or if.\n",
+    "\n",
+    "has type Inconsistencies, the graph expects all tensors to be the same type\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "11. When would you need to create a dynamic Keras model? How do you do that? Why not make all your models dynamic?\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Creating a dynamic Keras model can be useful for debugging, as it will not compile any custom component to a TF Function, and you can use any Python debugger to debug your code. It can also be useful if you want to include arbitrary Python code in your model (or in your training code), including calls to external libraries. To make a model dynamic, you must set dynamic=True when creating it. Alternatively, you can set run_eagerly=True when calling the model's compile() method. Making a model dynamic prevents Keras from using any of TensorFlow's graph features, so it will slow down training and inference, and you will not have the possibility to export the computation graph, which will limit your model's portability.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n"
+   ]
   }
  ],
  "metadata": {