notes_unsupervised_learning.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:80525a2acef065021d84cb49da2994b6cc1c7f5d8c558b42f78e1be853bc2c37"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#<span style=\"background-color:#66FF99)\">Unsupervised Learning</span>\n",
      "* For setting where only a set of x features measured on n observations\n",
      "* Not interested in prediction, only to discover interesting thing about the measurements\n",
      "* Often used for exploratory data analysis\n",
      "* No universally accepted mechanism for performing CV - no way to check our work\n",
      "\n",
      "###<span style=\"background-color:#FF99FF)\">Principal Components Analysis</span>\n",
      "* When faced with large set of correlated variables, principal components can summarize with a smaller representative features that explains most of the variability\n",
      "* Principal Component Analysis (PCA) - process which PC are computed and use of these components to understand the data\n",
      "\n",
      "\n",
      "* **What are Principal Components**\n",
      "    * PCA seeks a small no. of components which are interesting - explains most of the variability\n",
      "    * Each dimension is a linear combination of p features\n",
      "    * First principal component of a set of features is the normalized linear combination of features which has the largest variance\n",
      "    * Optimization math can be solved using eigen decomposition (Linear Algebra)\n",
      "    * When PCA is performed, varaibles should be centered to have mean zero\n",
      "    * Recommended: Scale the variables to have standard deviation one\n",
      "\n",
      "* **Another interpretation of Principal Components**\n",
      "    * Principal components provide low-dimensional linear surfaces closest to the observations\n",
      "    * Seek a single dimension of the data that lies as close as possible to all data points\n",
      "\n",
      "* **More on PCA**\n",
      "    * Scaling the Variables (very important concept)\n",
      "        * Recommend scaling the variables to have standard deviation one (substantial effect)\n",
      "    * Uniqueness of Principal Components\n",
      "    * Proportion of Variance Explained\n",
      "        * How much information is lost when we do PCA? How much of the variance is NOT contained in the first few principal components?\n",
      "        * Use Proportion of Variance Explained (PVE) - plot PC against cumulative PVE (scree plot)\n",
      "    * Deciding how many principal components to use\n",
      "        * Only use the first few principal components to visualize/interpret the data\n",
      "        * Use scree plot to decide no. of PC to use\n",
      "        * If first few PC are not interesting, adding other PCs will not help\n",
      "\n",
      "* **Other uses of Principal Components**\n",
      "    * Perform regression using principal components to score vectors as features\n",
      "    * Many techniques can be applied to use PC"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "###<span style=\"background-color:#FF99FF)\">Clustering Methods</span>\n",
      "* Broad set of techniques for finding subgroups or clusters in a dataset\n",
      "* PCA find a low-dimensional representation of the observations that explains a good amount of variance\n",
      "* Clustering find homogenous subgroups among the observations\n",
      "* Focus on K-means clustering (pre-specified no. of clusters) and hierarchical clustering (do not know how many clusters, tree-like visual representation)\n",
      "\n",
      "\n",
      "* **K-Means Clustering**\n",
      "    * Must specific no. of desired clusters\n",
      "    * Key idea: Good clustering is one which within-cluster variation is as small as possible\n",
      "    * Math: The within-cluster variation for the kth cluster is teh sum of all pairwise squared Euclidean distances between the observations divded by total no. of observations\n",
      "\n",
      "* **Hierarchical Clustering**\n",
      "    * Does not require we commit to a particular choice of K\n",
      "    * More attractive tree-based representation (dendrogram)\n",
      "    * Most common hierarchical clustering: bottom-up/agglomerative clustering\n",
      "    * The horizontal cuts will determine the number of clusters\n",
      "    * Attractive aspect of HC: 1 single dendrogram can be used to obtain any number of clusters\n",
      "    * Might yield worse results than k-Means (because they are not nested)\n",
      "    * Algorithm: Use a dissimilarity measure between each pair of observations (Euclidean distance). Most similar are fused together\n",
      "    * How do we define the dissimilarity? 4 types of linkage: complete, average, single, and centroid\n",
      "    * Choice of Dissimilarity Measure: Euclidean or Correlation-based\n",
      "    * e.g If Euclidean distance used, shoppers who bought very few items will be cluster together\n",
      "    * If correlation-based used, shoppers with similar preferences (shoppers who bought A and B) will be clustered together\n",
      "    * Choosing dissimilarity measure important! \n",
      "    * Need to consider if variables should be scaled to stand standard deviation one\n",
      "\n",
      "* **Practical Issues in Clustering**\n",
      "    * Should the observations or features be standardized in some way? e.g some variables should be centered to have mean zero and scaled to have standard deviation one\n",
      "    * For K-means: How many clusters should we look for?\n",
      "    * For Hierarchical clustering: What dissimilaity measure should be used? What type of linkage? Where should we cut the dendrogram to obtain clusters?\n",
      "    * Validate the Clusters - various methods but no consensus a single best approach\n",
      "    * Clusters may be heavily distorted due to presence of outliers that do not belong to any cluster\n",
      "    * Reccomend perform clustering with different choices of parameters\n",
      "    * Careful how results of clustering analysis are reported, should be not taken as absolute truth = just a good starting point for development of a hypothesis. Always use independent data set to test the results."
     ]
    }
   ],
   "metadata": {}
  }
 ]
}