- PCA helps in identifying and finding patterns to reduce the dimensions of the dataset with minimal loss of information.
- Sites to understand PCA:
- Ways to compute eigen values, eigen vectors and its use in PCA - Eigen Value Decomposition
- If the data matrix is centered to have zero mean then EVD and the SVD are exactly the same. Using eigen value decomposition is a better way, if you cannot verify the mean is tending to zero
- Standardizing the features is good practice, though it won't affect the eigen values, eigen vectors will be different hence a different new subspace, standardized new subspace will be good for further machine learning tasks, especially if the features are measured on different scales.
- Deciding the value of
k
= Number of features in the new sub space can be done as :-- Select the P percent of information to be sustained.
- Calculate cumulative % of information from top to bottom eigen values.
- If the cumulative % is greater than or equal to P exit or else go to step 2.
- Select the eigen vectors of the select top 'k' eigen values.
- Using these selected vectors transform the original data to the desired new sub space.