Dimensionality Reduction

Abstract

Having more features for inference may be thought of better than having just a handful of features. However, in machine learning, data scientists usually treat having many features as a “curse of dimensionality.” The reasoning behind this is having large dimensions makes data exploration and visualization difficult. It is also computationally expensive to train models on high-dimensional datasets. After all, every dimension may not play a significant role in machine learning. Thus, it is always an advantage to reduce the dimensions. There are several techniques available for dimensionality reductions. I have covered almost 14 different techniques in this chapter. Some of these are trivial and require manual inspection; while there are many advanced techniques which are fully automated. To list a few, I discuss factor analysis, PCA, ICA, t-SNE, UMAP, SVD, and LDA. I describe each technique with the implementation code on an appropriate dataset and perform a series of experiments to show you the effectiveness of each technique. This will help you in gaining a solid knowledge of dimensionality reduction, which is a critical step in data science process.