What is Dimensionality Reduction? A Brief Introduction

Name

Oluwaseun Adeojo

Published on 5/2/2023

(opens in a new tab) (opens in a new tab) (opens in a new tab) (opens in a new tab) (opens in a new tab) (opens in a new tab)

In the realm of data science, Dimensionality Reduction or Dimension Reduction is a pivotal process. This concept, essentially, revolves around the reduction of the number of input variables when developing a predictive model. More often than not, datasets come with an overwhelming number of features, some of which might not contribute significantly to the prediction. Dimension reduction techniques help us simplify the model without compromising the predictive accuracy.

High dimensional data is challenging to work with for several reasons. It increases computational complexity, can lead to overfitting and often hinders the visualization of the data. That's where Dimensionality Reduction Algorithms step in to save the day by helping us reduce the dimensions of the dataset without losing important information.

Want to quickly create Data Visualisation from Python Pandas Dataframe with No code?

PyGWalker is a Python library for Exploratory Data Analysis with Visualization. PyGWalker (opens in a new tab) can simplify your Jupyter Notebook data analysis and data visualization workflow, by turning your pandas dataframe (and polars dataframe) into a Tableau-style User Interface for visual exploration.

(opens in a new tab)

Different Dimensionality Reduction Techniques

Before we plunge into Python coding examples, let's get acquainted with some commonly employed Dimension Reduction Techniques.

Principal Component Analysis (PCA)

PCA is an unsupervised method used widely for Dimensional Reduction in fields like face recognition and image compression. It works by identifying the directions (Principal Components) that maximize the variance in the data.

Linear Discriminant Analysis (LDA)

Contrary to PCA, LDA is a supervised method and is used for dimensionality reduction in the field of pattern classification. It aims to find a projection that maximizes the separation (or discrimination) between multiple classes.

t-distributed Stochastic Neighbor Embedding (t-SNE)

t-SNE is a non-linear, probabilistic technique mainly used for data exploration and visualizing high-dimensional data. In simpler terms, t-SNE gives you a feel or intuition of how the data is arranged in a high-dimensional space.

Implementing Dimension Reduction in Python

Now, let's dive into some examples of how to implement these dimensionality reduction techniques in Python.

Reducing Dimensionality with PCA

Firstly, we'll start by importing necessary libraries and load the data.

from sklearn.decomposition import PCA
from sklearn import datasets
 
## Load the data
iris = datasets.load_iris()
X = iris.data
y = iris.target

To apply PCA, we create a PCA object and specify how many dimensions we want our data to be reduced to.

## Create a PCA object
pca = PCA(n_components=2)
 
## Reduce dimensionality
X_reduced = pca.fit_transform(X)

In this instance, we have reduced the data from 4 dimensions to 2 dimensions.

Implementing LDA

Similarly, let's apply LDA on the same iris dataset.

from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA
 
## Create an LDA object
lda = LDA(n_components=2)
 
## Reduce dimensionality
X_lda = lda.fit_transform(X, y)

Here, we have again transformed the data to two dimensions using LDA.

Visualizing High-Dimensional Data with t-SNE

Let's visualize our high-dimensional data in two dimensions using the t-SNE algorithm.

from sklearn.manifold import TSNE
 
## Create a TSNE object
tsne = TSNE(n_components=2)
 
## Reduce dimensionality
X_tsne = tsne.fit_transform(X)

Conclusion: Dimension Reduction as an Essential Tool

In this era of Big Data, reduction of dimensionality can be a key tool in transforming vast volumes of data into manageable, usable datasets. While there is a wide variety of Dimensionality Reduction Techniques available, it's important to choose one that fits your specific needs and data structure.

Dimension reduction doesn't just make data more manageable. It can lead to better and faster results, less storage space, and fewer computational resources. While it might seem like we're losing data, good dimensionality reduction algorithms actually maintain the most significant structures and features within the dataset, enhancing our understanding and interpretation of the data.

When next faced with a high-dimensional dataset, remember these powerful techniques and how Python makes it so simple to reduce complexity and streamline your data processing pipeline.