Principal Component Analysis

Understanding PCA

Principal Component Analysis (PCA) is a powerful technique that helps you simplify and better understand complex datasets.

It does this by creating new features—called principal components—that summarize the original ones, while keeping as much important information as possible.

PCA helps you find patterns between your features and combine them into new ones.

For example, if you have height and weight, PCA can create one new feature that sums up both—like a measure of overall size.

To create a principal component, you need to standardize both features so you can compare them.

• Height range: 4.5 ft (54 in) to 6.5 ft (78 in)

• Weight range: 110 to 250 lbs

Okay now imagine we make these data points a set of coordinates (x= height, y=weight). Then we look for a line that best runs through these data points.

• The blue dots are your data points, where height is on the x-axis and weight is on the y-axis.

• The red dashed line represents the direction that best runs through the data—this is what PCA finds as the first principal component (PC1).

Let’s say PCA returns this slope:

• PC1 = 0.8 × Height + 0.6 × Weight

This means:

• Both height and weight contribute, but height has more influence.

• PC1 captures the main pattern in your data: people who are taller tend to weigh more.

This line captures the direction of PC1, and acts as a new axis for PC1.

You can now compute everyone's PC1 score using this formula: 0.8 × Height + 0.6 × Weight.

• This tells you where they fall along this new “summary” dimension.

• This tells you how far each person is along the PC1 axis: the new line that combines height and weight into one overall measure.

Hence you are turning the 2D plot graph into a 1D graph with everyone's PC1 score on it.

• Min:(0.8 × Height(0) + 0.6 × Weight(0))= 0

• Max:(0.8 × Height(1) + 0.6 × Weight(1))= 1.4