Understanding PCA and Explained Variance Ratio in Sklearn

Principal Component Analysis (PCA) is a popular technique often used in data science for dimensionality reduction, data visualization, speedup of machine learning algorithms, and finding hidden patterns in data. When performing PCA using Sklearn's PCA class, one essential property that is obtained is the explained_variance_ratio_. This article will delve into the concept of explained variance ratio in PCA, how to interpret it, and, critically, how to recover feature names from the explained variance ratio output of PCA with Sklearn.

What is Explained Variance Ratio?

The explained variance ratio indicates how much variance in the data each principal component accounts for. The PCA transformation results in a number of principal components equal to the number of original features. Each component captures a certain proportion of the total dataset's variance.

Mathematically, if $\lambda_i$ is the eigenvalue corresponding to the $i^{th}$ principal component, and $\Sigma$ is the sum of all eigenvalues, the explained variance ratio for the $i^{th}$ principal component can be calculated as:

$$EVR_i = \frac{\lambda_i}{\Sigma}$$

Performing PCA with Sklearn

To perform PCA using Sklearn, you need to first standardize the data, since PCA is sensitive to the scale of the features. The following is a basic workflow:

1import numpy as np
2from sklearn.preprocessing import StandardScaler
3from sklearn.decomposition import PCA
4
5# Sample data
6X = np.array([[2.5, 2.4], [0.5, 0.7], [2.2, 2.9], [1.9, 2.2], [3.1, 3.0], [2.3, 2.7], [2, 1.6], [1, 1.1], [1.5, 1.6], [1.1, 0.9]])
7
8# Standardization
9X_std = StandardScaler().fit_transform(X)
10
11# PCA fitting
12pca = PCA(n_components=2)
13principal_components = pca.fit_transform(X_std)

After performing PCA, the explained_variance_ratio_ attribute can be accessed to understand how much variance each component explains.

# Explained Variance Ratio
explained_variance_ratio = pca.explained_variance_ratio_

Recovering Feature Names

In PCA, the individual feature names do not have explicit output as they are transformed into principal components, which are linear combinations of the features. However, you can trace back to see which features have significant contributions to these components.

Features' Contribution to Principal Components

To ascertain which original features contribute most to each principal component, you can examine the components stored in pca.components_. These are the principal axes in feature space, and each row of this array corresponds to a principal component, while each column holds the coefficient value of the original feature.

1# Name the original features
2feature_names = ['Feature1', 'Feature2']
3
4# Retrieve the components
5components = pca.components_
6
7# Print each component's contribution
8for idx, component in enumerate(components):
9    print(f"Principal Component {idx+1}:")
10    for i, contribution in enumerate(component):
11        print(f"{feature_names[i]}: {contribution:.4f}")

Interpretation

Large absolute values in any column suggest that the particular feature contributes significantly to that principal component. One can create a comparative table for clarity:

This table shows that both features equally contribute to the first principal component, whereas the first feature contributes negatively while the second feature contributes positively to the second principal component.

Conclusion

PCA is a powerful method for reducing the dimensionality of data and identifying the most significant features in a dataset. Understanding the explained variance ratio offers insight into how much information is retained in each component, while examining the components themselves can help trace back which original features largely contribute to these components. Such discernment is crucial for data interpretation and ensuring that important information is preserved after transformation.

By becoming proficient with these PCA practices in Sklearn, you'll gain invaluable insights into your dataset and improve your data analysis capabilities.