Practice Now
data analysis
machine learning
dummy variable
one-hot encoding
feature engineering

What's the difference between dummy variable and one-hot encoding?

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design
Machine LearningData Science

Introduction

Dummy variable encoding and one-hot encoding both convert categorical variables into numeric format for machine learning models, but they differ in the number of columns created. One-hot encoding creates k binary columns for k categories. Dummy variable encoding creates k-1 columns, dropping one category as a reference (the "dummy variable trap" avoidance). Use dummy variables for linear regression to avoid multicollinearity. Use one-hot encoding for tree-based models and neural networks where multicollinearity is not an issue.

One-Hot Encoding

Creates one binary column per category:

python
1import pandas as pd
2
3df = pd.DataFrame({"color": ["Red", "Blue", "Green", "Red", "Blue"]})
4
5# One-hot encoding: k columns for k categories
6one_hot = pd.get_dummies(df["color"])
7print(one_hot)
8#    Blue  Green  Red
9# 0     0      0    1
10# 1     1      0    0
11# 2     0      1    0
12# 3     0      0    1
13# 4     1      0    0

For 3 categories (Red, Blue, Green), one-hot encoding creates 3 columns. Each row has exactly one 1 and the rest 0.

Dummy Variable Encoding

Creates k-1 columns, dropping one reference category:

python
1# Dummy variables: k-1 columns (drop_first=True)
2dummies = pd.get_dummies(df["color"], drop_first=True)
3print(dummies)
4#    Green  Red
5# 0      0    1
6# 1      0    0   ← Blue is represented by (0, 0)
7# 2      1    0
8# 3      0    1
9# 4      0    0
10
11# Blue is the "reference" category — encoded as all zeros

The dropped category (Blue) is implicitly represented when all other columns are 0. This prevents the "dummy variable trap."

The Dummy Variable Trap

python
1# With one-hot encoding, columns are perfectly correlated:
2# Blue + Green + Red = 1 (always)
3# This means Blue = 1 - Green - Red (linear dependency)
4
5# In linear regression, this causes multicollinearity:
6# The model cannot independently estimate coefficients for all 3 columns
7# because one is a perfect linear combination of the others
8
9# Dummy encoding removes the dependency:
10# Green and Red are independent
11# Blue is implicit (when Green=0 and Red=0)

Multicollinearity inflates coefficient standard errors, making them unreliable. Dropping one category breaks the linear dependency.

Side-by-Side Comparison

python
1import pandas as pd
2from sklearn.preprocessing import OneHotEncoder
3
4data = pd.DataFrame({"size": ["Small", "Medium", "Large", "Small", "Large"]})
5
6# One-hot encoding (3 columns)
7one_hot = pd.get_dummies(data["size"])
8print("One-hot (k columns):")
9print(one_hot)
10#    Large  Medium  Small
11# 0      0       0      1
12# 1      0       1      0
13# 2      1       0      0
14# 3      0       0      1
15# 4      1       0      0
16
17# Dummy encoding (k-1 columns)
18dummy = pd.get_dummies(data["size"], drop_first=True)
19print("\nDummy (k-1 columns):")
20print(dummy)
21#    Medium  Small
22# 0       0      1
23# 1       1      0
24# 2       0      0   ← Large is (0, 0)
25# 3       0      1
26# 4       0      0

When to Use Each

Use Dummy Variables (k-1)

python
1from sklearn.linear_model import LinearRegression
2
3# Linear regression requires k-1 encoding
4X = pd.get_dummies(df[["color", "size"]], drop_first=True)
5y = df["price"]
6
7model = LinearRegression()
8model.fit(X, y)
9# Coefficients are interpretable:
10# coef for color_Red = price difference from Blue (reference)

Use dummy variables for:

  • Linear regression
  • Logistic regression
  • Any model that assumes feature independence
  • When interpretability of coefficients matters

Use One-Hot Encoding (k)

python
1from sklearn.ensemble import RandomForestClassifier
2
3# Tree-based models handle multicollinearity naturally
4X = pd.get_dummies(df[["color", "size"]])  # No drop_first
5y = df["target"]
6
7model = RandomForestClassifier()
8model.fit(X, y)

Use one-hot encoding for:

  • Decision trees and random forests
  • Neural networks
  • K-nearest neighbors
  • Any model that does not assume feature independence

Scikit-Learn Implementations

python
1from sklearn.preprocessing import OneHotEncoder, LabelEncoder
2import numpy as np
3
4data = np.array([["Red"], ["Blue"], ["Green"], ["Red"]])
5
6# OneHotEncoder — creates k columns by default
7encoder = OneHotEncoder(sparse_output=False)
8encoded = encoder.fit_transform(data)
9print(encoded)
10# [[0. 0. 1.]    Red
11#  [1. 0. 0.]    Blue
12#  [0. 1. 0.]    Green
13#  [0. 0. 1.]]   Red
14
15# Drop first for dummy encoding
16encoder = OneHotEncoder(sparse_output=False, drop="first")
17encoded = encoder.fit_transform(data)
18print(encoded)
19# [[0. 1.]    Red
20#  [0. 0.]    Blue (reference)
21#  [1. 0.]    Green
22#  [0. 1.]]   Red
23
24# Get feature names
25print(encoder.get_feature_names_out())
26# ['x0_Green', 'x0_Red']

Multiple Categorical Columns

python
1df = pd.DataFrame({
2    "color": ["Red", "Blue", "Green"],
3    "size": ["S", "M", "L"],
4    "price": [10, 20, 30]
5})
6
7# One-hot: 3 (color) + 3 (size) = 6 new columns
8one_hot = pd.get_dummies(df[["color", "size"]])
9print(one_hot.columns.tolist())
10# ['color_Blue', 'color_Green', 'color_Red', 'size_L', 'size_M', 'size_S']
11
12# Dummy: 2 (color) + 2 (size) = 4 new columns
13dummy = pd.get_dummies(df[["color", "size"]], drop_first=True)
14print(dummy.columns.tolist())
15# ['color_Green', 'color_Red', 'size_M', 'size_S']

Common Pitfalls

  • Using one-hot encoding with linear regression: Creates multicollinearity (the dummy variable trap). The model either fails to converge or produces unstable coefficients. Always use drop_first=True for linear models.
  • Dropping the wrong reference category: The dropped category becomes the baseline for interpreting coefficients. Choose a meaningful reference (e.g., "control group" or most common category) for better interpretability.
  • High cardinality: A column with 1,000 unique values creates 1,000 (or 999) new columns. This causes memory issues and overfitting. Use target encoding, frequency encoding, or embeddings for high-cardinality features.
  • New categories at prediction time: If the test set has a category not seen during training (e.g., "Purple" when training only had Red/Blue/Green), the encoder fails. Set handle_unknown="ignore" in OneHotEncoder to output all zeros for unknown categories.
  • Ordinal variables: Encoding ordered categories (Small < Medium < Large) as one-hot loses the ordering. Use ordinal encoding (0, 1, 2) instead to preserve the rank relationship.

Summary

  • One-hot encoding creates k binary columns for k categories — one column per category
  • Dummy variable encoding creates k-1 columns, dropping one reference category
  • Use dummy variables (drop_first=True) for linear/logistic regression to avoid multicollinearity
  • Use one-hot encoding for tree-based models and neural networks
  • In pandas, use pd.get_dummies(drop_first=True) for dummy encoding
  • In scikit-learn, use OneHotEncoder(drop="first") for dummy encoding
Related reading
Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design
Codemia
© 2026 Codemia
Resources
Legal
Social
All Rights Reserved.