sklearn.compose.ColumnTransformer fit_transform takes 2 positional arguments but 3 were given

python

sklearn

ColumnTransformer

error-handling

machine-learning

sklearn.compose.ColumnTransformer fit_transform takes 2 positional arguments but 3 were given

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Start Practicing Learn More

Introduction

The error fit_transform() takes 2 positional arguments but 3 were given occurs when you pass both X and y to a ColumnTransformer's fit_transform() method. Unlike estimators like RandomForestClassifier that expect fit(X, y), transformers in scikit-learn expect fit(X) or fit_transform(X) — they transform features without needing the target variable. The fix is to remove the y argument from the transformer call or restructure your pipeline so the transformer and estimator are properly chained.

The Error

python

1from sklearn.compose import ColumnTransformer
2from sklearn.preprocessing import StandardScaler, OneHotEncoder
3import pandas as pd
4
5df = pd.DataFrame({
6    "age": [25, 30, 35],
7    "city": ["NYC", "LA", "NYC"],
8    "salary": [50000, 60000, 70000]
9})
10
11X = df[["age", "city"]]
12y = df["salary"]
13
14ct = ColumnTransformer([
15    ("num", StandardScaler(), ["age"]),
16    ("cat", OneHotEncoder(), ["city"])
17])
18
19# WRONG: passing y to a transformer
20ct.fit_transform(X, y)
21# TypeError: fit_transform() takes 2 positional arguments but 3 were given

The Fix

python

1# FIX: transformers don't need y — only pass X
2X_transformed = ct.fit_transform(X)
3print(X_transformed)
4# [[-1.22  0.  1.]
5#  [ 0.    1.  0.]
6#  [ 1.22  0.  1.]]

ColumnTransformer.fit_transform() only accepts X. It transforms feature columns — the target variable y is not used.

Why Transformers Don't Take y

Transformers preprocess input features independently of the target. StandardScaler computes mean and standard deviation from X alone. OneHotEncoder maps categories from X alone. The target y is only needed by supervised estimators (classifiers and regressors) during fit().

python

1# Transformer: fit(X) — no y needed
2scaler = StandardScaler()
3X_scaled = scaler.fit_transform(X[["age"]])
4
5# Estimator: fit(X, y) — needs y for supervised learning
6from sklearn.linear_model import LinearRegression
7model = LinearRegression()
8model.fit(X_scaled, y)  # y is required here

Using Pipeline to Chain Transformer + Estimator

The correct way to combine preprocessing and prediction is with a Pipeline:

python

1from sklearn.pipeline import Pipeline
2from sklearn.compose import ColumnTransformer
3from sklearn.preprocessing import StandardScaler, OneHotEncoder
4from sklearn.linear_model import LinearRegression
5
6ct = ColumnTransformer([
7    ("num", StandardScaler(), ["age"]),
8    ("cat", OneHotEncoder(handle_unknown="ignore"), ["city"])
9])
10
11# Pipeline chains transformer → estimator
12pipeline = Pipeline([
13    ("preprocessor", ct),
14    ("regressor", LinearRegression())
15])
16
17# Now fit takes both X and y — Pipeline handles the routing
18pipeline.fit(X, y)
19predictions = pipeline.predict(X)
20print(predictions)

The Pipeline passes X through the transformer, then passes the transformed X and y to the estimator. You only call fit(X, y) on the Pipeline — it handles the argument routing internally.

Common ColumnTransformer Patterns

python

1import numpy as np
2from sklearn.compose import ColumnTransformer, make_column_selector
3from sklearn.preprocessing import StandardScaler, OneHotEncoder
4from sklearn.impute import SimpleImputer
5from sklearn.pipeline import Pipeline
6
7# Numeric pipeline: impute → scale
8numeric_pipeline = Pipeline([
9    ("imputer", SimpleImputer(strategy="median")),
10    ("scaler", StandardScaler())
11])
12
13# Categorical pipeline: impute → encode
14categorical_pipeline = Pipeline([
15    ("imputer", SimpleImputer(strategy="constant", fill_value="missing")),
16    ("encoder", OneHotEncoder(handle_unknown="ignore"))
17])
18
19# Auto-detect column types
20ct = ColumnTransformer([
21    ("num", numeric_pipeline, make_column_selector(dtype_include=np.number)),
22    ("cat", categorical_pipeline, make_column_selector(dtype_include=object))
23])
24
25# Transform only (no y)
26X_transformed = ct.fit_transform(X)
27
28# Or in a full pipeline with an estimator
29from sklearn.ensemble import RandomForestRegressor
30
31full_pipeline = Pipeline([
32    ("preprocessor", ct),
33    ("model", RandomForestRegressor(n_estimators=100))
34])
35
36full_pipeline.fit(X, y)  # Works — Pipeline handles y routing

TransformedTargetRegressor for y Transformation

If you need to transform the target y, use TransformedTargetRegressor:

python

1from sklearn.compose import TransformedTargetRegressor
2from sklearn.preprocessing import StandardScaler
3from sklearn.linear_model import LinearRegression
4
5# Transform y (e.g., log-transform the target)
6model = TransformedTargetRegressor(
7    regressor=LinearRegression(),
8    transformer=StandardScaler()
9)
10
11model.fit(X_transformed, y)  # y is scaled internally
12predictions = model.predict(X_transformed)  # predictions are inverse-transformed

Common Pitfalls

Passing y to ColumnTransformer.fit_transform(): Transformers preprocess features only — they do not use the target variable. Pass only X to fit_transform(). Use a Pipeline to connect the transformer to an estimator that needs y.
Confusing fit_transform(X) with fit(X, y): Transformers use fit(X) and transform(X) (or fit_transform(X)). Estimators use fit(X, y) and predict(X). Pipelines unify both interfaces so you can call pipeline.fit(X, y) and it routes arguments correctly.
Not using make_column_selector for dynamic column selection: Hard-coding column names breaks when columns change. make_column_selector(dtype_include=np.number) automatically selects numeric columns regardless of their names.
Forgetting handle_unknown="ignore" on OneHotEncoder: If test data contains categories not seen during training, OneHotEncoder raises an error by default. Set handle_unknown="ignore" to encode unknown categories as all-zeros.
Calling transform() before fit(): ColumnTransformer.transform(X) requires that fit(X) has been called first to learn the transformations. Calling transform() on an unfitted transformer raises NotFittedError.

Summary

ColumnTransformer.fit_transform() accepts only X — do not pass y
Transformers preprocess features independently of the target variable
Use Pipeline to chain ColumnTransformer with an estimator that needs y
Use make_column_selector for automatic column type detection
Use TransformedTargetRegressor if you need to transform the target variable
Always call fit() or fit_transform() before transform() on any transformer