Introduction

Scikit learn models require numeric input, so categorical columns must be encoded before training. The correct encoder depends on whether categories are nominal or ordered and whether the model can handle sparse high dimensional features. A robust solution also handles unseen values during inference.

Choose the Right Encoder for the Feature Type

For unordered categories such as city or color, one hot encoding is usually the safest default. It creates one binary column per category and avoids fake numeric ordering.

For ordered categories such as small, medium, large, use ordinal encoding only when that order is meaningful and model behavior matches it.

1import pandas as pd
2from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder
3
4df = pd.DataFrame({
5    "city": ["Toronto", "Montreal", "Toronto", "Ottawa"],
6    "size": ["small", "medium", "large", "medium"],
7    "price": [100, 120, 180, 130],
8})
9
10one_hot = OneHotEncoder(handle_unknown="ignore", sparse_output=False)
11city_encoded = one_hot.fit_transform(df[["city"]])
12
13ordinal = OrdinalEncoder(categories=[["small", "medium", "large"]])
14size_encoded = ordinal.fit_transform(df[["size"]])
15
16print(city_encoded.shape)
17print(size_encoded.ravel())

Use ColumnTransformer in a Pipeline

In real projects, you have mixed numeric and categorical columns. ColumnTransformer lets you define preprocessing once and keep it tied to the model in a single pipeline.

1import pandas as pd
2from sklearn.compose import ColumnTransformer
3from sklearn.pipeline import Pipeline
4from sklearn.preprocessing import OneHotEncoder, StandardScaler
5from sklearn.linear_model import LogisticRegression
6from sklearn.model_selection import train_test_split
7
8X = pd.DataFrame({
9    "city": ["Toronto", "Ottawa", "Montreal", "Toronto", "Ottawa"],
10    "rooms": [1, 2, 3, 2, 1],
11    "rent": [1500, 1800, 2400, 1900, 1600],
12})
13y = [0, 1, 1, 1, 0]
14
15categorical_features = ["city"]
16numeric_features = ["rooms", "rent"]
17
18preprocess = ColumnTransformer(
19    transformers=[
20        ("cat", OneHotEncoder(handle_unknown="ignore"), categorical_features),
21        ("num", StandardScaler(), numeric_features),
22    ]
23)
24
25model = Pipeline(
26    steps=[
27        ("preprocess", preprocess),
28        ("clf", LogisticRegression(max_iter=1000)),
29    ]
30)
31
32X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
33model.fit(X_train, y_train)
34print(model.score(X_test, y_test))

This pattern prevents train test leakage because encoding is learned only from training data inside the pipeline.

Handle High Cardinality Carefully

A feature with thousands of unique values can explode dimensionality with one hot encoding. Consider feature hashing, target encoding with leakage safeguards, or grouping rare categories into an other bucket.

When you serve models, preserve the fitted encoder object and reuse it at inference time. Re fitting encoders online can reorder columns and break model inputs.

For tree based libraries that accept categorical types directly, check native support before encoding by hand, but for core scikit learn estimators, explicit encoding remains standard.

Persist and Reuse the Same Encoder in Production

Training and inference must share the exact same preprocessing object. The easiest approach is to save the fitted pipeline as one artifact and load it in your prediction service. This guarantees the encoded column layout is identical across environments.

1import joblib
2
3joblib.dump(model, "rent_classifier.joblib")
4loaded_model = joblib.load("rent_classifier.joblib")
5print(loaded_model.predict(X.iloc[:2]))

When data contracts evolve, add compatibility tests that compare prediction input schemas between current and previous model versions. That keeps encoder changes from breaking downstream consumers unexpectedly.

Common Pitfalls

• Applying label encoding to unordered categories: this injects false ranking.
• Fitting encoder before train test split: this leaks category information from test data.
• Forgetting handle_unknown="ignore": prediction fails on unseen categories.
• Rebuilding feature columns manually: column order drift leads to wrong predictions.
• Dropping too many categories for convenience: model loses predictive signal.

Summary

• Use one hot encoding for nominal categories and ordinal encoding only for real order.
• Wrap preprocessing and model in a pipeline to avoid leakage.
• Configure unknown category handling for stable inference.
• Plan for high cardinality features early to control feature size.
• Persist the fitted preprocessing pipeline with the model artifact.
• Version your preprocessing artifacts alongside model versions so rollback and reproducibility remain straightforward during incident response.