Introduction

Text classification pipelines often need more than just one text column. Real datasets may include title text, body text, and numerical metadata such as length, score, or count features. In scikit-learn, the core idea is to build separate transformations for each feature group and then concatenate their outputs before classification. Historically, FeatureUnion was the common tool for this. In modern scikit-learn, ColumnTransformer is often cleaner, but understanding the union pattern still helps.

Think in Terms of Parallel Feature Pipelines

Suppose a dataset has:

1. title as short text.
2. body as longer text.
3. num_links as a numeric feature.
4. author_score as another numeric feature.

A good pipeline processes each group independently:

1. Vectorize title text.
2. Vectorize body text.
3. Scale or pass through numeric columns.
4. Concatenate everything.
5. Train the classifier.

That is exactly what FeatureUnion and ColumnTransformer are designed for.

A Practical ColumnTransformer Example

For dataframe-based workflows, ColumnTransformer is usually simpler than hand-built FeatureUnion selectors.

1import pandas as pd
2from sklearn.compose import ColumnTransformer
3from sklearn.feature_extraction.text import TfidfVectorizer
4from sklearn.pipeline import Pipeline
5from sklearn.preprocessing import StandardScaler
6from sklearn.linear_model import LogisticRegression
7
8X = pd.DataFrame({
9    "title": ["cheap flights", "team wins final", "discount travel deal"],
10    "body": ["book now and save", "the club won the match", "limited time offer"],
11    "num_links": [5, 1, 4],
12    "author_score": [0.2, 0.9, 0.3]
13})
14y = [1, 0, 1]
15
16preprocess = ColumnTransformer([
17    ("title_tfidf", TfidfVectorizer(), "title"),
18    ("body_tfidf", TfidfVectorizer(), "body"),
19    ("numeric", StandardScaler(), ["num_links", "author_score"]),
20])
21
22model = Pipeline([
23    ("preprocess", preprocess),
24    ("clf", LogisticRegression(max_iter=1000))
25])
26
27model.fit(X, y)
28print(model.predict(X))

This is usually the most maintainable answer for multiple text and numeric columns in current scikit-learn.

Where FeatureUnion Fits In

FeatureUnion is still useful when each branch is itself a custom pipeline and you want to combine their transformed outputs explicitly.

A common older pattern is:

1. Select one column.
2. Apply a vectorizer or numeric transformer.
3. Repeat for the next column.
4. Combine with FeatureUnion.

That requires custom column selectors because FeatureUnion itself does not know how to choose dataframe columns.

1from sklearn.base import BaseEstimator, TransformerMixin
2
3class ColumnSelector(BaseEstimator, TransformerMixin):
4    def __init__(self, column):
5        self.column = column
6
7    def fit(self, X, y=None):
8        return self
9
10    def transform(self, X):
11        return X[self.column]

Then you can build separate branches and union them.

A FeatureUnion Example

1from sklearn.pipeline import Pipeline, FeatureUnion
2from sklearn.preprocessing import FunctionTransformer, StandardScaler
3from sklearn.feature_extraction.text import TfidfVectorizer
4from sklearn.linear_model import LogisticRegression
5
6text_title = Pipeline([
7    ("select", ColumnSelector("title")),
8    ("tfidf", TfidfVectorizer())
9])
10
11text_body = Pipeline([
12    ("select", ColumnSelector("body")),
13    ("tfidf", TfidfVectorizer())
14])
15
16numeric = Pipeline([
17    ("select", FunctionTransformer(lambda df: df[["num_links", "author_score"]], validate=False)),
18    ("scale", StandardScaler())
19])
20
21features = FeatureUnion([
22    ("title", text_title),
23    ("body", text_body),
24    ("numeric", numeric)
25])
26
27model = Pipeline([
28    ("features", features),
29    ("clf", LogisticRegression(max_iter=1000))
30])

This works, but it is more manual than the ColumnTransformer version.

Choose the Right Tool for the Codebase

A practical rule is:

1. Use ColumnTransformer when your inputs are structured columns and the preprocessing is column-driven.
2. Use FeatureUnion when you already have independent transformation pipelines whose outputs should be concatenated.

The title question mentions FeatureUnion, but for many modern projects the better answer is “use ColumnTransformer unless you specifically need the union abstraction.”

Keep Sparse and Dense Features in Mind

Text vectorizers usually produce sparse matrices, while scaled numeric features may start dense. Scikit-learn can combine them, but you should still be aware of the resulting matrix format and the classifier's suitability.

Linear models such as logistic regression and linear SVMs usually work well with these mixed sparse feature spaces. Some other estimators are less convenient or less efficient in that setup.

Common Pitfalls

• Trying to feed multiple dataframe columns directly into one vectorizer without selecting them properly first.
• Using FeatureUnion without a column-selection step and getting shape or type errors.
• Ignoring that ColumnTransformer is often the simpler modern solution for mixed-column preprocessing.
• Combining sparse text features with dense numeric features without thinking about estimator compatibility.
• Preprocessing columns outside the pipeline and making training and inference transformations drift apart.

Summary

• Mixed text-and-numeric classification problems should be modeled as parallel preprocessing branches that are concatenated before classification.
• 'ColumnTransformer is usually the cleanest modern solution for multiple text and numeric columns.'
• 'FeatureUnion still works when you want to combine independently built transformation pipelines.'
• Column selection is the essential step that makes multi-column preprocessing correct.
• Keeping preprocessing inside the scikit-learn pipeline is the best way to preserve reproducibility between training and prediction.