Introduction

If Keras or scikit-learn refuses to turn a string column into a categorical matrix, the usual reason is simple: one-hot encoding functions such as keras.utils.to_categorical() expect integer class indices, not raw strings. The fix is to encode the strings first, then build the categorical matrix from those numeric labels.

Understand Which Tool Expects What

There are two common workflows:

• for target labels, convert strings to integer ids, then call to_categorical
• for feature columns, use an encoder such as scikit-learn's OneHotEncoder

These are related but not identical jobs.

Keras Target Labels: Encode Then Call to_categorical

If your label column looks like "cat", "dog", or "bird", first map those strings to integers.

1import numpy as np
2from sklearn.preprocessing import LabelEncoder
3from tensorflow.keras.utils import to_categorical
4
5labels = np.array(["cat", "dog", "cat", "bird"])
6
7encoder = LabelEncoder()
8encoded = encoder.fit_transform(labels)
9print(encoded)
10
11categorical = to_categorical(encoded)
12print(categorical)

to_categorical works here because the strings were converted into class ids first.

Feature Columns: Use OneHotEncoder

If the string data is an input feature rather than the target, scikit-learn's OneHotEncoder is usually the right tool. The scikit-learn documentation explicitly supports string-valued categorical features.

1import numpy as np
2from sklearn.preprocessing import OneHotEncoder
3
4X = np.array([
5    ["red"],
6    ["blue"],
7    ["green"],
8    ["red"]
9])
10
11encoder = OneHotEncoder(sparse_output=False)
12X_encoded = encoder.fit_transform(X)
13print(X_encoded)
14print(encoder.get_feature_names_out())

This produces one-hot input features suitable for many scikit-learn models.

Modern Keras Option: StringLookup

If you want to stay inside Keras preprocessing, StringLookup is a clean modern alternative.

1import tensorflow as tf
2
3lookup = tf.keras.layers.StringLookup(output_mode="one_hot")
4lookup.adapt(tf.constant(["cat", "dog", "bird"]))
5
6encoded = lookup(tf.constant(["cat", "bird", "dog"]))
7print(encoded.numpy())

This is especially useful when the preprocessing layer should be part of the model pipeline itself.

Know Whether You Are Encoding Inputs Or Labels

A lot of confusion comes from mixing label encoding with feature encoding.

• labels are usually a single class id per row
• features may become multiple one-hot columns

If you apply the wrong tool to the wrong role, the output shape will not match what the model expects.

Keep The Mapping Consistent

Whatever encoder you choose, fit it on training data and reuse the same fitted encoder for validation, test, and inference. Re-fitting later on a different sample can change category order and silently scramble the meaning of encoded columns.

Saving the encoder or embedding the lookup layer in the model is often the safest pattern.

Handle Missing And Unknown Categories

Real datasets often contain null values or categories that appear later at inference time. Plan for that early.

With scikit-learn, OneHotEncoder(handle_unknown="ignore") can prevent failures when unseen categories appear. With Keras lookup layers, you can reserve out-of-vocabulary buckets.

Common Pitfalls

A common mistake is calling to_categorical() directly on raw strings. That function expects integer class indices.

Another mistake is using LabelEncoder for an input feature matrix and assuming the resulting integers are meaningful numeric distances. They are usually not. For input features, one-hot encoding is often safer.

It is also easy to forget that training and inference must use the same fitted encoder. If you refit on new data, category mappings can change.

Summary

• 'to_categorical() needs integer labels, not raw strings.'
• Use LabelEncoder or a similar mapping for string targets before one-hot conversion.
• Use OneHotEncoder for string input features in scikit-learn pipelines.
• 'StringLookup is a strong Keras-native option for categorical strings.'
• Keep the encoder consistent between training and inference.