Introduction

cross_val_score works best when X is a single indexable object. Multi-input Keras models break that assumption because the model wants several tensors, while scikit-learn wants one feature container it can split into folds. The result is usually a shape error, a failed slice operation, or a wrapper that silently feeds the wrong data into the network.

Why Multi-Input Models Clash with cross_val_score

A two-branch Keras model might accept one numeric input and one categorical embedding input. Keras is fine with a call such as model.fit([x_num, x_cat], y), but cross_val_score has to do more than call fit. It must shuffle, slice, and hand each training fold to the estimator in a consistent way.

The old tf.keras.wrappers.scikit_learn.KerasClassifier is awkward here because scikit-learn expects X to behave like one array. A plain Python list containing two arrays is not a great fit for fold slicing. One practical fix is to package each sample as a single object so scikit-learn can split rows first, then unpack the two inputs inside the estimator.

A Scikit-Learn-Friendly Wrapper

The example below stores each sample as a tuple containing two NumPy arrays. That gives cross_val_score a one-dimensional object array to index, while the estimator reconstructs the two Keras inputs before training.

1import numpy as np
2from sklearn.base import BaseEstimator, ClassifierMixin
3from sklearn.model_selection import cross_val_score
4from sklearn.metrics import accuracy_score
5import tensorflow as tf
6
7
8def build_model(num_features: int, cat_features: int) -> tf.keras.Model:
9    num_input = tf.keras.Input(shape=(num_features,), name="num_input")
10    cat_input = tf.keras.Input(shape=(cat_features,), name="cat_input")
11
12    num_branch = tf.keras.layers.Dense(8, activation="relu")(num_input)
13    cat_branch = tf.keras.layers.Dense(4, activation="relu")(cat_input)
14    merged = tf.keras.layers.Concatenate()([num_branch, cat_branch])
15    output = tf.keras.layers.Dense(1, activation="sigmoid")(merged)
16
17    model = tf.keras.Model(inputs=[num_input, cat_input], outputs=output)
18    model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
19    return model
20
21
22class MultiInputKerasClassifier(BaseEstimator, ClassifierMixin):
23    def __init__(self, epochs=10, batch_size=16, verbose=0):
24        self.epochs = epochs
25        self.batch_size = batch_size
26        self.verbose = verbose
27        self.model_ = None
28
29    def _unpack(self, X):
30        x_num = np.stack([row[0] for row in X])
31        x_cat = np.stack([row[1] for row in X])
32        return x_num, x_cat
33
34    def fit(self, X, y):
35        x_num, x_cat = self._unpack(X)
36        self.model_ = build_model(x_num.shape[1], x_cat.shape[1])
37        self.model_.fit(
38            [x_num, x_cat],
39            y,
40            epochs=self.epochs,
41            batch_size=self.batch_size,
42            verbose=self.verbose,
43        )
44        return self
45
46    def predict(self, X):
47        x_num, x_cat = self._unpack(X)
48        probs = self.model_.predict([x_num, x_cat], verbose=0).ravel()
49        return (probs >= 0.5).astype(int)
50
51    def score(self, X, y):
52        return accuracy_score(y, self.predict(X))
53
54
55rng = np.random.default_rng(7)
56n_samples = 120
57x_num = rng.normal(size=(n_samples, 3)).astype("float32")
58x_cat = rng.integers(0, 2, size=(n_samples, 2)).astype("float32")
59y = ((x_num[:, 0] + x_cat[:, 0]) > 0).astype(int)
60
61X = np.empty(n_samples, dtype=object)
62for i in range(n_samples):
63    X[i] = (x_num[i], x_cat[i])
64
65estimator = MultiInputKerasClassifier(epochs=5)
66scores = cross_val_score(estimator, X, y, cv=3)
67print(scores)
68print(scores.mean())

This approach keeps scikit-learn in charge of the folds while leaving Keras in charge of the model definition.

When a Manual KFold Loop Is Better

If you already have several tensors, a manual loop is often simpler than forcing everything through cross_val_score. You keep full control over callbacks, early stopping, sample weights, and per-input preprocessing.

1from sklearn.model_selection import KFold
2
3kf = KFold(n_splits=3, shuffle=True, random_state=42)
4fold_scores = []
5
6for train_idx, test_idx in kf.split(x_num):
7    model = build_model(x_num.shape[1], x_cat.shape[1])
8    model.fit(
9        [x_num[train_idx], x_cat[train_idx]],
10        y[train_idx],
11        epochs=5,
12        verbose=0,
13    )
14    preds = model.predict([x_num[test_idx], x_cat[test_idx]], verbose=0).ravel()
15    fold_scores.append(accuracy_score(y[test_idx], preds >= 0.5))
16
17print(fold_scores)

This is a little more code, but it is easier to debug. For real projects, that tradeoff is usually worth it.

Common Pitfalls

The most common failure is passing X as a raw list such as [x_num, x_cat] and expecting cross_val_score to split both arrays correctly. Scikit-learn splits samples, not model inputs, so package the data in a single indexable container first.

Another problem is reusing the same model instance across folds. Cross-validation only makes sense when each fold starts from fresh weights. Build the model inside fit or inside the fold loop.

Randomness also causes confusion. Neural networks vary from run to run, so set seeds if you need comparable fold scores. Even then, exact reproducibility can still depend on hardware and TensorFlow settings.

Finally, watch preprocessing. If one branch is scaled and another is not, keep that logic inside the estimator or inside a per-fold pipeline. Do not fit preprocessing once on the full dataset before cross-validation, or you will leak information from the validation folds.

Summary

• 'cross_val_score expects one indexable X, while multi-input Keras models expect several tensors.'
• A custom estimator can unpack each sample into multiple Keras inputs after scikit-learn creates the fold split.
• A manual KFold loop is often clearer when your training flow needs callbacks, custom metrics, or complex preprocessing.
• Rebuild the model for every fold so weights do not leak across validation runs.
• Keep preprocessing and input alignment consistent across all branches of the network.