Introduction

VotingClassifier is a useful way to combine several models, but tuning it is slightly different from tuning a single estimator. The key idea is that RandomizedSearchCV can search not only the top-level VotingClassifier settings, but also the parameters of each named child estimator.

How Parameter Search Works With a VotingClassifier

Scikit-learn exposes nested estimator parameters through a double-underscore naming convention. That means you do not search C directly for a logistic regression inside a voting ensemble. You search lr__C if the estimator was named lr, svc__C if it was named svc, and so on.

The names come from the estimators list passed to VotingClassifier.

1from sklearn.ensemble import VotingClassifier
2from sklearn.linear_model import LogisticRegression
3from sklearn.svm import SVC
4from sklearn.ensemble import RandomForestClassifier
5
6voting = VotingClassifier(
7    estimators=[
8        ("lr", LogisticRegression(max_iter=2000)),
9        ("svc", SVC(probability=True)),
10        ("rf", RandomForestClassifier(random_state=42)),
11    ],
12    voting="soft",
13)

In that configuration, valid search keys include lr__C, svc__gamma, rf__max_depth, and even top-level options such as voting or weights.

A Complete RandomizedSearchCV Example

The example below uses the Iris dataset so the code is small and runnable. It tunes a few parameters for each base estimator and also searches several ensemble weight combinations.

1from scipy.stats import loguniform, randint
2from sklearn.datasets import load_iris
3from sklearn.ensemble import RandomForestClassifier, VotingClassifier
4from sklearn.linear_model import LogisticRegression
5from sklearn.model_selection import RandomizedSearchCV, train_test_split
6from sklearn.svm import SVC
7
8X, y = load_iris(return_X_y=True)
9X_train, X_test, y_train, y_test = train_test_split(
10    X, y, test_size=0.2, random_state=42, stratify=y
11)
12
13voting = VotingClassifier(
14    estimators=[
15        ("lr", LogisticRegression(max_iter=2000, random_state=42)),
16        ("svc", SVC(probability=True, random_state=42)),
17        ("rf", RandomForestClassifier(random_state=42)),
18    ],
19    voting="soft",
20)
21
22param_distributions = {
23    "lr__C": loguniform(1e-3, 1e2),
24    "svc__C": loguniform(1e-2, 1e2),
25    "svc__gamma": loguniform(1e-4, 1e-1),
26    "rf__n_estimators": randint(50, 200),
27    "rf__max_depth": randint(2, 12),
28    "weights": [(1, 1, 1), (2, 1, 1), (1, 2, 1), (1, 1, 2)],
29}
30
31search = RandomizedSearchCV(
32    estimator=voting,
33    param_distributions=param_distributions,
34    n_iter=20,
35    cv=5,
36    scoring="accuracy",
37    random_state=42,
38    n_jobs=-1,
39)
40
41search.fit(X_train, y_train)
42
43print("Best params:", search.best_params_)
44print("Validation score:", search.best_score_)
45print("Test score:", search.best_estimator_.score(X_test, y_test))

This is the core pattern. If the ensemble fits successfully without the search, it will usually work under RandomizedSearchCV as long as the parameter names are correct.

When a Pipeline Is Involved

Many real models need preprocessing. If one of the child estimators is itself a Pipeline, the parameter path simply gets longer. For example, if the logistic regression estimator is a pipeline named lrpipe and its scaler step is named scale, then a parameter might look like lrpipe__clf__C or lrpipe__scale__with_mean, depending on the step names.

That nested naming scheme is the main thing to remember. RandomizedSearchCV does not care whether the target parameter lives one level deep or three levels deep, as long as the key matches the estimator tree.

Hard Voting Versus Soft Voting

For tuning, soft voting is often more flexible because it uses predicted probabilities and allows weights to matter more naturally. Hard voting only uses final class labels, which throws away some information.

There is one important constraint: if you set voting="soft", every estimator must support probability predictions. In practice that means calling SVC(probability=True) for support vector machines. Forgetting that option is a common reason why the search fails during fitting.

Designing a Good Search Space

Do not search every possible parameter just because you can. A wide but weakly informed search often wastes time and produces unstable comparisons between estimators.

A better pattern is:

• choose a few high-impact parameters per estimator
• use sensible distributions such as loguniform for regularization strengths
• keep n_iter aligned with the budget you actually have
• start with one metric, then revisit if the business goal differs from plain accuracy

If one model clearly needs scaling and another does not, consider wrapping only the relevant estimator in a pipeline rather than scaling the full dataset indiscriminately.

Common Pitfalls

The most common mistake is using the wrong parameter names. C will not work by itself inside a VotingClassifier; you need lr__C or svc__C based on the estimator name.

Another frequent failure is soft voting with an estimator that does not expose probabilities. For SVC, set probability=True before running the search.

It is also easy to mix incompatible preprocessing assumptions. Logistic regression and SVMs usually benefit from scaled features, while tree models do not require it. If you ignore that, your comparison between estimators is less meaningful.

Finally, avoid treating the cross-validation score as final truth. After the search, evaluate best_estimator_ on a held-out test set to confirm that the tuned ensemble generalizes.

Summary

• 'RandomizedSearchCV works with VotingClassifier through nested parameter names such as lr__C and rf__max_depth.'
• The estimator names in VotingClassifier(estimators=...) determine the search keys.
• Soft voting is often preferable, but every estimator must provide class probabilities.
• Keep the search space targeted instead of tuning every parameter blindly.
• Always confirm the best ensemble on a separate test set after cross-validation.