Introduction

In scikit-learn, the fit method trains a model on your data. But what happens when you call fit a second time? The answer depends on the type of estimator you are using. Most scikit-learn models reset their parameters completely when you call fit again, meaning the second call starts fresh and discards everything from the first. However, some models support incremental learning through the partial_fit method, which lets you update an existing model with new data. This article explains both behaviors, shows when each approach is appropriate, and provides code examples for common scenarios.

Default Behavior: fit Resets the Model

For the majority of scikit-learn estimators, calling fit a second time completely replaces the model's learned parameters. The first training session is discarded.

1from sklearn.linear_model import LinearRegression
2import numpy as np
3
4X_train1 = np.array([[1], [2], [3]])
5y_train1 = np.array([2, 4, 6])
6
7X_train2 = np.array([[10], [20], [30]])
8y_train2 = np.array([15, 25, 35])
9
10model = LinearRegression()
11
12# First training
13model.fit(X_train1, y_train1)
14print(f"After first fit: coef={model.coef_}, intercept={model.intercept_:.2f}")
15
16# Second training (completely replaces the first)
17model.fit(X_train2, y_train2)
18print(f"After second fit: coef={model.coef_}, intercept={model.intercept_:.2f}")

The second fit call does not build on the first. The model is retrained from scratch on X_train2 and y_train2 as if the first call never happened. This applies to RandomForestClassifier, SVC, DecisionTreeClassifier, KNeighborsClassifier, and essentially all standard estimators.

When Calling fit Twice Is Useful

Even though the second call resets the model, there are valid reasons to call fit multiple times on the same estimator object:

Hyperparameter tuning. You change the model's hyperparameters between calls to compare results:

1from sklearn.ensemble import RandomForestClassifier
2from sklearn.datasets import make_classification
3from sklearn.model_selection import train_test_split
4
5X, y = make_classification(n_samples=1000, random_state=42)
6X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
7
8model = RandomForestClassifier(n_estimators=10, random_state=42)
9model.fit(X_train, y_train)
10print(f"10 trees: {model.score(X_test, y_test):.3f}")
11
12model.n_estimators = 100
13model.fit(X_train, y_train)
14print(f"100 trees: {model.score(X_test, y_test):.3f}")

Retraining on updated data. When new data arrives and you want a fresh model that incorporates everything:

1import numpy as np
2
3# Combine old and new data
4X_combined = np.vstack([X_train1, X_train2])
5y_combined = np.concatenate([y_train1, y_train2])
6
7model.fit(X_combined, y_combined)

This trains a brand new model on the combined dataset, which is the correct approach for most scikit-learn estimators.

Incremental Learning with partial_fit

Some estimators support partial_fit, which updates the model with new data without discarding what was learned before. This is essential for large datasets that do not fit in memory or for online learning scenarios where data arrives in streams.

1from sklearn.linear_model import SGDClassifier
2from sklearn.datasets import make_classification
3import numpy as np
4
5X, y = make_classification(n_samples=10000, random_state=42)
6
7model = SGDClassifier(random_state=42)
8
9# Train in chunks of 1000
10chunk_size = 1000
11classes = np.unique(y)
12
13for i in range(0, len(X), chunk_size):
14    X_chunk = X[i:i + chunk_size]
15    y_chunk = y[i:i + chunk_size]
16    model.partial_fit(X_chunk, y_chunk, classes=classes)
17    print(f"Chunk {i // chunk_size + 1}: "
18          f"score={model.score(X, y):.3f}")

The classes parameter is required on the first call to partial_fit so the model knows all possible target values upfront. On subsequent calls, you can omit it.

Estimators That Support partial_fit

Not all models support incremental learning. Here are the most commonly used ones:

• SGDClassifier and SGDRegressor for linear models with stochastic gradient descent
• MultinomialNB, BernoulliNB, and ComplementNB for Naive Bayes classifiers
• MiniBatchKMeans for clustering
• Perceptron for simple linear classification
• PassiveAggressiveClassifier and PassiveAggressiveRegressor

1from sklearn.naive_bayes import MultinomialNB
2import numpy as np
3
4model = MultinomialNB()
5
6# First batch
7X_batch1 = np.array([[1, 0, 1], [0, 1, 1]])
8y_batch1 = np.array([0, 1])
9model.partial_fit(X_batch1, y_batch1, classes=[0, 1])
10
11# Second batch (updates, does not reset)
12X_batch2 = np.array([[1, 1, 0], [0, 0, 1]])
13y_batch2 = np.array([0, 1])
14model.partial_fit(X_batch2, y_batch2)
15
16print(model.predict([[1, 0, 1]]))

warm_start: Continuing Tree-Based Training

Some ensemble models like RandomForestClassifier and GradientBoostingClassifier support a warm_start parameter. When set to True, calling fit again adds more estimators to the existing ensemble instead of starting over.

1from sklearn.ensemble import GradientBoostingClassifier
2from sklearn.datasets import make_classification
3
4X, y = make_classification(n_samples=1000, random_state=42)
5
6model = GradientBoostingClassifier(
7    n_estimators=50,
8    warm_start=True,
9    random_state=42
10)
11
12# First fit: trains 50 trees
13model.fit(X, y)
14print(f"After first fit: {len(model.estimators_)} trees")
15
16# Increase n_estimators and fit again: adds 50 more trees
17model.n_estimators = 100
18model.fit(X, y)
19print(f"After second fit: {len(model.estimators_)} trees")

With warm_start=True, the second fit call adds 50 new trees to the existing 50, giving you 100 total. Without warm_start, both calls would train 50 trees from scratch.

Cross-Validation: Training Multiple Times Automatically

If your goal is to evaluate a model by training it on different data splits, use cross_val_score instead of manually calling fit multiple times:

1from sklearn.model_selection import cross_val_score
2from sklearn.ensemble import RandomForestClassifier
3
4model = RandomForestClassifier(n_estimators=100, random_state=42)
5scores = cross_val_score(model, X, y, cv=5)
6
7print(f"Scores: {scores}")
8print(f"Mean: {scores.mean():.3f}, Std: {scores.std():.3f}")

This trains the model 5 times, each time on a different 80/20 split of the data, and returns the score for each fold.

Common Pitfalls

Assuming fit accumulates knowledge. This is the most common misconception. Calling fit twice on different datasets does not combine the learnings. The second call always starts fresh unless you use partial_fit or warm_start.

Forgetting to pass classes to partial_fit. On the first call, partial_fit needs to know all possible classes. If a class does not appear in the first batch but appears later, the model will raise an error because it was not told about it during initialization.

Using warm_start with a smaller n_estimators. If you set n_estimators to a number smaller than or equal to the current count, fit does nothing because there are no new estimators to add. This is a silent no-op that can be confusing.

Mixing up fit and partial_fit on the same model. Calling fit after partial_fit resets the model completely. If you have been training incrementally and accidentally call fit, all incremental progress is lost.

Summary

In scikit-learn, calling fit a second time resets the model and trains from scratch on the new data. This is the standard behavior for most estimators. For incremental learning where you want to update a model with new data without losing prior knowledge, use partial_fit on estimators that support it. For ensemble methods where you want to continue adding trees, use warm_start=True. Choose the approach that matches your use case: full retraining for clean starts, partial_fit for streaming data, and warm_start for growing ensembles.