Introduction

If you see AttributeError: 'XGBModel' object has no attribute 'evals_result_', the most likely issue is that you are mixing API styles. In current XGBoost scikit-learn wrappers, evaluation history is typically retrieved with the evals_result() method after fitting with an eval_set, not through an evals_result_ attribute.

Why the Error Happens

XGBoost exposes more than one Python interface. The native training API and the scikit-learn wrapper do not surface training history in exactly the same way, and older blog posts often mix them.

A common failing pattern looks like this:

1from xgboost import XGBRegressor
2
3model = XGBRegressor(n_estimators=50)
4model.fit(X_train, y_train)
5
6print(model.evals_result_)

There are two problems here:

• no evaluation set was supplied, so no per-round validation history was recorded
• the wrapper API expects evals_result() in modern documentation

That combination is what usually produces the attribute error.

The Correct Pattern in the Scikit-Learn Wrapper

To record evaluation metrics during training, pass an eval_set to fit(). After that, call evals_result().

1from sklearn.datasets import make_regression
2from sklearn.model_selection import train_test_split
3from xgboost import XGBRegressor
4
5X, y = make_regression(n_samples=500, n_features=10, noise=5.0, random_state=42)
6X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)
7
8model = XGBRegressor(
9    n_estimators=100,
10    learning_rate=0.1,
11    max_depth=4,
12    eval_metric="rmse",
13)
14
15model.fit(
16    X_train,
17    y_train,
18    eval_set=[(X_train, y_train), (X_val, y_val)],
19    verbose=False,
20)
21
22results = model.evals_result()
23print(results.keys())
24print(results["validation_0"]["rmse"][:3])
25print(results["validation_1"]["rmse"][:3])

This returns a dictionary keyed by validation set name, usually validation_0, validation_1, and so on.

Plotting the Training Curve

The evaluation history is most useful when you graph it. That makes overfitting or early convergence much easier to spot.

1import matplotlib.pyplot as plt
2
3results = model.evals_result()
4train_rmse = results["validation_0"]["rmse"]
5val_rmse = results["validation_1"]["rmse"]
6
7plt.plot(train_rmse, label="train")
8plt.plot(val_rmse, label="validation")
9plt.xlabel("boosting round")
10plt.ylabel("rmse")
11plt.legend()
12plt.tight_layout()
13plt.show()

If the validation curve stops improving while the training curve keeps improving, you are probably training for too many boosting rounds.

Early Stopping Works with the Same History

Evaluation history becomes especially useful when you enable early stopping.

1from xgboost import XGBClassifier
2from sklearn.datasets import load_breast_cancer
3from sklearn.model_selection import train_test_split
4
5X, y = load_breast_cancer(return_X_y=True)
6X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)
7
8model = XGBClassifier(
9    n_estimators=300,
10    learning_rate=0.05,
11    max_depth=3,
12    eval_metric="logloss",
13    early_stopping_rounds=20,
14)
15
16model.fit(
17    X_train,
18    y_train,
19    eval_set=[(X_val, y_val)],
20    verbose=False,
21)
22
23results = model.evals_result()
24print("rounds recorded:", len(results["validation_0"]["logloss"]))
25print("best iteration:", model.best_iteration)

Now you can inspect both the stopping point and the metric history that led to it.

Native API Looks Different

The native xgboost.train() function uses a different pattern. There you pass a dictionary that gets filled during training.

1import xgboost as xgb
2from sklearn.datasets import make_regression
3from sklearn.model_selection import train_test_split
4
5X, y = make_regression(n_samples=200, n_features=5, random_state=42)
6X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)
7
8dtrain = xgb.DMatrix(X_train, label=y_train)
9dval = xgb.DMatrix(X_val, label=y_val)
10
11evals_result = {}
12
13booster = xgb.train(
14    params={"objective": "reg:squarederror", "eval_metric": "rmse"},
15    dtrain=dtrain,
16    num_boost_round=50,
17    evals=[(dtrain, "train"), (dval, "val")],
18    evals_result=evals_result,
19    verbose_eval=False,
20)
21
22print(evals_result["train"]["rmse"][:3])

This is a separate interface, so code examples from it should not be copied directly into XGBRegressor or XGBClassifier examples.

A Safe Way to Access Results

If you are working in a codebase with mixed XGBoost versions or styles, guard the call explicitly.

1if hasattr(model, "evals_result"):
2    history = model.evals_result()
3    print(history)
4else:
5    print("This model wrapper does not expose evals_result().")

That is much safer than assuming an underscore-suffixed attribute will exist.

Common Pitfalls

A common mistake is forgetting the eval_set. Without it, there is nothing to record for each boosting round.

Another issue is copying outdated examples that use evals_result_ instead of the wrapper method documented in current releases.

Developers also sometimes expect evaluation history to survive every serialization path automatically. Save the history separately if your workflow depends on it.

Finally, do not mix native API patterns with scikit-learn wrapper patterns unless you are deliberately switching interfaces.

Summary

• In the XGBoost scikit-learn wrapper, use evals_result() rather than evals_result_.
• Pass an eval_set to fit() if you want per-round metric history.
• Use the returned dictionary to inspect training and validation metrics.
• Early stopping and evaluation history work naturally together.
• Keep native xgboost.train() examples separate from wrapper-based code.