Introduction

Many tuning problems have more than one target. You may want the highest validation score, but you may also care about latency, model size, or training cost. Optuna supports this directly with multi-objective studies, where the goal is to discover a Pareto front instead of a single best trial.

Define Objectives Explicitly

In a multi-objective study, the objective function returns one value per metric. You also tell Optuna whether each metric should be minimized or maximized.

This example minimizes training time and minimizes error rate for a random forest model:

1import time
2
3import optuna
4from sklearn.datasets import load_wine
5from sklearn.ensemble import RandomForestClassifier
6from sklearn.metrics import accuracy_score
7from sklearn.model_selection import train_test_split
8
9X, y = load_wine(return_X_y=True)
10X_train, X_valid, y_train, y_valid = train_test_split(
11    X, y, test_size=0.25, random_state=42, stratify=y
12)
13
14
15def objective(trial):
16    n_estimators = trial.suggest_int("n_estimators", 20, 200)
17    max_depth = trial.suggest_int("max_depth", 2, 16)
18    min_samples_split = trial.suggest_int("min_samples_split", 2, 10)
19
20    model = RandomForestClassifier(
21        n_estimators=n_estimators,
22        max_depth=max_depth,
23        min_samples_split=min_samples_split,
24        random_state=42,
25        n_jobs=-1,
26    )
27
28    start = time.perf_counter()
29    model.fit(X_train, y_train)
30    fit_time = time.perf_counter() - start
31
32    predictions = model.predict(X_valid)
33    error_rate = 1.0 - accuracy_score(y_valid, predictions)
34    return fit_time, error_rate
35
36
37study = optuna.create_study(directions=["minimize", "minimize"])
38study.optimize(objective, n_trials=40)

The important difference from single-objective tuning is the return value. Instead of returning one scalar, the objective returns a tuple with one value per metric.

Read the Pareto Front

There is no single "best" trial unless you choose one trade-off yourself. Optuna stores the non-dominated trials in study.best_trials.

for trial in study.best_trials:
    print("values:", trial.values, "params:", trial.params)

Each trial on the Pareto front is competitive in a different way. One may be slightly slower but much more accurate. Another may be fast enough for production while giving up a small amount of quality. That is the real value of multi-objective optimization: it exposes choices instead of hiding them behind one aggregate score.

If you want a visual view of the trade-off, Optuna also provides a Pareto plot:

1from optuna.visualization import plot_pareto_front
2
3fig = plot_pareto_front(
4    study,
5    target_names=["fit_time", "error_rate"],
6)
7fig.show()

Choose Metrics That Reflect Real Constraints

A common mistake is optimizing metrics that do not match the production problem. For example, accuracy and training time are fine for a tutorial, but a real system might care about:

• validation loss and inference latency
• recall and model size
• revenue impact and false-positive rate

If one metric is only a soft preference, you can still use multi-objective search, then apply a business rule after tuning. For instance, you might filter to all trials with latency below 30 milliseconds and then choose the most accurate one from that filtered set.

Another practical detail is direction. Metrics such as error, loss, latency, and memory use are usually minimized. Metrics such as accuracy, F1, and AUC are usually maximized. If the directions list is wrong, the study will optimize the opposite of what you want.

When to Combine Metrics Instead

Multi-objective optimization is useful when trade-offs are real and you do not want to force them into one number too early. It is not always the right tool. If you already know that one metric must dominate and others are just penalties, a single weighted score can be simpler.

For example, if latency is only relevant above a threshold, you can encode that directly in one scalar objective. But if you genuinely want to explore the frontier between speed and quality, keeping the objectives separate is more informative and usually easier to reason about later.

Common Pitfalls

The first pitfall is expecting study.best_trial to work like a single-objective study. Multi-objective studies usually use study.best_trials, because there are multiple non-dominated candidates.

Another problem is mixing metrics with unstable evaluation noise. If one objective varies wildly across runs, the Pareto front becomes difficult to trust. Use fixed seeds, stable validation splits, or repeated evaluation where practical.

It is also easy to choose too many objectives. Two or three metrics are manageable. Past that, the search space and decision process become much harder to interpret.

Finally, do not forget deployment constraints. A beautiful Pareto front on offline validation may still contain models that are too large, too slow, or too expensive to serve in the real system.

Summary

• In Optuna, multi-objective optimization means returning multiple values from the objective function.
• 'create_study(directions=[...]) defines whether each metric should be minimized or maximized.'
• The result is a Pareto front, not one universally best trial.
• 'study.best_trials helps you inspect the non-dominated candidates.'
• Use separate objectives when trade-offs are real and worth preserving.