Introduction

TensorBoard is useful during hyperparameter tuning because it lets you compare loss curves, learning rates, and metrics across multiple runs. The main trick when combining it with a grid search is to give every training run its own log directory, otherwise different parameter combinations overwrite each other.

Why TensorBoard Needs Special Handling in Grid Search

A normal Keras training run can use a single TensorBoard callback and a fixed log_dir. That stops working once you loop through many parameter combinations, because each fit call writes event files to the same location.

For grid search you want:

• A unique log directory per parameter combination.
• Usually another unique suffix per validation fold.
• A reproducible mapping from run name to hyperparameters.

If you do not enforce that, TensorBoard will show mixed curves that are hard to interpret.

A Manual Grid Search Pattern That Works Well

The simplest reliable approach is to run the parameter grid yourself and create the callback inside the loop. This example uses ParameterGrid from scikit-learn and logs each run separately.

1from pathlib import Path
2from sklearn.datasets import make_classification
3from sklearn.model_selection import ParameterGrid, train_test_split
4from sklearn.preprocessing import StandardScaler
5import tensorflow as tf
6
7X, y = make_classification(
8    n_samples=1000,
9    n_features=20,
10    n_informative=10,
11    random_state=42,
12)
13
14X_train, X_val, y_train, y_val = train_test_split(
15    X, y, test_size=0.2, random_state=42
16)
17
18scaler = StandardScaler()
19X_train = scaler.fit_transform(X_train)
20X_val = scaler.transform(X_val)
21
22def build_model(units, learning_rate):
23    model = tf.keras.Sequential(
24        [
25            tf.keras.layers.Input(shape=(20,)),
26            tf.keras.layers.Dense(units, activation="relu"),
27            tf.keras.layers.Dense(1, activation="sigmoid"),
28        ]
29    )
30    model.compile(
31        optimizer=tf.keras.optimizers.Adam(learning_rate=learning_rate),
32        loss="binary_crossentropy",
33        metrics=["accuracy"],
34    )
35    return model
36
37grid = ParameterGrid(
38    {
39        "units": [16, 32],
40        "learning_rate": [1e-2, 1e-3],
41        "batch_size": [32],
42    }
43)
44
45base_log_dir = Path("runs")
46results = []
47
48for run_id, params in enumerate(grid, start=1):
49    model = build_model(params["units"], params["learning_rate"])
50    log_dir = base_log_dir / f"run_{run_id}_u{params['units']}_lr{params['learning_rate']}"
51    tensorboard = tf.keras.callbacks.TensorBoard(
52        log_dir=str(log_dir),
53        histogram_freq=1,
54    )
55
56    history = model.fit(
57        X_train,
58        y_train,
59        validation_data=(X_val, y_val),
60        epochs=5,
61        batch_size=params["batch_size"],
62        verbose=0,
63        callbacks=[tensorboard],
64    )
65
66    best_val_acc = max(history.history["val_accuracy"])
67    results.append((params, best_val_acc))
68
69print(sorted(results, key=lambda item: item[1], reverse=True)[0])

This pattern is explicit, easy to debug, and TensorBoard-friendly.

Launch TensorBoard and Compare Runs

After training, point TensorBoard at the base directory:

tensorboard --logdir runs

TensorBoard groups the event files by subdirectory, so each parameter combination becomes a separate run. If your naming convention includes the hyperparameters, comparison becomes much easier.

Practical naming advice:

• Keep run names short.
• Include the parameters that actually vary.
• Avoid timestamps unless you really need them, because deterministic names are easier to compare.

What About GridSearchCV With Keras Wrappers

You can wrap a Keras model for use with scikit-learn style search, but callback management becomes more awkward. The search object clones estimators and refits across folds, so you still need a way to create unique log directories per fit.

That is why many teams prefer one of these approaches:

• Manual ParameterGrid loops for full callback control.
• KerasTuner for dedicated neural-network tuning workflows.
• Custom wrapper code that builds callbacks dynamically inside each fit call.

If TensorBoard analysis is important, manual loops are usually the least surprising option.

Logging More Than Scalars

TensorBoard can record more than training loss and accuracy. When useful, enable:

• Histograms for weights and activations.
• Profiling for selected batches.
• Embeddings for learned representations.

Those are powerful, but start with scalar metrics first. During a grid search, the biggest win usually comes from clear run separation, not from logging every possible artifact.

Common Pitfalls

• Reusing one log_dir across every grid-search run.
• Naming runs so vaguely that you cannot tell which hyperparameters produced them.
• Logging too much information and creating huge event files for a small tuning exercise.
• Using a wrapper-based search without thinking about how callbacks are recreated for each fit.
• Comparing runs that used different preprocessing pipelines and then blaming the network settings.

Summary

• The key to TensorBoard plus grid search is one log directory per run.
• A manual ParameterGrid loop is often the simplest way to keep TensorBoard output clean.
• Build the TensorBoard callback inside the loop so each fit call gets its own path.
• Launch TensorBoard on the parent directory to compare runs side by side.
• Keep preprocessing and run naming consistent, or the comparison will be misleading.