Introduction

When multiple TensorFlow experiments write into the same log path, TensorBoard becomes hard to trust because lines overlap and run names lose meaning. A clean run separation strategy lets you compare models, hyperparameters, and datasets without confusion. The key is to treat every training execution as a traceable experiment with a unique log directory and saved metadata.

Why Run Separation Matters

TensorBoard groups events by log directory. If two jobs write to one folder, metrics from different experiments may appear under one run label. That leads to bad decisions, especially when you are tuning learning rate, batch size, or optimizer.

A reliable setup gives you three guarantees:

1. Every run has a unique path.
2. Every path has enough metadata to explain what was trained.
3. The parent directory is structured so TensorBoard can compare runs side by side.

Use a consistent naming scheme from day one. A practical format is model, dataset, key hyperparameters, random seed, and timestamp.

logs/fit/resnet18_cifar10_adam_lr1e-3_bs64_seed42_20260304_101500

Pattern 1: Custom Training Loop with Explicit Writer

For a custom loop, create the writer once per run and write summaries at meaningful intervals.

1from pathlib import Path
2from datetime import datetime
3import tensorflow as tf
4
5run_name = datetime.now().strftime("mlp_mnist_sgd_lr1e-2_seed7_%Y%m%d_%H%M%S")
6logdir = Path("logs/fit") / run_name
7logdir.mkdir(parents=True, exist_ok=True)
8
9writer = tf.summary.create_file_writer(str(logdir))
10
11for step in range(1, 51):
12    loss = 1.0 / step
13    acc = min(0.99, 0.5 + step * 0.01)
14    with writer.as_default():
15        tf.summary.scalar("train/loss", loss, step=step)
16        tf.summary.scalar("train/accuracy", acc, step=step)
17
18writer.flush()
19print(f"Wrote TensorBoard events to {logdir}")

This layout keeps tags stable and easy to compare. Use a namespace pattern like train/loss and val/loss so charts remain organized.

Pattern 2: Keras model.fit Callback per Experiment

If you use model.fit, attach a TensorBoard callback with a run specific directory. Avoid hardcoded paths reused across runs.

1from pathlib import Path
2from datetime import datetime
3import tensorflow as tf
4
5run_name = datetime.now().strftime("dense_regression_adam_lr1e-3_bs32_%Y%m%d_%H%M%S")
6logdir = Path("logs/fit") / run_name
7logdir.mkdir(parents=True, exist_ok=True)
8
9x = tf.random.normal((512, 10))
10y = tf.reduce_sum(x, axis=1, keepdims=True) + tf.random.normal((512, 1), stddev=0.1)
11
12model = tf.keras.Sequential([
13    tf.keras.layers.Input(shape=(10,)),
14    tf.keras.layers.Dense(32, activation="relu"),
15    tf.keras.layers.Dense(1)
16])
17model.compile(optimizer="adam", loss="mse", metrics=["mae"])
18
19tb = tf.keras.callbacks.TensorBoard(
20    log_dir=str(logdir),
21    histogram_freq=0,
22    update_freq="epoch"
23)
24
25model.fit(x, y, validation_split=0.2, epochs=5, callbacks=[tb], verbose=1)
26print(f"Run directory: {logdir}")

This gives a clean run boundary for each training invocation.

Save Experiment Metadata Next to Event Files

Event files alone are not enough for reproducibility. Save a small config file in the same run folder so you can reconstruct the experiment later.

1import json
2from pathlib import Path
3
4config = {
5    "model": "dense_regression",
6    "optimizer": "adam",
7    "learning_rate": 1e-3,
8    "batch_size": 32,
9    "seed": 7,
10    "dataset": "synthetic"
11}
12
13run_dir = Path("logs/fit/example_run")
14run_dir.mkdir(parents=True, exist_ok=True)
15(run_dir / "config.json").write_text(json.dumps(config, indent=2))

Store the same fields for every run. Consistency matters more than the exact schema.

Launch and Compare Runs

Point TensorBoard at the common parent directory.

tensorboard --logdir logs/fit --port 6006

Once loaded, compare only runs from the same task and metric definition. If you renamed a metric halfway through a project, old and new charts may look unrelated even when training behavior is similar.

Also keep retention rules. Large projects produce many event files, so archive stable milestone runs and remove broken exploratory runs.

Common Pitfalls

• Reusing the same log directory for multiple jobs, which merges metrics and destroys comparison quality.
• Using run names that hide critical context such as learning rate or seed.
• Forgetting to save metadata files, making old runs impossible to interpret.
• Comparing runs with inconsistent metric tags such as loss in one run and train_loss in another.
• Keeping every run forever and letting log storage grow until TensorBoard becomes slow.

Summary

• Give each TensorFlow run a unique log directory under one shared parent path.
• Use stable run naming conventions that encode key experiment settings.
• Write summaries with consistent tag names so cross run charts are meaningful.
• Save configuration metadata next to TensorBoard event files for reproducibility.
• Compare runs intentionally and manage retention so experiment tracking stays usable.