How to visualize a tensor summary in tensorboard

tensorboard

tensor visualization

deep learning

machine learning tools

data analysis

How to visualize a tensor summary in tensorboard

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Deep Learning TensorFlow & Keras Machine Learning Data Science

Introduction

To visualize tensor summaries in TensorBoard, use tf.summary writers to log scalar, histogram, image, and text data during training, then launch TensorBoard to view the results. In TensorFlow 2, the workflow is: create a tf.summary.create_file_writer(), call tf.summary.scalar(), tf.summary.histogram(), or tf.summary.image() inside the training loop, and run tensorboard --logdir=logs to open the dashboard. TensorBoard reads the logged event files and renders interactive visualizations of loss curves, weight distributions, activation maps, and more.

Basic Setup

python

1import tensorflow as tf
2import datetime
3
4# Create a log directory with timestamp
5log_dir = "logs/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
6summary_writer = tf.summary.create_file_writer(log_dir)

Scalar Summaries (Loss, Accuracy)

The most common visualization — tracking metrics over training steps:

python

1model = tf.keras.Sequential([
2    tf.keras.layers.Dense(128, activation='relu'),
3    tf.keras.layers.Dense(10, activation='softmax')
4])
5model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
6
7# Using Keras callback (simplest approach)
8tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
9model.fit(x_train, y_train, epochs=10, validation_data=(x_val, y_val),
10          callbacks=[tensorboard_callback])

Manual scalar logging in a custom training loop:

python

1optimizer = tf.keras.optimizers.Adam()
2loss_fn = tf.keras.losses.SparseCategoricalCrossentropy()
3
4for epoch in range(10):
5    for step, (x_batch, y_batch) in enumerate(train_dataset):
6        with tf.GradientTape() as tape:
7            predictions = model(x_batch, training=True)
8            loss = loss_fn(y_batch, predictions)
9
10        gradients = tape.gradient(loss, model.trainable_variables)
11        optimizer.apply_gradients(zip(gradients, model.trainable_variables))
12
13        # Log scalar values
14        with summary_writer.as_default():
15            tf.summary.scalar('loss', loss, step=epoch * steps_per_epoch + step)
16            tf.summary.scalar('learning_rate', optimizer.learning_rate, step=epoch)

Histogram Summaries (Weights, Gradients)

Visualize the distribution of tensors over time:

python

1for epoch in range(10):
2    # Training step...
3
4    with summary_writer.as_default():
5        # Log weight distributions
6        for layer in model.layers:
7            for weight in layer.weights:
8                tf.summary.histogram(f'weights/{weight.name}', weight, step=epoch)
9
10        # Log gradient distributions
11        with tf.GradientTape() as tape:
12            predictions = model(x_batch, training=True)
13            loss = loss_fn(y_batch, predictions)
14        gradients = tape.gradient(loss, model.trainable_variables)
15
16        for grad, var in zip(gradients, model.trainable_variables):
17            if grad is not None:
18                tf.summary.histogram(f'gradients/{var.name}', grad, step=epoch)

In TensorBoard, histograms appear in the "Histograms" tab and show how tensor value distributions change across training steps.

Image Summaries

Log images to visualize inputs, predictions, or feature maps:

python

1with summary_writer.as_default():
2    # Log a batch of input images
3    tf.summary.image('training_images', x_batch[:8], step=epoch, max_outputs=8)
4
5    # Log model predictions as images
6    pred_images = tf.reshape(predictions[:8], [-1, 28, 28, 1])
7    tf.summary.image('predictions', pred_images, step=epoch, max_outputs=8)
8
9    # Log feature maps from an intermediate layer
10    feature_model = tf.keras.Model(inputs=model.input, outputs=model.layers[0].output)
11    features = feature_model(x_batch[:1])
12    # Reshape features for visualization (batch, h, w, channels)
13    feature_maps = tf.transpose(features[0:1], perm=[3, 1, 2, 0])  # (channels, h, w, 1)
14    tf.summary.image('feature_maps', feature_maps[:16], step=epoch, max_outputs=16)

Text Summaries

Log text data like hyperparameters or sample predictions:

python

1with summary_writer.as_default():
2    # Log hyperparameters as text
3    hparams_text = f"lr={0.001}, batch_size={32}, epochs={10}"
4    tf.summary.text('hyperparameters', hparams_text, step=0)
5
6    # Log sample predictions
7    for i in range(5):
8        tf.summary.text('predictions', f'Input: {inputs[i]}, Pred: {preds[i]}', step=i)

Custom Summaries with tf.summary.experimental

python

1# Log a confusion matrix as an image
2import matplotlib.pyplot as plt
3import numpy as np
4from sklearn.metrics import confusion_matrix
5import io
6
7def plot_confusion_matrix(y_true, y_pred, class_names):
8    cm = confusion_matrix(y_true, y_pred)
9    fig, ax = plt.subplots(figsize=(8, 8))
10    ax.imshow(cm, cmap='Blues')
11    ax.set_xticks(range(len(class_names)))
12    ax.set_yticks(range(len(class_names)))
13    ax.set_xticklabels(class_names, rotation=45)
14    ax.set_yticklabels(class_names)
15    for i in range(len(class_names)):
16        for j in range(len(class_names)):
17            ax.text(j, i, str(cm[i, j]), ha='center', va='center')
18    buf = io.BytesIO()
19    fig.savefig(buf, format='png')
20    plt.close(fig)
21    buf.seek(0)
22    image = tf.image.decode_png(buf.getvalue(), channels=4)
23    return tf.expand_dims(image, 0)
24
25with summary_writer.as_default():
26    cm_image = plot_confusion_matrix(y_true, y_pred, class_names)
27    tf.summary.image('confusion_matrix', cm_image, step=epoch)

Launching TensorBoard

bash

1# Start TensorBoard
2tensorboard --logdir=logs
3
4# With a specific port
5tensorboard --logdir=logs --port=6007
6
7# Compare multiple runs
8tensorboard --logdir=logs  # Each subdirectory is a separate run
9
10# In Jupyter notebook
11%load_ext tensorboard
12%tensorboard --logdir logs

Common Pitfalls

Forgetting summary_writer.as_default() context: Summary operations only write to the active writer. Without with summary_writer.as_default():, tf.summary.scalar() silently does nothing. Always wrap summary calls in the writer context manager.
Not flushing the writer: Summary data is buffered. If your script crashes or you check TensorBoard before training finishes, recent data may not appear. Call summary_writer.flush() periodically or at the end of each epoch to force writes.
Stale logs from previous runs: TensorBoard reads all event files in the log directory. Old runs show up as overlapping curves. Use timestamped subdirectories (logs/run_20240101_120000/) or delete old logs before starting a new experiment.
Wrong tensor shape for tf.summary.image: Images must have shape (batch, height, width, channels) with values in [0, 1] (float) or [0, 255] (uint8). Forgetting to normalize or reshape causes blank images or errors.
Using histogram_freq without validation data: The TensorBoard callback's histogram_freq parameter logs weight histograms every N epochs, but only when validation_data is provided to model.fit(). Without it, histograms are silently skipped.

Summary

Use tf.summary.scalar() for loss and accuracy curves, tf.summary.histogram() for weight distributions
Use tf.summary.image() to visualize inputs, predictions, and feature maps
The TensorBoard Keras callback handles most logging automatically with histogram_freq=1
For custom training loops, create a tf.summary.create_file_writer() and use the as_default() context manager
Launch with tensorboard --logdir=logs and use timestamped subdirectories to separate experiments

Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course

Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

How to visualize a tensor summary in tensorboard

ML System Design practice on Codemia

Introduction

Basic Setup

Scalar Summaries (Loss, Accuracy)

Histogram Summaries (Weights, Gradients)

Image Summaries

Text Summaries

Custom Summaries with tf.summary.experimental

Launching TensorBoard

Common Pitfalls

Summary

Related reading

Tackling System Design Interview Problems

Track what you have practised

ML System Design practice on Codemia