Introduction
TensorFlow summaries (tf.summary) let you log scalars, histograms, images, and other data to TensorBoard for visualization during training. When using tf.slim or tf.layers, summaries are not automatically collected — you must explicitly add them or configure the layers to emit them. In TF 1.x, summaries are graph operations merged with tf.summary.merge_all() and written by a tf.summary.FileWriter. In TF 2.x, tf.summary uses eager-mode writers and works differently.
Adding Summaries with tf.layers (TF 1.x)
tf.layers does not create summary operations by default. You add them manually after defining layers:
1import tensorflow as tf
2
3inputs = tf.placeholder(tf.float32, [None, 784])
4
5# Define layers
6hidden = tf.layers.dense(inputs, 256, activation=tf.nn.relu, name='hidden')
7logits = tf.layers.dense(hidden, 10, name='output')
8
9# Add histogram summaries for layer weights
10for var in tf.trainable_variables():
11 tf.summary.histogram(var.name, var)
12
13# Add activation summary
14tf.summary.histogram('hidden_activations', hidden)
15
16# Merge all summaries
17merged = tf.summary.merge_all()
18
19with tf.Session() as sess:
20 writer = tf.summary.FileWriter('./logs', sess.graph)
21 sess.run(tf.global_variables_initializer())
22
23 for step in range(1000):
24 _, summary = sess.run([train_op, merged], feed_dict={inputs: batch_x})
25 writer.add_summary(summary, step)
26
27 writer.close()
Adding Summaries with tf.slim
tf.slim provides slim.summarize_tensors() and slim.summarize_collection() to batch-add summaries:
1import tensorflow as tf
2import tf_slim as slim # or tensorflow.contrib.slim in older TF
3
4inputs = tf.placeholder(tf.float32, [None, 28, 28, 1])
5
6# Define a model with slim
7net = slim.conv2d(inputs, 32, [3, 3], scope='conv1')
8net = slim.max_pool2d(net, [2, 2], scope='pool1')
9net = slim.conv2d(net, 64, [3, 3], scope='conv2')
10net = slim.flatten(net)
11logits = slim.fully_connected(net, 10, activation_fn=None, scope='output')
12
13# Summarize all trainable variables (weights and biases)
14slim.summarize_tensors(tf.trainable_variables())
15
16# Or summarize all model variables
17slim.summarize_collection(tf.GraphKeys.MODEL_VARIABLES)
18
19# Summaries for activations — add them to a collection during model definition
20slim.summarize_activation(net)
21
22merged = tf.summary.merge_all()
slim.summarize_tensors() adds histogram summaries for each tensor in the list. slim.summarize_activation() adds both a histogram and a scalar summary (for sparsity) of an activation tensor.
Using slim.learning.train() with Summaries
slim.learning.train() handles summary writing automatically:
1import tf_slim as slim
2
3# Define model, loss, optimizer
4loss = slim.losses.softmax_cross_entropy(logits, labels)
5optimizer = tf.train.AdamOptimizer(0.001)
6train_op = slim.learning.create_train_op(loss, optimizer)
7
8# Add summaries
9tf.summary.scalar('loss', loss)
10slim.summarize_tensors(tf.trainable_variables())
11
12# train() handles merging summaries, writing them, and checkpointing
13slim.learning.train(
14 train_op,
15 logdir='./logs',
16 number_of_steps=10000,
17 save_summaries_secs=60, # Write summaries every 60 seconds
18 save_interval_secs=600 # Save checkpoint every 10 minutes
19)
Custom Summaries for Loss and Metrics
1# Scalar summaries for loss tracking
2loss = tf.losses.sparse_softmax_cross_entropy(labels, logits)
3tf.summary.scalar('cross_entropy_loss', loss)
4
5# Accuracy metric
6predictions = tf.argmax(logits, axis=1)
7accuracy = tf.reduce_mean(tf.cast(tf.equal(predictions, labels), tf.float32))
8tf.summary.scalar('accuracy', accuracy)
9
10# Image summaries for input visualization
11tf.summary.image('input_images', inputs, max_outputs=4)
12
13# Gradient summaries
14grads = optimizer.compute_gradients(loss)
15for grad, var in grads:
16 if grad is not None:
17 tf.summary.histogram(f'gradients/{var.name}', grad)
TF 2.x Approach (Keras and Eager Mode)
In TF 2.x, tf.slim and tf.layers are deprecated in favor of tf.keras.layers. Summaries use a different API:
1import tensorflow as tf
2
3# Create a summary writer
4writer = tf.summary.create_file_writer('./logs')
5
6model = tf.keras.Sequential([
7 tf.keras.layers.Dense(256, activation='relu'),
8 tf.keras.layers.Dense(10)
9])
10
11optimizer = tf.keras.optimizers.Adam(0.001)
12loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
13
14for step, (x_batch, y_batch) in enumerate(dataset):
15 with tf.GradientTape() as tape:
16 logits = model(x_batch, training=True)
17 loss = loss_fn(y_batch, logits)
18
19 grads = tape.gradient(loss, model.trainable_variables)
20 optimizer.apply_gradients(zip(grads, model.trainable_variables))
21
22 # Write summaries manually
23 with writer.as_default(step=step):
24 tf.summary.scalar('loss', loss)
25 for var in model.trainable_variables:
26 tf.summary.histogram(var.name, var)
Or use the built-in TensorBoard callback with model.fit():
tensorboard_cb = tf.keras.callbacks.TensorBoard(log_dir='./logs', histogram_freq=1)
model.fit(x_train, y_train, epochs=10, callbacks=[tensorboard_cb])
Common Pitfalls
Forgetting tf.summary.merge_all(): In TF 1.x, individual tf.summary.* calls create operations but do not run them. You must merge them and evaluate the merged op in sess.run(). Without this, no summaries are written to disk.
Adding summaries after merge_all(): merge_all() only merges summaries that exist at the time it is called. Summaries added afterward are not included. Define all summaries before calling merge_all().
Summary name collisions: Two summaries with the same name overwrite each other in TensorBoard. Use unique names or scopes (with tf.name_scope('train'):) to namespace your summaries.
Writing summaries every step: Writing summaries on every training step slows training significantly because histogram computation and disk I/O are expensive. Write every N steps or use save_summaries_secs to throttle.
Mixing TF 1.x and TF 2.x summary APIs: tf.compat.v1.summary (TF 1.x) and tf.summary (TF 2.x) use different writers and are not interchangeable. Pick one API and use it consistently.
Summary
tf.layers requires manual tf.summary.histogram() calls for weights and activations
tf.slim provides summarize_tensors() and summarize_activation() helpers
slim.learning.train() handles summary merging and writing automatically
In TF 1.x, always call tf.summary.merge_all() and run the merged op in the session
In TF 2.x, use tf.summary.create_file_writer() with eager mode or the TensorBoard Keras callback
Write summaries periodically (not every step) to avoid slowing down training
