Practice Now
tensorflow
tf-slim
tf-layers
summary-operations
machine-learning

Using summary with tf slim or tf layers

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design
TensorFlow & KerasMachine Learning

Introduction

TensorFlow summaries (tf.summary) let you log scalars, histograms, images, and other data to TensorBoard for visualization during training. When using tf.slim or tf.layers, summaries are not automatically collected — you must explicitly add them or configure the layers to emit them. In TF 1.x, summaries are graph operations merged with tf.summary.merge_all() and written by a tf.summary.FileWriter. In TF 2.x, tf.summary uses eager-mode writers and works differently.

Adding Summaries with tf.layers (TF 1.x)

tf.layers does not create summary operations by default. You add them manually after defining layers:

python
1import tensorflow as tf
2
3inputs = tf.placeholder(tf.float32, [None, 784])
4
5# Define layers
6hidden = tf.layers.dense(inputs, 256, activation=tf.nn.relu, name='hidden')
7logits = tf.layers.dense(hidden, 10, name='output')
8
9# Add histogram summaries for layer weights
10for var in tf.trainable_variables():
11    tf.summary.histogram(var.name, var)
12
13# Add activation summary
14tf.summary.histogram('hidden_activations', hidden)
15
16# Merge all summaries
17merged = tf.summary.merge_all()
18
19with tf.Session() as sess:
20    writer = tf.summary.FileWriter('./logs', sess.graph)
21    sess.run(tf.global_variables_initializer())
22
23    for step in range(1000):
24        _, summary = sess.run([train_op, merged], feed_dict={inputs: batch_x})
25        writer.add_summary(summary, step)
26
27    writer.close()

Adding Summaries with tf.slim

tf.slim provides slim.summarize_tensors() and slim.summarize_collection() to batch-add summaries:

python
1import tensorflow as tf
2import tf_slim as slim  # or tensorflow.contrib.slim in older TF
3
4inputs = tf.placeholder(tf.float32, [None, 28, 28, 1])
5
6# Define a model with slim
7net = slim.conv2d(inputs, 32, [3, 3], scope='conv1')
8net = slim.max_pool2d(net, [2, 2], scope='pool1')
9net = slim.conv2d(net, 64, [3, 3], scope='conv2')
10net = slim.flatten(net)
11logits = slim.fully_connected(net, 10, activation_fn=None, scope='output')
12
13# Summarize all trainable variables (weights and biases)
14slim.summarize_tensors(tf.trainable_variables())
15
16# Or summarize all model variables
17slim.summarize_collection(tf.GraphKeys.MODEL_VARIABLES)
18
19# Summaries for activations — add them to a collection during model definition
20slim.summarize_activation(net)
21
22merged = tf.summary.merge_all()

slim.summarize_tensors() adds histogram summaries for each tensor in the list. slim.summarize_activation() adds both a histogram and a scalar summary (for sparsity) of an activation tensor.

Using slim.learning.train() with Summaries

slim.learning.train() handles summary writing automatically:

python
1import tf_slim as slim
2
3# Define model, loss, optimizer
4loss = slim.losses.softmax_cross_entropy(logits, labels)
5optimizer = tf.train.AdamOptimizer(0.001)
6train_op = slim.learning.create_train_op(loss, optimizer)
7
8# Add summaries
9tf.summary.scalar('loss', loss)
10slim.summarize_tensors(tf.trainable_variables())
11
12# train() handles merging summaries, writing them, and checkpointing
13slim.learning.train(
14    train_op,
15    logdir='./logs',
16    number_of_steps=10000,
17    save_summaries_secs=60,    # Write summaries every 60 seconds
18    save_interval_secs=600     # Save checkpoint every 10 minutes
19)

Custom Summaries for Loss and Metrics

python
1# Scalar summaries for loss tracking
2loss = tf.losses.sparse_softmax_cross_entropy(labels, logits)
3tf.summary.scalar('cross_entropy_loss', loss)
4
5# Accuracy metric
6predictions = tf.argmax(logits, axis=1)
7accuracy = tf.reduce_mean(tf.cast(tf.equal(predictions, labels), tf.float32))
8tf.summary.scalar('accuracy', accuracy)
9
10# Image summaries for input visualization
11tf.summary.image('input_images', inputs, max_outputs=4)
12
13# Gradient summaries
14grads = optimizer.compute_gradients(loss)
15for grad, var in grads:
16    if grad is not None:
17        tf.summary.histogram(f'gradients/{var.name}', grad)

TF 2.x Approach (Keras and Eager Mode)

In TF 2.x, tf.slim and tf.layers are deprecated in favor of tf.keras.layers. Summaries use a different API:

python
1import tensorflow as tf
2
3# Create a summary writer
4writer = tf.summary.create_file_writer('./logs')
5
6model = tf.keras.Sequential([
7    tf.keras.layers.Dense(256, activation='relu'),
8    tf.keras.layers.Dense(10)
9])
10
11optimizer = tf.keras.optimizers.Adam(0.001)
12loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
13
14for step, (x_batch, y_batch) in enumerate(dataset):
15    with tf.GradientTape() as tape:
16        logits = model(x_batch, training=True)
17        loss = loss_fn(y_batch, logits)
18
19    grads = tape.gradient(loss, model.trainable_variables)
20    optimizer.apply_gradients(zip(grads, model.trainable_variables))
21
22    # Write summaries manually
23    with writer.as_default(step=step):
24        tf.summary.scalar('loss', loss)
25        for var in model.trainable_variables:
26            tf.summary.histogram(var.name, var)

Or use the built-in TensorBoard callback with model.fit():

python
tensorboard_cb = tf.keras.callbacks.TensorBoard(log_dir='./logs', histogram_freq=1)
model.fit(x_train, y_train, epochs=10, callbacks=[tensorboard_cb])

Common Pitfalls

  • Forgetting tf.summary.merge_all(): In TF 1.x, individual tf.summary.* calls create operations but do not run them. You must merge them and evaluate the merged op in sess.run(). Without this, no summaries are written to disk.
  • Adding summaries after merge_all(): merge_all() only merges summaries that exist at the time it is called. Summaries added afterward are not included. Define all summaries before calling merge_all().
  • Summary name collisions: Two summaries with the same name overwrite each other in TensorBoard. Use unique names or scopes (with tf.name_scope('train'):) to namespace your summaries.
  • Writing summaries every step: Writing summaries on every training step slows training significantly because histogram computation and disk I/O are expensive. Write every N steps or use save_summaries_secs to throttle.
  • Mixing TF 1.x and TF 2.x summary APIs: tf.compat.v1.summary (TF 1.x) and tf.summary (TF 2.x) use different writers and are not interchangeable. Pick one API and use it consistently.

Summary

  • tf.layers requires manual tf.summary.histogram() calls for weights and activations
  • tf.slim provides summarize_tensors() and summarize_activation() helpers
  • slim.learning.train() handles summary merging and writing automatically
  • In TF 1.x, always call tf.summary.merge_all() and run the merged op in the session
  • In TF 2.x, use tf.summary.create_file_writer() with eager mode or the TensorBoard Keras callback
  • Write summaries periodically (not every step) to avoid slowing down training
Related reading
Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design
Codemia
© 2026 Codemia
Resources
Legal
Social
All Rights Reserved.