Tensorboard without fit using keras and tf
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
TensorBoard does not depend on model.fit(). It only needs event files, and you can write those yourself from a custom training loop with tf.summary.
Create a Log Directory and Writer
The first step is to create a per-run log directory and a summary writer. Treat each run as a separate folder so TensorBoard can compare experiments cleanly.
That writer is the only piece model.fit() normally hides from you. Once it exists, you can log anything you want at any step.
Log Metrics from a Custom Training Loop
A manual loop is useful when training does not fit the standard epoch and batch pattern. Reinforcement learning, alternating losses, and gradient accumulation are common examples.
Here is a minimal loop that trains a binary classifier and logs loss every ten steps:
The key rule is that every summary needs a name and a step. TensorBoard builds its charts from those two pieces of information.
Record Validation Metrics and Debug Signals
Once you own the loop, you are not limited to the metrics Keras callbacks happen to expose. You can log validation accuracy, learning rate, gradient norms, or any internal signal that helps explain model behavior.
This is one of the biggest advantages of manual logging. You decide when validation runs and what data belongs in TensorBoard, instead of forcing your workflow through fit().
Export a Graph When You Need It
If you want the Graphs tab, trace the function explicitly. That is especially useful when checking whether tf.function compiled your step function the way you expect.
Graph tracing is optional. Scalars are enough for most projects, but graphs are valuable when you are debugging custom layers or tracing behavior.
Start TensorBoard
After writing summaries, launch TensorBoard against the parent directory:
If you keep one run per subdirectory, TensorBoard can overlay curves from multiple experiments. That becomes much more useful than the default fit() output once you start comparing hyperparameters or custom schedules.
When This Approach Is Better Than fit()
Custom summary writing is usually the right choice when:
- one training step contains multiple optimizers or losses,
- evaluation happens at irregular intervals,
- you want to log custom tensors or intermediate activations,
- training state comes from an environment or simulator instead of a dataset iterator.
In those situations, TensorBoard is still the same tool. The only difference is that you take responsibility for writing the summaries yourself.
Common Pitfalls
- Reusing the same log directory across unrelated runs, which mixes curves and makes comparisons meaningless.
- Forgetting
writer.as_default(), so summaries are not written where you expect. - Logging metrics with inconsistent tag names such as
loss,train_loss, andtrain/lossfor the same concept. - Using step values that reset or jump around, which produces confusing charts.
- Never calling
writer.flush()in short-lived scripts, causing recent summaries to appear missing.
Summary
- TensorBoard works without
model.fit()because it reads event files, not Keras callbacks. - Use
tf.summary.create_file_writerto create a writer and log directory for each run. - Write scalars, histograms, and traces manually from your custom loop.
- Stable tag names and monotonic step values make TensorBoard runs easy to compare.
- Manual logging is the better approach whenever your training process does not match the standard
fit()workflow.
Related reading
- Tensorflow-GPU import tensorflow ImportError Could not find 'cudnn64_7.dll
- Tensorflow-gpu issue CUDA runtime error device kernel image is invalid
- tensorflow-GPU OOM issue after several epochs
- TensorFlow - Difference between tf.keras.layers.Layer vs tf.keras.Model
- Tensorboard/tensorflow with s3 logdir - curl returned error code 6
- tensorflow-deps - PackagesNotFoundError
- Tensorflow-Lite pretrained model does not work in Android demo
- TensorFlow-Slim data provider for in-memory dataset
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.