Introduction

Backpropagation in TensorFlow is usually something you trigger, not something you code from scratch. You define a forward pass, compute a loss, record operations with tf.GradientTape, and let TensorFlow differentiate that computation graph to produce gradients for each trainable variable.

The Core Training Loop

At a high level, one training step has four parts:

• run the model on input data
• compute a scalar loss
• differentiate that loss with respect to trainable variables
• apply the gradients with an optimizer

TensorFlow handles the derivative bookkeeping automatically. You do not manually write chain-rule expressions for each layer.

Here is a complete minimal example:

1import tensorflow as tf
2
3tf.random.set_seed(7)
4
5x = tf.constant([[1.0], [2.0], [3.0], [4.0]])
6y = tf.constant([[3.0], [5.0], [7.0], [9.0]])
7
8model = tf.keras.Sequential([
9    tf.keras.layers.Dense(1)
10])
11
12optimizer = tf.keras.optimizers.SGD(learning_rate=0.05)
13loss_fn = tf.keras.losses.MeanSquaredError()
14
15for step in range(200):
16    with tf.GradientTape() as tape:
17        predictions = model(x, training=True)
18        loss = loss_fn(y, predictions)
19
20    gradients = tape.gradient(loss, model.trainable_variables)
21    optimizer.apply_gradients(zip(gradients, model.trainable_variables))
22
23print("loss:", float(loss))
24print("prediction for 5:", float(model(tf.constant([[5.0]]))))

This model learns the mapping y = 2x + 1. The important line is tape.gradient(loss, model.trainable_variables), which is where backpropagation happens.

What GradientTape Is Recording

Inside the with tf.GradientTape() block, TensorFlow records the operations that transform inputs into predictions and predictions into loss. That recorded graph is then traversed backward to compute gradients.

In other words:

• the forward pass builds values
• the backward pass computes how much each variable contributed to the loss

For a dense layer, TensorFlow knows how matrix multiplication, addition, and the activation function behave under differentiation. Those low-level derivative rules are composed automatically through the network.

You can inspect the gradients directly:

1import tensorflow as tf
2
3model = tf.keras.Sequential([tf.keras.layers.Dense(1)])
4x = tf.constant([[1.0], [2.0]])
5y = tf.constant([[2.0], [4.0]])
6
7with tf.GradientTape() as tape:
8    predictions = model(x)
9    loss = tf.reduce_mean(tf.square(y - predictions))
10
11gradients = tape.gradient(loss, model.trainable_variables)
12
13for variable, gradient in zip(model.trainable_variables, gradients):
14    print(variable.name)
15    print(gradient.numpy())

That is useful for debugging when training is unstable or gradients are unexpectedly None.

Backpropagation in Keras fit

If you use model.compile() and model.fit(), the same basic logic still exists under the hood. Keras builds the training step for you and uses backpropagation automatically.

1import tensorflow as tf
2
3x = tf.constant([[1.0], [2.0], [3.0], [4.0]])
4y = tf.constant([[3.0], [5.0], [7.0], [9.0]])
5
6model = tf.keras.Sequential([
7    tf.keras.layers.Dense(8, activation="relu"),
8    tf.keras.layers.Dense(1)
9])
10
11model.compile(
12    optimizer="adam",
13    loss="mse"
14)
15
16model.fit(x, y, epochs=100, verbose=0)
17print(model(tf.constant([[5.0]])).numpy())

The difference is not whether backpropagation exists. The difference is whether you want direct control over the training step.

When Gradients Become None

TensorFlow can only differentiate through recorded, differentiable operations. If you break the computation path, the gradient for a variable may be None.

Common reasons include:

• using NumPy operations inside the taped section
• converting tensors to Python numbers too early
• asking for gradients with respect to variables that were not involved in the loss

When that happens, inspect the forward pass and make sure the loss truly depends on the variables you expect to train.

Common Pitfalls

The most common mistake is assuming TensorFlow magically updates weights without a loss function. Backpropagation needs a scalar objective, so define one clearly.

Another issue is doing work outside the GradientTape block that should be part of the differentiable computation. If the operation is not recorded, TensorFlow cannot backpropagate through it.

Developers also sometimes mix NumPy and TensorFlow carelessly in the training step. NumPy is fine for data preparation, but once you are computing the loss, stay inside TensorFlow ops if you want gradients.

Finally, remember that exploding or vanishing gradients are optimization problems, not API bugs. The training loop may be correct while the model architecture or learning rate is not.

Summary

• TensorFlow backpropagation is driven by tf.GradientTape.
• You write the forward pass and loss; TensorFlow computes the gradients.
• 'optimizer.apply_gradients(...) uses those gradients to update variables.'
• 'model.fit() performs the same idea automatically at a higher level.'
• If gradients are None, the recorded computation path is usually broken.