Introduction

tf.gradients() is the TensorFlow graph-mode API for asking, "How does this output change when an input changes?" It is the core building block behind backpropagation in TensorFlow 1.x style code and still appears in tf.compat.v1 when maintaining older models.

What tf.gradients() Actually Returns

Unlike eager execution APIs, tf.gradients() does not immediately calculate numbers. It builds new nodes in the computation graph that represent symbolic derivatives. When a session later runs those nodes, TensorFlow walks backward from the target tensor, applies the chain rule, and accumulates partial derivatives for every path that connects the target to the requested input.

That behavior matters because the function expects a graph of tensor operations, not plain Python math. If there is no path from ys back to one of the tensors in xs, TensorFlow usually returns None for that entry unless you ask for zeros on unconnected gradients.

A Small Graph-Mode Example

The example below uses tf.compat.v1 so it works in current TensorFlow installations while still demonstrating the original tf.gradients() model.

1import tensorflow as tf
2
3tf.compat.v1.disable_eager_execution()
4
5x = tf.compat.v1.placeholder(tf.float32, name="x")
6y = x * x + 3 * x
7
8dy_dx = tf.gradients(y, x)[0]
9
10with tf.compat.v1.Session() as session:
11    result = session.run(dy_dx, feed_dict={x: 2.0})
12    print(result)  # 7.0

The derivative of x * x + 3 * x is 2 * x + 3, so at x = 2 the result is 7. The important detail is that TensorFlow did not use numerical approximation. It differentiated the graph symbolically and produced another tensor.

Multiple Inputs and Gradient Aggregation

If an output depends on several tensors, tf.gradients() returns one gradient tensor per requested input. TensorFlow sums contributions from all graph paths that reach the same variable. That is why the API accepts a list for xs.

1import tensorflow as tf
2
3tf.compat.v1.disable_eager_execution()
4
5a = tf.compat.v1.placeholder(tf.float32, name="a")
6b = tf.compat.v1.placeholder(tf.float32, name="b")
7z = a * b + a * a
8
9grad_a, grad_b = tf.gradients(z, [a, b])
10
11with tf.compat.v1.Session() as session:
12    values = session.run([grad_a, grad_b], feed_dict={a: 4.0, b: 5.0})
13    print(values)  # [13.0, 4.0]

Here, z depends on a in two places, so the gradient with respect to a is the sum of both contributions. That is one reason automatic differentiation is so useful in neural networks: the framework tracks every branch and merge for you.

When to Use GradientTape Instead

In TensorFlow 2, eager execution is the default, and the preferred API is tf.GradientTape. The underlying idea is the same, but the programming model is different. GradientTape records operations as they run, while tf.gradients() assumes you already have a graph.

1import tensorflow as tf
2
3x = tf.Variable(2.0)
4
5with tf.GradientTape() as tape:
6    y = x * x + 3 * x
7
8print(tape.gradient(y, x).numpy())  # 7.0

If you are reading older code, understanding tf.gradients() helps you reason about training ops, custom losses, and manual optimization loops. For new code, GradientTape is usually easier to debug and compose.

Unconnected Gradients and Stopped Paths

One confusing case is when the target tensor does not depend on an input you asked about. Another is when tf.stop_gradient intentionally cuts the path. In both situations the gradient is not available from the graph.

1import tensorflow as tf
2
3tf.compat.v1.disable_eager_execution()
4
5x = tf.compat.v1.placeholder(tf.float32)
6y = tf.constant(10.0)
7
8grad = tf.gradients(
9    y,
10    x,
11    unconnected_gradients=tf.UnconnectedGradients.ZERO,
12)[0]
13
14with tf.compat.v1.Session() as session:
15    print(session.run(grad, feed_dict={x: 5.0}))  # 0.0

Using zero for unconnected gradients can simplify downstream code because you avoid checking for None, but it also makes it easier to miss a modeling mistake. Use it deliberately.

Common Pitfalls

• Mixing eager mode and graph mode causes confusing errors. tf.gradients() belongs to graph-style TensorFlow and is usually accessed through tf.compat.v1.
• Expecting an immediate numeric answer leads to confusion. The function returns symbolic tensors, so you still need a session in graph mode.
• Forgetting that some paths are disconnected can produce None gradients. Check whether the output truly depends on the input.
• Using non-differentiable operations, such as discrete indexing in the wrong place, can break the gradient chain.
• Calling the API in new TensorFlow 2 code makes maintenance harder. Prefer tf.GradientTape unless you are working with legacy graph code.

Summary

• 'tf.gradients() builds symbolic derivative nodes in a TensorFlow computation graph.'
• TensorFlow applies the chain rule backward from ys to every tensor listed in xs.
• Gradients from multiple graph paths are accumulated automatically.
• Unconnected inputs can return None or zero, depending on configuration.
• In modern TensorFlow, tf.GradientTape is the preferred API for new code.