Introduction

In TensorFlow 1 graph mode, tf.gradients is the core API for symbolic reverse-mode differentiation. It does not immediately compute numbers. Instead, it adds gradient operations to the graph so that when the graph runs in a session, TensorFlow can backpropagate derivatives from one set of tensors to another.

What tf.gradients Returns

The basic pattern is tf.gradients(ys, xs). Conceptually, this asks TensorFlow to compute the derivative of ys with respect to xs.

In graph mode, the function returns tensors that represent those gradients, not final Python values.

1import tensorflow as tf
2
3tf.compat.v1.disable_eager_execution()
4
5x = tf.compat.v1.placeholder(tf.float32)
6y = x * x + 3.0 * x
7
8dy_dx = tf.gradients(y, x)[0]
9
10with tf.compat.v1.Session() as sess:
11    value = sess.run(dy_dx, feed_dict={x: 2.0})
12    print(value)

At x = 2, the derivative is 2 * x + 3, so the result is 7.

Why It Works

TensorFlow records the graph of operations that produced y. When you call tf.gradients, it walks that graph backward and applies gradient rules for each operation, combining the local derivatives with the chain rule.

That is why the function is so central to training: optimizers need derivatives of the loss with respect to model parameters, and tf.gradients builds exactly that symbolic backpropagation path.

Multiple Inputs And Outputs

The API also supports lists. If ys contains multiple targets, TensorFlow sums their gradient contributions with respect to each variable in xs.

1w = tf.Variable(2.0)
2x = tf.constant(3.0)
3y1 = w * x
4y2 = w * w
5
6grads = tf.gradients([y1, y2], [w])

Here the result is the gradient of y1 + y2 with respect to w, not two separate unrelated runs. This behavior is useful because neural network losses are often built from many pieces that ultimately contribute to one total objective.

The Role Of grad_ys

The optional grad_ys argument lets you supply upstream gradients manually. You can think of it as the gradient already flowing into ys from later computation.

This matters when ys is not a scalar. For scalar losses, TensorFlow usually behaves as though the upstream gradient is 1. For vector outputs, grad_ys lets you weight or combine the components intentionally.

What Happens With Unconnected Paths

If there is no differentiable path from ys to a tensor in xs, the gradient may be None or zero depending on configuration. That behavior is important when debugging models, because a None gradient often means the graph was disconnected or an operation blocked gradient flow.

Operations such as tf.stop_gradient intentionally cut that path. Non-differentiable operations can also lead to missing gradients.

Another useful detail is that gradient aggregation happens naturally when the same variable influences the target through multiple graph paths. TensorFlow adds those contributions together, which is exactly what backpropagation requires. This is why a parameter shared in several parts of a graph still receives one combined gradient tensor.

Common Pitfalls

One common mistake is expecting tf.gradients to return a numeric value immediately. In TensorFlow 1 it returns symbolic tensors that must still be evaluated in a session.

Another mistake is forgetting that multiple ys values are aggregated. The result is the gradient of their sum with respect to xs, not a separate gradient object for each output by default.

A third issue is ignoring None gradients. They usually signal a disconnected graph, an unsupported derivative, or an intentional gradient stop.

Summary

• In TensorFlow 1, tf.gradients builds symbolic backpropagation ops in the graph.
• The returned objects are tensors that you evaluate later in a session.
• Gradients are computed by reverse-mode autodiff using the chain rule.
• If a gradient is None, check for disconnected graph paths or non-differentiable operations.