Introduction

In TensorFlow 1 style graph training, both optimizer.compute_gradients and tf.gradients can produce derivatives, but they solve different layers of the training problem. One API is optimizer-aware and tied to update logic, while the other is a low-level graph differentiation primitive. Knowing the distinction helps when you need gradient clipping, custom updates, or debugging gradient flow.

What tf.gradients Does

tf.gradients computes symbolic derivatives of one tensor with respect to one or more tensors. It does not know anything about optimizers, learning rates, slot variables, or parameter updates. It simply returns gradient tensors that you can inspect or use in custom math.

This makes tf.gradients useful when you want full control over downstream behavior. For example, you might combine gradients from multiple losses, apply manual scaling, or inspect gradient norms before update steps.

Example with tf.compat.v1:

1import tensorflow as tf
2
3tf.compat.v1.disable_eager_execution()
4
5x = tf.compat.v1.placeholder(tf.float32, shape=[None, 1])
6y = tf.compat.v1.placeholder(tf.float32, shape=[None, 1])
7
8w = tf.Variable([[2.0]], dtype=tf.float32)
9b = tf.Variable([0.5], dtype=tf.float32)
10
11pred = tf.matmul(x, w) + b
12loss = tf.reduce_mean(tf.square(pred - y))
13
14grads = tf.gradients(ys=loss, xs=[w, b])
15
16with tf.compat.v1.Session() as sess:
17    sess.run(tf.compat.v1.global_variables_initializer())
18    g_w, g_b = sess.run(
19        grads,
20        feed_dict={x: [[1.0], [2.0]], y: [[3.0], [5.0]]}
21    )
22    print("grad w:", g_w)
23    print("grad b:", g_b)

You get gradient values, but no variable updates happen unless you define and run update ops yourself.

What optimizer.compute_gradients Adds

optimizer.compute_gradients is part of the optimizer workflow. It takes a loss, computes gradients with respect to trainable variables, and returns gradient and variable pairs. Those pairs are designed for optimizer.apply_gradients.

This method is convenient because it aligns gradient computation with optimizer internals and variable selection rules. It is also the standard place to insert gradient transformations before updates.

1import tensorflow as tf
2
3tf.compat.v1.disable_eager_execution()
4
5x = tf.compat.v1.placeholder(tf.float32, shape=[None, 1])
6y = tf.compat.v1.placeholder(tf.float32, shape=[None, 1])
7
8w = tf.Variable([[0.0]], dtype=tf.float32)
9b = tf.Variable([0.0], dtype=tf.float32)
10
11pred = tf.matmul(x, w) + b
12loss = tf.reduce_mean(tf.square(pred - y))
13
14opt = tf.compat.v1.train.AdamOptimizer(learning_rate=0.1)
15grads_and_vars = opt.compute_gradients(loss, var_list=[w, b])
16
17clipped = []
18for g, v in grads_and_vars:
19    if g is not None:
20        g = tf.clip_by_norm(g, clip_norm=1.0)
21    clipped.append((g, v))
22
23train_op = opt.apply_gradients(clipped)
24
25with tf.compat.v1.Session() as sess:
26    sess.run(tf.compat.v1.global_variables_initializer())
27    for _ in range(50):
28        sess.run(train_op, feed_dict={x: [[1.0], [2.0]], y: [[3.0], [5.0]]})
29    w_val, b_val = sess.run([w, b])
30    print("trained w:", w_val, "trained b:", b_val)

This pattern gives a clean hook for clipping, scaling, or filtering gradients before applying updates.

Practical Differences That Matter

Key behavioral differences:

• tf.gradients returns only gradients.
• optimizer.compute_gradients returns gradient and variable pairs.
• tf.gradients is optimizer-agnostic.
• optimizer.compute_gradients is built for a specific optimizer and its update flow.
• tf.gradients is better for custom derivative pipelines.
• optimizer.compute_gradients is better for normal training loops with optional gradient edits.

Another detail is variable scope and default variable collection behavior. Optimizers typically target trainable variables unless you pass var_list, while raw tf.gradients requires explicit xs targets. This affects safety in larger models where some variables should remain frozen.

TensorFlow 2 Perspective

In TensorFlow 2, tf.GradientTape replaces most direct tf.gradients usage for eager mode. The conceptual split still exists:

• tape gives raw gradients
• optimizer applies updates

So the TF1 distinction maps naturally to modern code: gradient computation versus optimizer-managed update.

When to Choose Which

Use optimizer.compute_gradients when:

• you have a standard optimizer-driven training step
• you want easy gradient clipping or logging before apply
• you want clear pairing between each gradient and variable

Use tf.gradients when:

• you need custom gradient algebra not tied to one optimizer
• you are building meta-objectives with multiple losses
• you are debugging graph derivative structure

In large codebases, mixing both is normal. Many teams compute custom objectives with tf.gradients, then route the result through optimizer logic for consistent updates.

Common Pitfalls

• Assuming tf.gradients performs updates by itself. It does not.
• Forgetting to run apply_gradients after compute_gradients.
• Ignoring None gradients in pair lists, which can break custom gradient transforms.
• Clipping gradients after apply instead of before apply.
• Mixing TF1 graph patterns and TF2 eager code without compatibility boundaries.

Summary

• tf.gradients is a low-level symbolic differentiation API.
• optimizer.compute_gradients is an optimizer-integrated gradient pipeline step.
• Use raw gradients for custom derivative logic and analysis.
• Use optimizer pairs for controlled training updates with clipping and filtering.
• In TF2, the same separation exists through GradientTape and optimizer update calls.