Introduction

TensorFlow variables hold mutable state, so it is common to ask for both the value before an update and the value after it. This comes up in custom training loops, debugging, metric collection, and optimizer experiments.

Snapshot the Old Value Before You Update

In TensorFlow 2, tf.Variable is mutable, which means the variable object itself keeps changing over time. If you want the previous value, do not keep another reference to the same variable object. Instead, create a tensor snapshot before calling assign, assign_add, or assign_sub.

1import tensorflow as tf
2
3counter = tf.Variable(10.0, name="counter")
4
5old_value = tf.identity(counter)
6counter.assign_add(2.5)
7new_value = tf.identity(counter)
8
9print("old:", old_value.numpy())
10print("new:", new_value.numpy())

tf.identity(counter) reads the current value and returns a tensor snapshot. After the assignment, new_value reflects the updated state while old_value still contains the pre-update number.

If you write old_value = counter, both names refer to the same mutable variable object. That does not preserve the earlier state.

Using Old and New Values Inside tf.function

The same pattern works inside compiled TensorFlow functions. Read the variable first, then update it, then read it again if you need both results.

1import tensorflow as tf
2
3weight = tf.Variable(3.0)
4
5@tf.function
6def update_weight(delta):
7    before = tf.identity(weight)
8    weight.assign_add(delta)
9    after = tf.identity(weight)
10    change = after - before
11    return before, after, change
12
13before, after, change = update_weight(tf.constant(0.75))
14tf.print("before =", before, "after =", after, "change =", change)

This is the basic recipe for any custom state transition:

1. Read the current value into a tensor
2. Perform the assignment
3. Read the new value if needed
4. Compute any derived quantity from the two snapshots

Because both before and after are tensors, they are safe to return from tf.function and log later.

Example in a Custom Training Step

The pattern becomes especially useful when you want to monitor how much a parameter changes after an optimization step.

1import tensorflow as tf
2
3w = tf.Variable(5.0)
4optimizer = tf.keras.optimizers.SGD(learning_rate=0.1)
5
6def loss_fn():
7    return (w - 1.0) ** 2
8
9@tf.function
10def train_step():
11    with tf.GradientTape() as tape:
12        loss = loss_fn()
13
14    old_w = tf.identity(w)
15    gradient = tape.gradient(loss, [w])
16    optimizer.apply_gradients(zip(gradient, [w]))
17    new_w = tf.identity(w)
18
19    return {
20        "loss": loss,
21        "old_w": old_w,
22        "new_w": new_w,
23        "delta": new_w - old_w,
24    }
25
26result = train_step()
27for name, value in result.items():
28    tf.print(name, "=", value)

This example captures the parameter before the optimizer mutates it, then reads it again after the update. That makes it easy to inspect the true step size and verify that your learning rate behaves as expected.

When You Need More Than One Previous Value

If you need a longer history, store explicit snapshots in a Python list or a TensorFlow data structure. Variables only keep the current state. TensorFlow will not automatically remember earlier versions for you.

For example, a debugging callback might append tf.identity(variable) every few steps. That approach is more predictable than trying to recover previous values after the mutation has already happened.

Common Pitfalls

• 'old_value = variable does not copy the current value. It only creates another reference to the same mutable variable.'
• Calling .numpy() too early can move data out of TensorFlow execution flow and is usually unnecessary inside tf.function.
• If an optimizer updates several variables, snapshot each one before apply_gradients if you need reliable before-and-after comparisons.
• Remember that variables store current state only. TensorFlow does not keep built-in history for previous assignments.

Summary

• Use tf.identity(variable) or another tensor read to snapshot the old value before mutation.
• Update the variable with assign or an optimizer step, then read it again for the new value.
• The same pattern works in eager mode and inside tf.function.
• Never rely on a second Python reference to preserve an earlier variable state.