Introduction

In TensorFlow, both constants and variables hold tensor data, but they serve different roles. A constant is an immutable value used as fixed input or configuration, while a variable is mutable state that TensorFlow can update during training.

Create a Constant

Use tf.constant when a value should not change during the life of the computation.

1import tensorflow as tf
2
3learning_rate = tf.constant(0.01, dtype=tf.float32)
4features = tf.constant([[1.0, 2.0], [3.0, 4.0]])
5
6print(learning_rate)
7print(features)

Constants are useful for:

• fixed scalar values
• hard-coded lookup tensors
• input examples in small demos
• configuration values inside a computation

They can participate in math operations normally, but you cannot assign a new value to them.

Create a Variable

Use tf.Variable when the value must change over time, especially during optimization.

1weights = tf.Variable([[0.5], [0.2]], dtype=tf.float32)
2bias = tf.Variable([0.0], dtype=tf.float32)
3
4print(weights)
5print(bias)

Variables are the natural choice for model parameters because training algorithms update them repeatedly.

Variables Can Be Updated

Unlike constants, variables support assignment methods:

1counter = tf.Variable(0)
2
3counter.assign(5)
4counter.assign_add(2)
5counter.assign_sub(1)
6
7print(counter.numpy())  # 6

That mutability is the core distinction. If your code needs state that changes, it should usually be a variable.

Variables in a Training Step

TensorFlow tracks variables during gradient-based optimization. A small example shows why variables matter for machine learning:

1import tensorflow as tf
2
3x = tf.constant([[1.0], [2.0], [3.0]])
4y = tf.constant([[2.0], [4.0], [6.0]])
5
6w = tf.Variable([[0.1]])
7b = tf.Variable([0.0])
8
9optimizer = tf.keras.optimizers.SGD(learning_rate=0.1)
10
11for _ in range(5):
12    with tf.GradientTape() as tape:
13        predictions = tf.matmul(x, w) + b
14        loss = tf.reduce_mean((predictions - y) ** 2)
15
16    grads = tape.gradient(loss, [w, b])
17    optimizer.apply_gradients(zip(grads, [w, b]))
18
19print("w:", w.numpy())
20print("b:", b.numpy())

Here, w and b must be variables because the optimizer updates them on every step. If they were constants, training could not adjust them.

Constants Still Matter in Models

Constants are not just for toy scripts. They are often useful alongside variables for:

• fixed masks
• scaling factors
• constant embeddings or lookup tables that should not train
• static input tensors in tests

For example:

1scale = tf.constant(255.0)
2pixels = tf.constant([[0.0, 127.5, 255.0]])
3normalized = pixels / scale
4
5print(normalized.numpy())

The important point is that constants can be part of a model pipeline even though they are not trainable state.

Variable Tracking in Keras Layers

In tf.keras, variables are usually created inside layers and models. TensorFlow automatically tracks them as trainable or non-trainable weights.

1layer = tf.keras.layers.Dense(4)
2output = layer(tf.ones((1, 3)))
3
4print(layer.trainable_variables)

That tracking is built on top of tf.Variable. So even if you do not instantiate variables directly all the time, they are still the underlying state objects that hold learned parameters.

Common Pitfalls

The biggest mistake is using tf.constant for something that needs to change during training. Optimizers only update variables, not immutable tensors.

Another common issue is assuming variables and constants are separate data types in every mathematical sense. Both behave like tensors in computation, but only variables carry mutable state and assignment semantics.

People also forget that variables need an initial value. TensorFlow must know the starting tensor shape and dtype when the variable is created.

Finally, do not confuse "non-trainable" with "constant". A non-trainable variable can still be assigned manually, while a constant cannot be changed at all.

Summary

• Use tf.constant for fixed tensor values that should not change.
• Use tf.Variable for mutable state, especially model parameters.
• Variables support assign, assign_add, and gradient-based optimization.
• Constants still participate in normal TensorFlow computations.
• In machine learning code, learned weights are variables, not constants.