tensorflow
reverse
reverse_sequence
neural_networks
machine_learning

What is the primary difference between the reverse and reverse_sequence in tensorflow?

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Introduction

tf.reverse and tf.reverse_sequence both reverse tensor values, but they solve different problems. tf.reverse reverses an entire axis uniformly, while tf.reverse_sequence reverses only the leading part of each sequence based on per-item lengths.

tf.reverse Reverses Whole Axes

Use tf.reverse when you want a plain axis reversal with no notion of sequence length or padding.

python
1import tensorflow as tf
2
3x = tf.constant([
4    [1, 2, 3, 4],
5    [5, 6, 7, 8]
6])
7
8result = tf.reverse(x, axis=[1])
9print(result.numpy())

Output:

text
[[4 3 2 1]
 [8 7 6 5]]

Every row is reversed completely along axis 1. TensorFlow does not care whether the values represent padded sequences, time steps, or anything else. It just flips the specified dimension.

tf.reverse_sequence Reverses Only Valid Prefixes

tf.reverse_sequence is sequence-aware. According to the TensorFlow API docs, it slices the input along the batch dimension and reverses only the first seq_lengths[i] elements along the sequence dimension for each batch item.

That makes it useful for padded batches where each row has a different true sequence length.

python
1import tensorflow as tf
2
3x = tf.constant([
4    [1, 2, 3, 0, 0],
5    [4, 5, 0, 0, 0],
6    [6, 7, 8, 9, 0]
7])
8
9lengths = tf.constant([3, 2, 4])
10
11result = tf.reverse_sequence(
12    x,
13    seq_lengths=lengths,
14    seq_axis=1,
15    batch_axis=0
16)
17
18print(result.numpy())

Output:

text
[[3 2 1 0 0]
 [5 4 0 0 0]
 [9 8 7 6 0]]

Notice what changed:

  • row 1 reversed only the first 3 values
  • row 2 reversed only the first 2 values
  • row 3 reversed only the first 4 values

The padded tail stayed in place.

The Core Difference

The practical distinction is simple:

  • 'tf.reverse treats the axis as one uniform block'
  • 'tf.reverse_sequence treats the batch as many separate sequences with different valid lengths'

If the tensor contains padded minibatches for sequence models, tf.reverse_sequence is often the correct tool. If you just want to flip an axis, use tf.reverse.

Why reverse_sequence Exists

Sequence models frequently batch variable-length data by padding shorter examples. Reversing the full time axis would move padding to the front and corrupt the intended sequence content.

For example, suppose the second row below has only two real tokens:

text
[4, 5, 0, 0, 0]

Using tf.reverse would produce:

text
[0, 0, 0, 5, 4]

That is usually wrong for sequence logic, because the padding is now leading the sequence. tf.reverse_sequence avoids that by reversing only the valid prefix.

Higher-Dimensional Example

The same idea works for tensors beyond rank 2. You just need to tell TensorFlow which dimension is the batch and which dimension is the sequence.

python
1import tensorflow as tf
2
3x = tf.constant([
4    [[1], [2], [3], [0]],
5    [[4], [5], [0], [0]]
6], dtype=tf.int32)
7
8lengths = tf.constant([3, 2])
9
10result = tf.reverse_sequence(
11    x,
12    seq_lengths=lengths,
13    seq_axis=1,
14    batch_axis=0
15)
16
17print(result.numpy())

The reversal happens along seq_axis, separately for each item along batch_axis. The trailing feature dimension is left alone.

Choosing the Right Operation

Use tf.reverse when:

  • all elements along the chosen axis should flip
  • there is no per-example valid length
  • you are manipulating tensor layout rather than padded sequences

Use tf.reverse_sequence when:

  • examples in a batch have different sequence lengths
  • padding must stay at the end
  • you need per-example partial reversal

That is the mental model that prevents most mistakes.

Common Pitfalls

The most common mistake is using tf.reverse on padded data and then wondering why zeros or padding tokens moved to the front.

Another mistake is passing the wrong seq_axis or batch_axis to tf.reverse_sequence. The function is powerful, but only if those dimensions reflect the real layout of the tensor.

Developers also sometimes provide seq_lengths with the wrong size. It must match the size of the batch dimension, and each value must be less than or equal to the length of the sequence axis.

Finally, do not use tf.reverse_sequence when you do not actually have variable-length sequences. The plain tf.reverse operation is simpler when a full-axis reversal is what you mean.

Summary

  • 'tf.reverse reverses complete axes with no awareness of sequence lengths.'
  • 'tf.reverse_sequence reverses only the valid prefix of each sequence in a batch.'
  • Use tf.reverse_sequence for padded variable-length sequence data.
  • Use tf.reverse for ordinary full-axis flips.
  • The key parameters in tf.reverse_sequence are seq_lengths, seq_axis, and batch_axis.

Related reading
Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design