Introduction

return_sequences=True and stateful=True solve two different problems in an LSTM. The first controls what outputs the layer returns for each input sequence, while the second controls whether hidden state is carried from one batch to the next, so you should enable them based on architecture and data flow rather than treat them as a standard pair.

What return_sequences=True Does

By default, an LSTM returns only the output from the last time step. With return_sequences=True, it returns one output vector for every time step.

Without it:

• input shape: (batch, time, features)
• output shape: (batch, units)

With it:

• input shape: (batch, time, features)
• output shape: (batch, time, units)

This matters when the next layer still needs a sequence rather than a single summary vector.

1import tensorflow as tf
2
3x = tf.random.normal([4, 10, 8])
4layer = tf.keras.layers.LSTM(16, return_sequences=True)
5y = layer(x)
6print(y.shape)

When To Use return_sequences=True

Use it when:

• you are stacking another recurrent layer on top
• you need one prediction per time step
• you are building sequence-to-sequence or tagging models

For example, stacking LSTMs requires the lower layer to return the full sequence.

1model = tf.keras.Sequential([
2    tf.keras.layers.LSTM(32, return_sequences=True, input_shape=(20, 8)),
3    tf.keras.layers.LSTM(16),
4    tf.keras.layers.Dense(1)
5])

Without return_sequences=True on the first LSTM, the second LSTM would receive a 2D tensor instead of a time sequence.

What stateful=True Does

stateful=True tells Keras to carry the final state of each sample in one batch into the initial state of the corresponding sample in the next batch.

That only makes sense when batches are arranged carefully so sample i in batch t+1 is the logical continuation of sample i in batch t.

layer = tf.keras.layers.LSTM(16, stateful=True, batch_input_shape=(2, 5, 3))

A stateful LSTM therefore imposes stricter requirements:

• fixed batch size
• consistent sample ordering across batches
• explicit state resets when sequence boundaries are reached

When stateful=True Is Useful

Stateful LSTMs are helpful when a long sequence is split across batches intentionally, such as:

• streaming sensor data
• chunked time-series forecasting
• online inference where state continuity matters

They are usually not appropriate for randomly shuffled mini-batches, because random ordering breaks the state continuity assumption.

These Flags Are Independent

A common misunderstanding is thinking these flags belong together. They do not.

Possible combinations:

• 'return_sequences=False, stateful=False: default many-to-one behavior'
• 'return_sequences=True, stateful=False: full sequence output, independent batches'
• 'return_sequences=False, stateful=True: final output only, but state carries across batches'
• 'return_sequences=True, stateful=True: full sequence output plus cross-batch continuity'

Choose each flag for its own reason.

Example With Manual State Reset

If you use stateful LSTMs, resetting state at the right moment matters.

1import tensorflow as tf
2
3model = tf.keras.Sequential([
4    tf.keras.layers.LSTM(8, stateful=True, batch_input_shape=(1, 4, 2)),
5    tf.keras.layers.Dense(1)
6])
7
8model.compile(optimizer="adam", loss="mse")
9model.reset_states()

Reset state between unrelated sequences. Otherwise, the model keeps context that no longer belongs to the next sample stream.

Common Pitfalls

The most common mistake is enabling stateful=True while still shuffling batches randomly. That causes state to flow between unrelated sequences.

Another mistake is setting return_sequences=True only because it sounds more informative, even when the next layer needs just one output vector.

A third issue is forgetting that stacked LSTMs need the earlier recurrent layer to return a sequence, not just the final step output.

Summary

• 'return_sequences=True returns one LSTM output per time step.'
• 'stateful=True preserves hidden state across batches for aligned sample streams.'
• The two flags solve different problems and should be chosen independently.
• Stacked recurrent layers often need return_sequences=True in lower layers.
• Stateful LSTMs require fixed batch handling and explicit state management.