Introduction

When you run inference with a Keras RNN, LSTM, or GRU, the layer must start from some internal state. If you do not provide one explicitly, Keras uses the cell's default initial state, which for standard built-in recurrent layers is a zero state unless you are working with a stateful layer that is reusing state from an earlier batch.

Default Behavior During Inference

The first important point is that inference does not get a special hidden-state rule that is fundamentally different from training. Keras recurrent layers still need an initial state on the first timestep, and if you do not pass one with initial_state, the layer creates its default initial state for you.

For common built-in layers:

• 'SimpleRNN starts from zeros.'
• 'GRU starts from zeros.'
• 'LSTM starts both the hidden state and the cell state from zeros.'

That means a normal non-stateful LSTM call behaves as if you had provided tensors of zeros with the correct shape. This is usually what people want for isolated sequences such as sentence classification, sequence tagging, or one-shot forecasting windows.

Seeing the Behavior in Code

The example below shows two inference calls: one with Keras defaults, and one with a custom initial state. The outputs differ because the starting memory differs.

1import tensorflow as tf
2
3sequence = tf.constant([
4    [[1.0], [2.0], [3.0]]
5], dtype=tf.float32)
6
7lstm = tf.keras.layers.LSTM(4, return_state=True)
8
9output_default, h_default, c_default = lstm(sequence, training=False)
10print("Default final hidden state:")
11print(h_default.numpy())
12
13custom_h = tf.ones((1, 4), dtype=tf.float32)
14custom_c = tf.ones((1, 4), dtype=tf.float32)
15
16output_custom, h_custom, c_custom = lstm(
17    sequence,
18    initial_state=[custom_h, custom_c],
19    training=False,
20)
21
22print("Custom final hidden state:")
23print(h_custom.numpy())

The first call uses zero initial states. The second call uses all-ones tensors. Keras does not "remember" the first call here, because the layer is not stateful and no previous state is passed in.

When State Carries Across Calls

If you create a recurrent layer with stateful=True, the story changes. In that mode, Keras keeps the last state from one batch and uses it as the initial state for the next batch with the same sample ordering. That can be useful for streaming or chunked sequence processing, but it also creates bookkeeping requirements.

With stateful inference, the order of samples matters. Batch element 0 in the next call is assumed to continue batch element 0 from the previous call. If the ordering changes, the reused state no longer matches the data.

Keras also lets you reset the stored state explicitly:

1import tensorflow as tf
2
3layer = tf.keras.layers.LSTM(2, stateful=True, return_sequences=True)
4
5x1 = tf.random.normal((1, 3, 5))
6x2 = tf.random.normal((1, 3, 5))
7
8_ = layer(x1, training=False)
9_ = layer(x2, training=False)
10
11layer.reset_states()

After reset_states(), Keras falls back to the original initial state behavior. If no explicit state was set, that means zeros again.

Why Custom Initial State Is Useful

Passing initial_state is important when the sequence logically continues from earlier context that was computed elsewhere. A classic example is encoder-decoder modeling. The encoder processes one sequence, produces final states, and those states initialize the decoder.

It is also useful in rolling inference where long sequences are split into chunks. Instead of forcing each chunk to start from zeros, you can feed the final state from chunk one into chunk two and preserve continuity.

The crucial point is that Keras will not infer this intent automatically for ordinary non-stateful layers. If you need continuity, you must either:

1. use stateful=True and manage batches carefully, or
2. pass initial_state explicitly on each call.

Common Pitfalls

The most common misunderstanding is assuming that an LSTM remembers previous predict() calls by default. It does not. In the usual non-stateful setup, each call starts fresh from zero state unless you pass state in manually.

Another pitfall is mixing up training mode and inference mode with state initialization. The training flag mostly affects behaviors such as dropout. It does not change the basic rule that recurrent layers need an initial state and will use zeros by default when none is supplied.

Stateful layers introduce their own hazards. If you shuffle data or change batch alignment, the carried-over states become attached to the wrong examples. That can quietly damage results.

Finally, the shapes of the provided state tensors must match the layer's units and batch size. For an LSTM, you need both hidden state and cell state, not just one tensor.

Summary

• Keras recurrent layers use default initial states when you do not pass initial_state.
• For built-in RNN, GRU, and LSTM layers, those defaults are zero states.
• Non-stateful inference starts fresh on each call unless you explicitly provide previous state.
• Stateful layers can carry state across batches, but only when batch ordering is controlled.
• Use initial_state or stateful=True when sequence continuity matters.