Introduction

In Keras, an LSTM has two state tensors: the hidden state h and the cell state c. If you do nothing, Keras initializes both to zeros. That default is correct for many models, but you can also provide an explicit initial state when you need more control.

The Default Behavior

For a regular tf.keras.layers.LSTM, the initial hidden and cell states are zero tensors with the right batch size and unit count. This is what happens in ordinary sequence models:

1import tensorflow as tf
2
3lstm = tf.keras.layers.LSTM(8)
4x = tf.random.normal((4, 10, 6))
5y = lstm(x)
6
7print(y.shape)

You do not have to initialize anything manually unless your task specifically depends on nonzero or carried-over state.

That is why many introductory Keras examples never mention hidden-state initialization at all: the default zero state is already built into the layer behavior.

Passing an Explicit Initial State

If you want to control the starting state, pass initial_state when calling the layer. The LSTM expects:

• one tensor for hidden state h
• one tensor for cell state c

Example:

1import tensorflow as tf
2
3lstm = tf.keras.layers.LSTM(8, return_state=True)
4x = tf.random.normal((2, 5, 3))
5
6h0 = tf.zeros((2, 8))
7c0 = tf.ones((2, 8))
8
9output, h, c = lstm(x, initial_state=[h0, c0])
10
11print(output.shape)
12print(h.shape, c.shape)

The batch dimension and unit dimension of the initial state must match the LSTM configuration and the current input batch.

Stateful LSTMs Are Different

If you create the layer with stateful=True, Keras carries state across batches instead of resetting it automatically every call:

lstm = tf.keras.layers.LSTM(8, stateful=True)

This is useful for streaming or chunked sequence processing, but it adds constraints:

• batch size must stay fixed
• you must reset state deliberately when sequence boundaries matter

You can reset with:

lstm.reset_states()

That is often more important than custom initialization itself.

Using Encoder State in Sequence Models

One common advanced use case is encoder-decoder modeling. The encoder produces a final hidden and cell state, and the decoder starts from those states:

1encoder = tf.keras.layers.LSTM(8, return_state=True)
2decoder = tf.keras.layers.LSTM(8)
3
4x1 = tf.random.normal((2, 5, 3))
5x2 = tf.random.normal((2, 4, 3))
6
7_, h, c = encoder(x1)
8decoded = decoder(x2, initial_state=[h, c])
9
10print(decoded.shape)

This is a real reason to initialize LSTM state manually, and it shows why the API accepts both h and c.

It also illustrates that manual initialization is usually about information flow between model components, not about randomizing the start of training.

When Manual Initialization Helps

Manual state initialization is useful when:

• carrying context from one model component to another
• continuing sequence processing across chunks
• injecting learned or computed initial conditions

It is not required just to "help convergence" in ordinary models. For many tasks, zero initialization is the standard and correct choice.

Common Pitfalls

The biggest mistake is passing only one state tensor. Keras LSTM expects both hidden state and cell state.

Another mistake is forgetting batch-size alignment. If the input batch has shape (32, ...), the initial states must also start with batch size 32.

A third issue is using stateful=True without resetting state between logically separate sequences, which leaks information across examples.

Summary

• Keras LSTMs default to zero-initialized hidden and cell states.
• Pass initial_state=[h0, c0] when you need explicit control.
• Stateful LSTMs persist state across batches and require deliberate resets.
• Manual initialization is especially useful for encoder-decoder or streaming setups.
• In many ordinary models, the default zero state is already the right initialization.