Introduction

The parameter count of an LSTM layer comes from its gates, not from the sequence length. Once you know the input size and the number of hidden units, you can calculate the trainable parameter count with one compact formula and verify it against a framework implementation.

Start with the Four Internal Gates

A standard LSTM cell has four gate computations: input gate, forget gate, candidate cell update, and output gate. Each gate needs three trainable parts:

• weights from the current input
• recurrent weights from the previous hidden state
• a bias vector

If the input size is m and the hidden size is h, then one gate has:

• 'm * h input weights'
• 'h * h recurrent weights'
• 'h bias terms'

Because there are four gates, the total becomes:

4 * (m * h + h * h + h)

That is usually written as:

4h(m + h + 1)

Work Through a Concrete Example

Suppose your input vector has 5 features and the LSTM layer has 10 hidden units. Then the parameter count is:

4 * 10 * (5 + 10 + 1) = 640

So the layer has 640 trainable parameters.

You can also think about it gate by gate:

• input weights: 5 * 10 = 50
• recurrent weights: 10 * 10 = 100
• bias: 10
• one gate total: 160
• four gates total: 640

That slower breakdown is useful when you are checking framework summaries by hand.

Verify the Formula in Keras

A quick way to confirm the math is to create a tiny model and inspect the summary.

1import tensorflow as tf
2from tensorflow import keras
3
4model = keras.Sequential([
5    keras.layers.Input(shape=(20, 5)),
6    keras.layers.LSTM(10)
7])
8
9model.summary()

For that model, the LSTM layer receives input feature size 5 and hidden size 10, so the summary should show 640 parameters.

The sequence length 20 affects computation over time, but it does not change the parameter count because the same weights are reused at every time step.

Stacked and Bidirectional LSTMs Change the Count

For a stacked LSTM, each layer uses the previous layer output as its input size. If the first layer has hidden size h1 and the second has hidden size h2, then the second layer uses m = h1 if the first layer outputs the full sequence.

For a bidirectional LSTM, you usually double the parameter count of a single-direction layer because there is one LSTM moving forward and one moving backward.

1import tensorflow as tf
2from tensorflow import keras
3
4model = keras.Sequential([
5    keras.layers.Input(shape=(20, 5)),
6    keras.layers.Bidirectional(keras.layers.LSTM(10))
7])
8
9model.summary()

If one direction has 640 parameters, the bidirectional wrapper typically exposes 1280 for that layer.

Peepholes and Framework Variants Matter

The formula above assumes a standard LSTM without peephole connections. Some implementations add extra weights from the cell state into certain gates. If peepholes are present, the count changes.

Frameworks may also split or combine biases internally, but for a conventional Keras-style LSTM the standard formula is the right starting point.

Always check the exact layer variant before treating the formula as universal.

Sequence Length Does Not Add Parameters

This is the mistake people make most often. If your input shape changes from (10, 5) to (100, 5), the layer runs across more time steps, but it still uses the same gate weights at each step. Runtime and memory use change, but parameter count does not.

That distinction is important when estimating model size versus training cost.

Common Pitfalls

• Multiplying by sequence length even though LSTM weights are shared across time steps.
• Forgetting that each LSTM cell has four gates, not one.
• Ignoring recurrent weights and counting only input weights.
• Applying the standard formula to a variant that adds peepholes or other extra connections.
• Misreading stacked or bidirectional models and using the wrong input size for later layers.

Summary

• A standard LSTM layer has four gate computations.
• The parameter formula is 4h(m + h + 1).
• Sequence length affects computation, not parameter count.
• Stacked and bidirectional designs change the total in predictable ways.
• Verify the result against a framework summary when model architecture gets more complex.