Introduction

An LSTM is a reasonable baseline for binary classification when each example is a sequence, such as a sentence, a clickstream, or a sensor window. The model reads the sequence order, compresses useful temporal information, and outputs a probability for one of two classes.

The structure is simple, but most failures come from input preparation rather than the LSTM layer itself. Sequence padding, masking, label shape, and the final activation all need to line up.

What the Model Should Look Like

For binary classification, the usual Keras pattern is:

• sequence input
• one or more recurrent layers
• a final dense layer with one unit
• sigmoid activation
• 'binary_crossentropy loss'

If the inputs are token IDs, an Embedding layer usually comes first. If the inputs are already numeric feature vectors per timestep, you can feed them directly into the LSTM.

A Runnable Example with Padded Integer Sequences

1import numpy as np
2import tensorflow as tf
3
4X = np.array([
5    [4, 7, 2, 0, 0],
6    [8, 1, 9, 3, 2],
7    [5, 6, 0, 0, 0],
8    [9, 9, 8, 7, 1],
9], dtype="int32")
10
11y = np.array([0, 1, 0, 1], dtype="float32")
12
13model = tf.keras.Sequential([
14    tf.keras.layers.Embedding(input_dim=20, output_dim=8, mask_zero=True),
15    tf.keras.layers.LSTM(16),
16    tf.keras.layers.Dense(1, activation="sigmoid"),
17])
18
19model.compile(
20    optimizer="adam",
21    loss="binary_crossentropy",
22    metrics=["accuracy"],
23)
24
25model.fit(X, y, epochs=5, verbose=0)
26print(model.predict(X, verbose=0))

This example uses mask_zero=True, which tells Keras to ignore the padded zeros when the LSTM reads the sequence.

Input Shape Rules

LSTM layers expect a three-dimensional tensor:

• batch size
• timesteps
• features

When you use an Embedding layer, Keras converts integer token IDs shaped like (batch, timesteps) into embedded vectors shaped like (batch, timesteps, embedding_dim). If you skip embeddings and provide numeric features directly, your input should already be three-dimensional.

For binary labels, the target can usually be shaped as either (batch,) or (batch, 1) as long as the final layer has one sigmoid unit.

When To Use return_sequences

If there is only one LSTM layer and you want one final classification per sequence, leave return_sequences=False, which is the default. That returns only the last output state.

If you stack LSTM layers, all but the last recurrent layer usually need:

tf.keras.layers.LSTM(32, return_sequences=True)

Without that setting, the next recurrent layer will not receive a full sequence.

Practical Improvements

A basic LSTM often benefits from:

• dropout on the recurrent stack
• validation splitting and early stopping
• class weighting if labels are imbalanced
• proper tokenization and vocabulary control

Example of adding regularization:

1model = tf.keras.Sequential([
2    tf.keras.layers.Embedding(input_dim=2000, output_dim=64, mask_zero=True),
3    tf.keras.layers.LSTM(32, dropout=0.2, recurrent_dropout=0.2),
4    tf.keras.layers.Dense(1, activation="sigmoid"),
5])

Use this only after the base pipeline is correct. Regularization will not fix broken shapes or mislabeled targets.

Common Pitfalls

• Using two output units with sigmoid for a binary problem when one output unit is enough.
• Forgetting to pad sequences to a consistent length for batch training.
• Padding with zeros but not enabling masking when zeros are only placeholders.
• Feeding labels with the wrong shape or datatype for binary_crossentropy.
• Stacking LSTM layers without return_sequences=True on intermediate layers.

Summary

• A binary sequence classifier in Keras usually ends with one sigmoid output and binary_crossentropy.
• 'Embedding plus LSTM is a standard pattern for tokenized sequences.'
• Padding and masking are critical for variable-length inputs.
• Shape mismatches are a more common bug than the recurrent layer itself.
• Start with a small working model, then add regularization or deeper sequence stacks.