Introduction

Sequence-to-sequence models fail for predictable reasons: mismatched tensor shapes, inconsistent vocabularies, or decoder inputs that do not line up with the target sequence. The architecture is more fragile than a simple classifier because the encoder, decoder, and training targets all have to agree on time-step layout.

Start With the Core Shape Contract

A training-time seq2seq model usually has two inputs:

• 'encoder_inputs for the source sequence'
• 'decoder_inputs for the target sequence shifted right by one step'

The model output must align with the unshifted target tokens. A minimal Keras example makes that contract visible.

1import numpy as np
2import tensorflow as tf
3
4vocab_size = 32
5embedding_dim = 16
6latent_dim = 32
7sequence_length = 6
8
9encoder_inputs = tf.keras.Input(shape=(sequence_length,), name="encoder_inputs")
10decoder_inputs = tf.keras.Input(shape=(sequence_length,), name="decoder_inputs")
11
12encoder_embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim, mask_zero=True)
13decoder_embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim, mask_zero=True)
14
15_, state_h, state_c = tf.keras.layers.LSTM(latent_dim, return_state=True)(
16    encoder_embedding(encoder_inputs)
17)
18
19decoder_outputs, _, _ = tf.keras.layers.LSTM(
20    latent_dim,
21    return_sequences=True,
22    return_state=True,
23)(decoder_embedding(decoder_inputs), initial_state=[state_h, state_c])
24
25logits = tf.keras.layers.Dense(vocab_size)(decoder_outputs)
26
27model = tf.keras.Model([encoder_inputs, decoder_inputs], logits)
28model.compile(
29    optimizer="adam",
30    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
31)
32
33x_encoder = np.array([
34    [1, 2, 3, 4, 0, 0],
35    [4, 3, 2, 1, 0, 0],
36], dtype="int32")
37
38y_target = np.array([
39    [4, 3, 2, 1, 10, 0],
40    [1, 2, 3, 4, 10, 0],
41], dtype="int32")
42
43y_decoder_in = np.array([
44    [9, 4, 3, 2, 1, 10],
45    [9, 1, 2, 3, 4, 10],
46], dtype="int32")
47
48model.fit([x_encoder, y_decoder_in], y_target, epochs=1)

The start token is 9 and the end token is 10. The decoder input begins with the start token, while the target sequence is what the model should predict next.

Fix the Most Common Build Errors

One of the most frequent mistakes is forgetting return_sequences=True on the decoder LSTM. Without it, the decoder produces one output per batch instead of one per time step, so the final dense layer shape does not match the target shape.

Another common issue is using the wrong loss function for the target format. Integer token ids should be paired with SparseCategoricalCrossentropy. One-hot encoded targets should use CategoricalCrossentropy instead.

Vocabulary mismatches are also common. Every token id in encoder and decoder data must be smaller than vocab_size, or the embedding lookup will fail at runtime.

Debug the Data Pipeline Before the Model

Before training on real data, inspect one batch closely.

1print(x_encoder.shape)
2print(y_decoder_in.shape)
3print(y_target.shape)
4print(x_encoder[0])
5print(y_decoder_in[0])
6print(y_target[0])

These checks catch many problems faster than reading a long stack trace. If the decoder input and target do not have the same time length, or if padding rules differ between them, the model may build but still fail in fit().

Masking matters too. Using mask_zero=True on embedding layers helps recurrent layers ignore padded tokens.

Separate Training From Inference Logic

Another source of confusion is mixing the training decoder path with inference-time decoding. During training, the decoder receives shifted ground-truth tokens. During inference, the model generates one token at a time from its own previous prediction.

If those two paths are mixed together too early, debugging becomes much harder. First get the training graph stable with a tiny synthetic dataset. Only then add the inference loop.

Common Pitfalls

• Forgetting return_sequences=True on the decoder.
• Feeding unshifted target tokens as decoder input during training.
• Using integer targets with the wrong loss function.
• Letting token ids exceed vocab_size.
• Debugging the full architecture before checking one real batch of data.

Summary

• Seq2seq training depends on aligned encoder inputs, shifted decoder inputs, and target tensors with matching time dimensions.
• The decoder usually needs return_sequences=True.
• Choose a loss function that matches whether targets are sparse ids or one-hot vectors.
• Inspect shapes and one sample batch before changing the architecture.