Error when running Tensorflow Sequence to Sequence Tutorial

TensorFlow

Sequence to Sequence

Tutorial Error

Machine Learning

Neural Networks

Error when running Tensorflow Sequence to Sequence Tutorial

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

TensorFlow & Keras Machine Learning

Introduction

Errors when running TensorFlow's Sequence-to-Sequence (Seq2Seq) tutorial typically fall into three categories: version incompatibility (tutorial code written for TF 1.x but running on TF 2.x), deprecated API references (tf.contrib.seq2seq was removed in TF 2.0), and shape mismatches in encoder/decoder tensors. The fix depends on which TF version you are using — for TF 2.x, use tf.keras layers and the tfa.seq2seq module from TensorFlow Addons, or rewrite with the native Keras API.

Common Error 1: tf.contrib Not Found

python

# ERROR in TF 2.x
from tensorflow.contrib.seq2seq import BasicDecoder, TrainingHelper
# ModuleNotFoundError: No module named 'tensorflow.contrib'

tf.contrib was removed entirely in TensorFlow 2.0. The Seq2Seq utilities moved to TensorFlow Addons:

bash

pip install tensorflow-addons

python

1# TF 2.x replacement
2import tensorflow_addons as tfa
3
4decoder = tfa.seq2seq.BasicDecoder(cell, sampler, output_layer)
5helper = tfa.seq2seq.TrainingSampler(...)

Common Error 2: Session-Based Code on TF 2.x

python

1# TF 1.x style — does not work in TF 2.x
2import tensorflow as tf
3
4encoder_inputs = tf.placeholder(tf.int32, [None, None])
5decoder_inputs = tf.placeholder(tf.int32, [None, None])
6
7# AttributeError: module 'tensorflow' has no attribute 'placeholder'

TF 2.x uses eager execution by default — no sessions or placeholders:

python

1# TF 2.x — use Keras layers and eager execution
2import tensorflow as tf
3
4class Encoder(tf.keras.Model):
5    def __init__(self, vocab_size, embedding_dim, enc_units):
6        super().__init__()
7        self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim)
8        self.gru = tf.keras.layers.GRU(enc_units, return_state=True)
9
10    def call(self, x):
11        x = self.embedding(x)
12        output, state = self.gru(x)
13        return output, state
14
15class Decoder(tf.keras.Model):
16    def __init__(self, vocab_size, embedding_dim, dec_units):
17        super().__init__()
18        self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim)
19        self.gru = tf.keras.layers.GRU(dec_units, return_sequences=True, return_state=True)
20        self.fc = tf.keras.layers.Dense(vocab_size)
21
22    def call(self, x, hidden):
23        x = self.embedding(x)
24        output, state = self.gru(x, initial_state=hidden)
25        predictions = self.fc(output)
26        return predictions, state

Common Error 3: Shape Mismatch

python

# ValueError: Dimensions must be equal, but are 256 and 512
# for encoder hidden state (256) fed into decoder (512)

The encoder's hidden state dimension must match the decoder's expected initial state:

python

1# WRONG: mismatched units
2encoder = Encoder(vocab_size=5000, embedding_dim=256, enc_units=256)
3decoder = Decoder(vocab_size=5000, embedding_dim=256, dec_units=512)
4# Encoder outputs state of shape (batch, 256)
5# Decoder expects initial_state of shape (batch, 512)
6
7# FIX: match encoder and decoder units
8encoder = Encoder(vocab_size=5000, embedding_dim=256, enc_units=512)
9decoder = Decoder(vocab_size=5000, embedding_dim=256, dec_units=512)

Common Error 4: Incompatible TF and Python Versions

bash

# ImportError: cannot import name 'xxx' from 'tensorflow'
# or
# AttributeError: module 'tensorflow' has no attribute 'xxx'

bash

1# Check your versions
2python -c "import tensorflow as tf; print(tf.__version__)"
3python --version
4
5# Install compatible versions
6pip install tensorflow==2.15.0  # Latest stable as of early 2025
7
8# Or use the compat module for TF 1.x code on TF 2.x
9import tensorflow.compat.v1 as tf
10tf.disable_v2_behavior()

Complete Seq2Seq Example (TF 2.x)

python

1import tensorflow as tf
2import numpy as np
3
4class Encoder(tf.keras.Model):
5    def __init__(self, vocab_size, embedding_dim, units):
6        super().__init__()
7        self.units = units
8        self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim)
9        self.gru = tf.keras.layers.GRU(units, return_sequences=True, return_state=True)
10
11    def call(self, x):
12        x = self.embedding(x)
13        output, state = self.gru(x)
14        return output, state
15
16class BahdanauAttention(tf.keras.layers.Layer):
17    def __init__(self, units):
18        super().__init__()
19        self.W1 = tf.keras.layers.Dense(units)
20        self.W2 = tf.keras.layers.Dense(units)
21        self.V = tf.keras.layers.Dense(1)
22
23    def call(self, query, values):
24        query_with_time = tf.expand_dims(query, 1)
25        score = self.V(tf.nn.tanh(self.W1(values) + self.W2(query_with_time)))
26        attention_weights = tf.nn.softmax(score, axis=1)
27        context_vector = attention_weights * values
28        context_vector = tf.reduce_sum(context_vector, axis=1)
29        return context_vector, attention_weights
30
31class Decoder(tf.keras.Model):
32    def __init__(self, vocab_size, embedding_dim, units):
33        super().__init__()
34        self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim)
35        self.gru = tf.keras.layers.GRU(units, return_sequences=True, return_state=True)
36        self.fc = tf.keras.layers.Dense(vocab_size)
37        self.attention = BahdanauAttention(units)
38
39    def call(self, x, hidden, enc_output):
40        context, weights = self.attention(hidden, enc_output)
41        x = self.embedding(x)
42        x = tf.concat([tf.expand_dims(context, 1), x], axis=-1)
43        output, state = self.gru(x)
44        output = tf.reshape(output, (-1, output.shape[2]))
45        predictions = self.fc(output)
46        return predictions, state, weights
47
48# Training step
49@tf.function
50def train_step(inp, targ, encoder, decoder, optimizer, loss_fn):
51    loss = 0
52    with tf.GradientTape() as tape:
53        enc_output, enc_hidden = encoder(inp)
54        dec_hidden = enc_hidden
55        dec_input = tf.expand_dims([start_token] * batch_size, 1)
56
57        for t in range(1, targ.shape[1]):
58            predictions, dec_hidden, _ = decoder(dec_input, dec_hidden, enc_output)
59            loss += loss_fn(targ[:, t], predictions)
60            dec_input = tf.expand_dims(targ[:, t], 1)  # Teacher forcing
61
62    batch_loss = loss / int(targ.shape[1])
63    variables = encoder.trainable_variables + decoder.trainable_variables
64    gradients = tape.gradient(loss, variables)
65    optimizer.apply_gradients(zip(gradients, variables))
66    return batch_loss

Migrating TF 1.x Seq2Seq Code

python

1# Quick migration using compat mode
2import tensorflow.compat.v1 as tf
3tf.disable_v2_behavior()
4
5# Now TF 1.x code runs as-is
6# But this is a temporary fix — rewrite to Keras for long-term support
7
8# For tf.contrib.seq2seq specifically:
9# TF 1.x:
10#   tf.contrib.seq2seq.BasicDecoder
11#   tf.contrib.seq2seq.dynamic_decode
12#   tf.contrib.seq2seq.TrainingHelper
13#
14# TF 2.x (TensorFlow Addons):
15#   tfa.seq2seq.BasicDecoder
16#   tfa.seq2seq.dynamic_decode
17#   tfa.seq2seq.TrainingSampler

Common Pitfalls

Using tutorials written for TF 1.x: Most Seq2Seq tutorials (especially from 2017-2019) use tf.contrib, tf.placeholder, and tf.Session. These APIs do not exist in TF 2.x. Look for tutorials that use tf.keras and @tf.function.
TensorFlow Addons deprecation: tensorflow-addons is in maintenance mode and may not support the latest TF versions. For new projects, implement attention and decoding using native Keras layers instead of tfa.seq2seq.
Teacher forcing mismatch: During training, the decoder receives the ground-truth token at each step (teacher forcing). During inference, it receives its own previous output. Using training code for inference without switching to autoregressive decoding produces garbage output.
Forgetting @tf.function for training: Training loops without @tf.function run in eager mode, which is 2-10x slower than graph mode. Decorate your train_step function for production-speed training.
Memory errors on large vocabularies: The final Dense layer has vocab_size outputs. With vocabulary sizes of 30,000+, this layer uses significant memory. Use tf.nn.sampled_softmax_loss during training to reduce memory for large vocabularies.

Summary

Most Seq2Seq tutorial errors come from TF 1.x code running on TF 2.x
tf.contrib.seq2seq was removed in TF 2.0 — use tfa.seq2seq or native Keras layers
Replace tf.placeholder and tf.Session with Keras models and eager execution
Ensure encoder and decoder hidden dimensions match to avoid shape errors
Use tensorflow.compat.v1 as a temporary migration path, but rewrite to Keras long-term
For new Seq2Seq projects, use tf.keras with custom attention layers

Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course

Basis Lab

No course covers this one yet

Basis Lab writes one for you from a sentence about what you want to be able to do, then teaches it and checks you understood. Your first course is free.

Build my course

Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Error when running Tensorflow Sequence to Sequence Tutorial

ML System Design practice on Codemia

Introduction

Common Error 1: tf.contrib Not Found

Common Error 2: Session-Based Code on TF 2.x

Common Error 3: Shape Mismatch

Common Error 4: Incompatible TF and Python Versions

Complete Seq2Seq Example (TF 2.x)

Migrating TF 1.x Seq2Seq Code

Common Pitfalls

Summary

Related reading

Tackling System Design Interview Problems

No course covers this one yet

Track what you have practised

ML System Design practice on Codemia