In TensorFlow, tf.contrib.seq2seq was an essential module that supported building sequence-to-sequence models, a type of model architecture popular in tasks like machine translation, text summarization, and more. While the Helper class in the tf.contrib.seq2seq module typically operates on embedding data, there are ways to use it even with non-embedding input data. While tf.contrib.seq2seq is deprecated in later versions of TensorFlow in favor of tf.addons.seq2seq , this article will focus on using Helper in earlier versions of TensorFlow for non-embedding data.

Sequence Modeling with Helper

Class

The Helper class is designed to assist in iterating over sequences during the decoding process. It's primarily used during the training process to specify how inputs are provided to the decoder, but it can be adapted for use with non-embedding data. Let's explore how to achieve that.

Understanding the Helper

Class

The Helper class and its derivatives (TrainingHelper , GreedyEmbeddingHelper , etc.) dictate how inputs are fed to the RNN decoder. Different Helper instances define different strategies for input feeding, whether it's during training, where ground truth inputs are available, or during inference, where predictions need to be generated iteratively.

For non-embedding data, instead of directly utilizing embedding matrices, we can operate on raw data forms such as one-hot encoded vectors or any non-transformed input data.

Implementing a Custom Helper for Non-Embedding Data

To handle non-embedding data, we may need to customize the Helper class to suit our specific input format:

• Initialization: Takes in inputs and sequence_length .
• Batch Size: Returns the batch size by using tf.shape .
• Sample: Samples the output predictions, here the argmax is computed across the output logits.
• Next Inputs: Determines the next input as we iterate over time-steps. If the sequence is not finished, it fetches the next slice of input.
• TensorFlow Version: Since tf.contrib.seq2seq was deprecated, ensure compatibility of versions or switch to tf.addons.seq2seq .
• Eager Execution: Implementations might require slight modifications if using eager execution.
• Static Shape: A static input shape enables more efficient graph compilation in TensorFlow.