Introduction

A TextVectorization layer is part of your preprocessing pipeline, so saving it matters just as much as saving the model weights. The safest pattern is to save it as part of a Keras model after the layer has been adapted. If you need the layer on its own, you can also persist its vocabulary and configuration and then rebuild it later.

Adapt the Layer Before Saving It

A TextVectorization layer is not fully configured until it has been adapted or given an explicit vocabulary.

1import tensorflow as tf
2
3texts = tf.data.Dataset.from_tensor_slices([
4    "tensorflow makes text preprocessing easier",
5    "text vectorization builds a vocabulary",
6    "models need consistent preprocessing",
7]).batch(2)
8
9vectorizer = tf.keras.layers.TextVectorization(
10    max_tokens=1000,
11    output_mode="int",
12    output_sequence_length=6,
13)
14vectorizer.adapt(texts)
15
16print(vectorizer.get_vocabulary()[:10])

If you save the layer before adaptation, you are not really saving the learned vocabulary state that makes the layer useful.

The Easiest Method: Save It Inside a Keras Model

The most robust approach is to wrap the preprocessing layer in a Keras model and save the model.

1import tensorflow as tf
2
3inputs = tf.keras.Input(shape=(1,), dtype=tf.string)
4outputs = vectorizer(inputs)
5preprocessing_model = tf.keras.Model(inputs, outputs)
6
7preprocessing_model.save("text_vectorizer.keras")

Then load it later with:

loaded = tf.keras.models.load_model("text_vectorizer.keras")
print(loaded(tf.constant([["consistent text preprocessing matters"]])))

This keeps the layer configuration and learned vocabulary together in one artifact.

Save the Full Training Model When Possible

If your text model already uses TextVectorization as its first layer, the cleanest deployment artifact is usually the entire end-to-end model.

1inputs = tf.keras.Input(shape=(1,), dtype=tf.string)
2x = vectorizer(inputs)
3x = tf.keras.layers.Embedding(input_dim=1000, output_dim=8)(x)
4x = tf.keras.layers.GlobalAveragePooling1D()(x)
5outputs = tf.keras.layers.Dense(1, activation="sigmoid")(x)
6
7model = tf.keras.Model(inputs, outputs)
8model.save("full_text_model.keras")

That avoids vocabulary drift between training and inference because the preprocessing travels with the model.

Save the Vocabulary Separately if You Need Only the Layer State

If you do not want to save a model artifact, you can store the vocabulary and recreate the layer manually.

1vocab = vectorizer.get_vocabulary()
2with open("vocab.txt", "w", encoding="utf-8") as f:
3    for token in vocab:
4        f.write(token + "\n")

Later, rebuild the layer with the same configuration.

1import tensorflow as tf
2
3with open("vocab.txt", "r", encoding="utf-8") as f:
4    vocab = [line.rstrip("\n") for line in f]
5
6restored = tf.keras.layers.TextVectorization(
7    max_tokens=1000,
8    output_mode="int",
9    output_sequence_length=6,
10    vocabulary=vocab,
11)
12
13print(restored(tf.constant([["text preprocessing matters"]])))

This is useful when you want lightweight portability, but it requires discipline because you must recreate the same configuration values manually.

Configuration Matters as Much as Vocabulary

The vocabulary is not the only state. The following options affect behavior and must match on restore:

• 'standardize'
• 'split'
• 'ngrams'
• 'output_mode'
• 'output_sequence_length'
• 'max_tokens'

If those change, the restored layer may tokenize or encode text differently even if the vocabulary file is correct.

Why Saving the Layer Separately Can Be Risky

Teams sometimes save only model weights and then rebuild preprocessing from memory. That is fragile. If one deployment uses a slightly different vocabulary or standardization rule, predictions become inconsistent.

For production systems, the main goal is reproducibility. Saving the preprocessing together with the model is usually the most reliable way to get it.

Common Pitfalls

• Saving the layer before calling adapt.
• Saving only model weights and forgetting the text preprocessing state.
• Restoring the vocabulary but not the original layer configuration.
• Letting training and inference use different tokenization rules.
• Treating TextVectorization as disposable preprocessing instead of part of the learned pipeline.

Summary

• A TextVectorization layer should be saved after adaptation.
• The easiest and safest method is to save it inside a Keras model.
• Saving the full end-to-end model avoids preprocessing drift.
• If needed, you can persist the vocabulary and rebuild the layer manually.
• Reproducible text preprocessing depends on both the vocabulary and the original configuration.