Keras
Embedding Layer
Neural Networks
Machine Learning
Deep Learning

Keras - How to construct a shared Embedding Layer for each Input-Neuron

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

In Keras, a shared embedding layer is just one Embedding instance reused on multiple inputs. You create the layer once, call it on each input tensor, and Keras automatically shares the same weights across every branch. That is the right pattern when different inputs should learn from the same token or category lookup table.

Why Share an Embedding Layer

A shared embedding is useful when multiple inputs represent values from the same vocabulary. Typical examples include:

  • two text fields tokenized with the same word index
  • user and query features that use the same categorical ID space
  • comparison models such as siamese networks

Sharing keeps the representation consistent and reduces parameter count because all branches learn one common embedding table.

The Core Keras Pattern

The important detail is that you do not create several identical Embedding layers. You create one layer and reuse it.

python
1import tensorflow as tf
2from tensorflow.keras import layers, Model
3
4shared_embedding = layers.Embedding(input_dim=1000, output_dim=16, name="shared_embed")
5
6input_a = tf.keras.Input(shape=(5,), dtype="int32", name="input_a")
7input_b = tf.keras.Input(shape=(5,), dtype="int32", name="input_b")
8
9embed_a = shared_embedding(input_a)
10embed_b = shared_embedding(input_b)
11
12vec_a = layers.GlobalAveragePooling1D()(embed_a)
13vec_b = layers.GlobalAveragePooling1D()(embed_b)
14
15merged = layers.Concatenate()([vec_a, vec_b])
16output = layers.Dense(1, activation="sigmoid")(merged)
17
18model = Model(inputs=[input_a, input_b], outputs=output)
19model.summary()

Because both branches call the same shared_embedding object, they share the same trainable weights.

Training Example

Here is a small runnable example with dummy integer sequences.

python
1import numpy as np
2import tensorflow as tf
3
4x_a = np.array([
5    [1, 2, 3, 0, 0],
6    [4, 5, 6, 0, 0],
7    [1, 5, 2, 0, 0],
8    [7, 8, 9, 0, 0],
9], dtype="int32")
10
11x_b = np.array([
12    [1, 2, 2, 0, 0],
13    [4, 4, 6, 0, 0],
14    [3, 5, 2, 0, 0],
15    [7, 1, 9, 0, 0],
16], dtype="int32")
17
18y = np.array([1, 0, 1, 0], dtype="float32")
19
20model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
21model.fit([x_a, x_b], y, epochs=3, verbose=1)

The shared embedding weights are updated by gradients coming from both input branches.

When Sharing Is the Right Choice

Share the embedding only when the semantic meaning of token ID 17 should be the same in every branch.

Do not share if the inputs come from different vocabularies or different ID systems. For example, a product ID space and a country-code space may both use integers, but that does not mean they should share the same embedding table.

What “for Each Input-Neuron” Usually Means

In Keras, the embedding is applied to an input tensor, not literally to individual “neurons” in the informal sense. The layer maps integer indices to dense vectors.

So the implementation question is really about shared lookup tables across inputs, not about wiring one embedding per neuron manually.

Common Pitfalls

A common mistake is creating two separate Embedding(...) objects with the same constructor arguments and assuming they are shared. They are not. Separate instances mean separate weights.

Another mistake is sharing one embedding layer across inputs that use different vocabularies. That forces unrelated IDs into one embedding table and usually hurts learning.

A third issue is feeding non-integer data into the embedding layer. Keras embeddings expect integer indices, not one-hot vectors or floating-point features.

Summary

  • To share an embedding in Keras, create one Embedding layer instance and reuse it on multiple inputs
  • Shared embeddings are appropriate when the inputs use the same vocabulary or ID space
  • Reusing one layer keeps the representation consistent and reduces parameter count
  • Separate Embedding instances do not share weights, even if their arguments are identical
  • Use shared embeddings only when the underlying categories have the same semantic meaning across branches

Course illustration
Course illustration

All Rights Reserved.