Introduction

When an RNN must learn from both continuous sequence features and categorical sequence features, the usual solution is to encode the categories with Embedding layers and then merge those embeddings with the numerical inputs before the recurrent layer. In Keras, that almost always means using the Functional API instead of trying to stack everything into one Sequential model.

Why numerical and categorical sequence features need different treatment

Numerical sequence features can usually go directly into an RNN after normalization. Examples include:

• sensor values
• prices
• durations
• counts

Categorical features are different. A category such as weekday=3 or product_id=27 is not a meaningful continuous quantity, so feeding raw integer IDs into an RNN is usually a bad idea. An embedding layer learns a dense vector representation for each category instead.

That gives you two streams:

• numeric tensors already shaped as sequence features
• embedded categorical tensors learned from integer category IDs

The model then merges those streams.

Merge them by concatenating per time step

The most common pattern is:

1. one input for the numeric sequence
2. one input for the categorical sequence
3. embedding on the categorical branch
4. concatenate along the feature axis
5. pass the combined tensor to the RNN

Example:

1import tensorflow as tf
2
3timesteps = 20
4num_features = 3
5num_categories = 100
6embedding_dim = 8
7
8numeric_input = tf.keras.Input(
9    shape=(timesteps, num_features), name="numeric_input"
10)
11
12category_input = tf.keras.Input(
13    shape=(timesteps,), dtype="int32", name="category_input"
14)
15
16category_embedding = tf.keras.layers.Embedding(
17    input_dim=num_categories,
18    output_dim=embedding_dim
19)(category_input)
20
21merged = tf.keras.layers.Concatenate(axis=-1)(
22    [numeric_input, category_embedding]
23)
24
25x = tf.keras.layers.LSTM(32)(merged)
26output = tf.keras.layers.Dense(1)(x)
27
28model = tf.keras.Model(
29    inputs=[numeric_input, category_input],
30    outputs=output
31)
32
33model.compile(optimizer="adam", loss="mse")
34model.summary()

The important part is that both inputs line up by time step. At each time index, the RNN receives the numeric features plus the embedding features for the categorical value at that same index.

Match the shapes correctly

The biggest shape rule is this:

• numeric branch: (batch, timesteps, numeric_features)
• categorical branch before embedding: (batch, timesteps)
• categorical branch after embedding: (batch, timesteps, embedding_dim)

After concatenation:

• merged branch: (batch, timesteps, numeric_features + embedding_dim)

If the time dimension does not match between the branches, you cannot concatenate them directly.

That is why trying to combine one Sequential model for embeddings and another for numeric input often becomes awkward. The merge happens at the tensor-shape level, so the Functional API is the natural fit.

Multiple categorical features

If you have several categorical sequence features, embed each one separately, then concatenate all branches:

1category_a = tf.keras.Input(shape=(timesteps,), dtype="int32")
2category_b = tf.keras.Input(shape=(timesteps,), dtype="int32")
3
4emb_a = tf.keras.layers.Embedding(50, 4)(category_a)
5emb_b = tf.keras.layers.Embedding(20, 3)(category_b)
6
7merged = tf.keras.layers.Concatenate(axis=-1)(
8    [numeric_input, emb_a, emb_b]
9)

This scales much better than trying to manually one-hot encode everything into a giant sparse feature axis.

Common Pitfalls

The biggest mistake is feeding raw categorical integer IDs directly into the RNN and assuming the model will interpret them meaningfully. Category IDs are labels, not continuous values.

Another mistake is forgetting that the categorical input to Embedding should normally be integer indices, not one-hot vectors. The embedding layer expects lookup indices.

Developers also run into shape mismatches by forgetting that the embedding output adds a feature dimension. After embedding, the categorical branch becomes three-dimensional.

Finally, do not force this into Sequential if there are multiple inputs. The Functional API is the right tool for multi-branch sequence models.

Summary

• Feed numerical sequence features directly and categorical sequence features through embeddings.
• Use the Functional API to build separate input branches and merge them cleanly.
• Concatenate branches along the feature axis so each time step carries both numeric and embedding information.
• Keep the time dimension aligned across all branches.
• For multiple categorical features, embed each one separately and merge them before the RNN.