TensorFlow
machine learning
cross feature
categorical data
continuous data

Categorical and continuous cross feature column in Tensorflow

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

In TensorFlow's classic feature-column API, a crossed feature is built from categorical inputs, not directly from raw continuous values. So if you want to cross a continuous feature with a categorical one, the usual answer is to bucketize the continuous feature first.

That is the key idea behind "categorical and continuous cross feature columns." The continuous value has to be turned into discrete buckets, and those buckets can then participate in a cross just like any other categorical feature.

Why Raw Continuous Features Cannot Be Crossed Directly

tf.feature_column.crossed_column expects categorical-style inputs. A continuous numeric feature such as age, price, or latitude has infinitely many possible values in principle, which makes a direct categorical cross impractical.

So the standard workflow is:

  1. define the numeric column
  2. bucketize it into ranges
  3. cross the bucketized result with another categorical column

That gives the model a way to learn interactions such as "device type plus age range" or "price bucket plus product category."

Bucketize the Continuous Feature

Here is a concrete example using age as the continuous feature and occupation as the categorical feature:

python
1import tensorflow as tf
2
3age = tf.feature_column.numeric_column("age")
4age_buckets = tf.feature_column.bucketized_column(
5    age,
6    boundaries=[18, 25, 35, 50, 65]
7)
8
9occupation = tf.feature_column.categorical_column_with_vocabulary_list(
10    "occupation",
11    ["student", "engineer", "teacher", "retired"]
12)

Now age_buckets is no longer a raw continuous column in the model interaction sense. It is a categorical-like representation of age ranges.

That makes it suitable for crossing.

Cross the Bucketized and Categorical Features

Once the continuous feature is bucketized, create the crossed column:

python
1age_occupation_cross = tf.feature_column.crossed_column(
2    [age_buckets, occupation],
3    hash_bucket_size=100
4)
5
6age_occupation_indicator = tf.feature_column.indicator_column(age_occupation_cross)

This crossed feature can represent interactions such as:

  • age in the 18 through 24 bucket and occupation student
  • age in the 35 through 49 bucket and occupation engineer

Those joint patterns are often more predictive than either feature alone.

Full Runnable Feature-Layer Example

The feature column API is older TensorFlow style, but a minimal runnable example still looks like this:

python
1import tensorflow as tf
2
3age = tf.feature_column.numeric_column("age")
4age_buckets = tf.feature_column.bucketized_column(age, boundaries=[18, 25, 35, 50, 65])
5
6occupation = tf.feature_column.categorical_column_with_vocabulary_list(
7    "occupation",
8    ["student", "engineer", "teacher", "retired"]
9)
10
11cross = tf.feature_column.crossed_column([age_buckets, occupation], hash_bucket_size=100)
12feature_layer = tf.keras.layers.DenseFeatures([
13    tf.feature_column.indicator_column(cross)
14])
15
16inputs = {
17    "age": tf.constant([22.0, 42.0]),
18    "occupation": tf.constant(["student", "engineer"]),
19}
20
21output = feature_layer(inputs)
22print(output.shape)

This does not show training, but it shows the feature construction pattern correctly.

When Cross Features Help

Crossed features help when the effect of one feature depends on another. For example:

  • the same price range may behave differently across product categories
  • the same age range may behave differently across occupations
  • the same region may behave differently across device types

Without the cross, a linear model may only learn the independent contributions of each feature. The cross lets it represent the interaction explicitly.

That said, too many bucket boundaries or too many crossed categories can explode feature space size. Even hashed crosses need careful design.

Common Pitfalls

  • Trying to cross a raw numeric column directly instead of bucketizing it first.
  • Using far too many bucket boundaries, which creates noisy or overly sparse crossed features.
  • Choosing a hash bucket size that is too small, causing excessive collisions.
  • Assuming a crossed feature is always better than separate features. It only helps when a real interaction exists.
  • Forgetting that the feature-column API is older TensorFlow style and may not be the preferred design in newer Keras preprocessing workflows.

Summary

  • TensorFlow crossed columns work with categorical-style inputs.
  • A continuous feature must usually be bucketized before it can be crossed.
  • The standard pattern is numeric column, bucketized column, categorical column, then crossed column.
  • Crossed features help models learn interactions that separate features cannot express as directly.
  • Use them deliberately, because excessive bucketing or crossing can create sparse, hard-to-train features.

Course illustration
Course illustration

All Rights Reserved.