Introduction

Yes, the built-in L1 regularizer in Keras and TensorFlow is genuinely an L1 penalty: it adds a term proportional to the sum of absolute weight values to the training objective. The confusion usually comes from expecting exact zero weights immediately, while in practice the observed sparsity depends on optimizer behavior, regularization strength, and which parameters the regularizer is attached to.

What L1 Regularization Means in Practice

The textbook idea of L1 regularization is simple: take the original loss and add a penalty based on the absolute values of the weights. In words, the objective becomes:

• prediction loss
• plus lambda * sum(abs(weights))

That is exactly the penalty Keras applies when you use an L1 regularizer on a layer parameter such as the kernel.

A minimal example looks like this:

1import tensorflow as tf
2
3model = tf.keras.Sequential([
4    tf.keras.layers.Dense(
5        32,
6        activation="relu",
7        kernel_regularizer=tf.keras.regularizers.L1(1e-4),
8        input_shape=(10,)
9    ),
10    tf.keras.layers.Dense(1)
11])
12
13model.compile(optimizer="adam", loss="mse")

When you train this model, TensorFlow adds the regularization term to the model loss automatically.

Where the Penalty Shows Up in Keras

In Keras, regularizers contribute through the layer losses collection. You can inspect that directly.

1import tensorflow as tf
2
3layer = tf.keras.layers.Dense(
4    4,
5    kernel_regularizer=tf.keras.regularizers.L1(0.01)
6)
7
8x = tf.ones((2, 3))
9_ = layer(x)
10
11print(layer.losses)

After the layer has been called once and its weights exist, layer.losses contains the L1 penalty term. At the model level, Keras includes these extra losses during training, so the optimizer minimizes both the prediction loss and the regularization loss together.

That behavior is not an approximation of some other penalty. It is the standard L1 form applied to the selected parameter tensor.

Why You May Not See Exact Zeros Immediately

This is where most confusion starts. In theory, L1 regularization encourages sparsity. In practice, seeing lots of exact zeros depends on training details.

For example:

• a very small regularization factor may barely move weights toward zero
• adaptive optimizers may not produce sharp sparsity as aggressively as people expect
• short training runs may stop before the regularizer has much visible effect
• the penalty may only be attached to kernels, not biases or other parameters

You can inspect the learned weights after training:

weights = model.layers[0].get_weights()[0]
print(weights)

If many values become small but not exactly zero, the regularizer is still working. It is pushing the solution toward sparsity, even if the final optimizer path does not produce a perfect pile of literal zeros.

Apply It to the Right Parameter

In Keras, you choose which parameters receive the penalty. The most common option is kernel_regularizer, but there are also bias_regularizer and activity_regularizer options.

1tf.keras.layers.Dense(
2    16,
3    kernel_regularizer=tf.keras.regularizers.L1(1e-4),
4    bias_regularizer=tf.keras.regularizers.L1(1e-5)
5)

If someone expects "the whole layer" to be L1-regularized but only sets kernel_regularizer, that expectation mismatch can look like the implementation is wrong when it is really just narrower than assumed.

Compare L1 to L2 and L1L2

Keras also exposes L2 and combined L1L2 regularizers.

1tf.keras.layers.Dense(
2    16,
3    kernel_regularizer=tf.keras.regularizers.L1L2(l1=1e-4, l2=1e-3)
4)

L2 pushes weights to be small in magnitude, while L1 pushes many weights toward zero more aggressively. If your model ends up dense but smooth, you may have chosen a regularization strength that behaves more like mild weight shrinkage than strong sparsity induction.

The Correct Answer to the Original Question

So, is Keras or TensorFlow "really" doing L1 regularization. Yes. The penalty term is the ordinary sum of absolute values multiplied by the specified coefficient. What varies is the optimization outcome, not the mathematical definition of the regularizer.

The practical lesson is to validate the full setup rather than judging from a single expectation such as "I do not see exact zeros after one epoch." The regularizer can be correct while the training configuration is too weak to make the sparsity obvious.

Common Pitfalls

• Expecting exact zeros immediately can lead to the wrong conclusion. L1 encourages sparsity, but the visible result depends on optimizer, coefficient strength, and training duration.
• Setting kernel_regularizer and then assuming biases are regularized too is incorrect. Only the specified parameters receive the penalty.
• Using a tiny L1 coefficient often makes the effect too small to notice in learned weights.
• Comparing L1 and L2 outcomes without checking the actual regularizer configuration can confuse shrinkage behavior with implementation correctness.
• Inspecting only prediction loss and ignoring model.losses can make it seem like regularization is missing when Keras is actually adding it internally.

Summary

• Keras and TensorFlow implement true L1 regularization as an absolute-value penalty added to the loss.
• The regularizer is mathematically correct even when trained weights are only near zero instead of exactly zero.
• In practice, sparsity depends on coefficient strength, optimizer choice, and training duration.
• 'kernel_regularizer, bias_regularizer, and related options apply the penalty only where you specify it.'
• To judge the effect correctly, inspect both the model configuration and the learned weights after training.