Introduction

In Keras, built-in constraints such as max_norm or non_neg apply to weights, not to relationships between the model input and output. If you want to link input and output with a rule such as y_pred >= x or sum(output) = input_total, you usually enforce that rule through model architecture, custom loss, or a custom training step rather than through a layer constraint argument.

Why Weight Constraints Are Not Input-Output Constraints

Keras layer constraints are attached to trainable parameters.

Examples:

• 'kernel_constraint'
• 'bias_constraint'

These can keep weights non-negative or bounded, but they do not directly express relationships between the current batch input and the current batch output.

So if your goal is something like:

• output must always be greater than or equal to input
• output must sum to a known function of input
• output must equal input plus a non-negative adjustment

then a weight constraint is the wrong tool.

Best Option: Encode the Rule in the Architecture

The strongest solution is often to build the rule directly into the model output.

Suppose the output must always be at least as large as the input. Predict a non-negative delta and add it to the input.

1import tensorflow as tf
2from tensorflow import keras
3
4inputs = keras.Input(shape=(1,))
5delta = keras.layers.Dense(8, activation="relu")(inputs)
6delta = keras.layers.Dense(1, activation="softplus")(delta)
7outputs = inputs + delta
8
9model = keras.Model(inputs, outputs)
10model.compile(optimizer="adam", loss="mse")
11model.summary()

Because softplus is non-negative, the model output is structurally constrained to be at least the input. No separate penalty is needed.

This is usually better than adding a soft penalty after the fact because the model literally cannot violate the rule.

Use a Custom Loss for Soft Constraints

If the relationship should be encouraged rather than enforced exactly, add a penalty term to the loss.

1import tensorflow as tf
2
3def constrained_loss(x_true):
4    def loss(y_true, y_pred):
5        base = tf.reduce_mean(tf.square(y_true - y_pred))
6        penalty = tf.reduce_mean(tf.nn.relu(x_true - y_pred))
7        return base + 10.0 * penalty
8    return loss

This style is useful when small violations are tolerable during optimization, or when the relationship is too complex to encode exactly in the architecture.

The downside is that soft constraints can still be violated if the tradeoff with the prediction loss makes that optimal.

Multi-Input Models Can Encode Richer Links

For more complex constraints, functional Keras models let you combine raw inputs and learned outputs explicitly.

1import tensorflow as tf
2from tensorflow import keras
3
4x = keras.Input(shape=(1,), name="x")
5scale = keras.layers.Dense(1, activation="sigmoid")(x)
6y = x * scale
7
8model = keras.Model(x, y)

Here the output is always a scaled version of the input between 0 and x for positive inputs. Again, the relationship is architectural, not just a regularization term.

This is the general pattern: if the constraint is mathematically important, try to express it in the computation graph itself.

When a Custom Training Step Is Worth It

If the rule depends on batch-level structure, multiple outputs, or nonstandard penalties, a custom train_step may be cleaner than trying to force everything into model.compile(loss=...).

That becomes useful when the constraint involves:

• several outputs at once
• external physics or accounting rules
• penalties that depend on intermediate states

But most ordinary input-output constraints can still be handled with architecture or a custom loss.

Common Pitfalls

• Trying to use kernel_constraint to enforce a rule that actually depends on the current input and output values.
• Using only a soft loss penalty when the requirement is truly hard and should be encoded in the model architecture.
• Forgetting that architectural constraints often make the model simpler and more stable than post hoc penalties.
• Writing a custom loss without understanding whether the needed input values are available to it.
• Treating all constraints as regularization when some are really structural invariants.

Summary

• Keras constraints on layers affect weights, not direct input-output relationships.
• For hard links between input and output, encode the rule in the model architecture when possible.
• For softer preferences, use a custom loss penalty.
• Multi-input functional models make explicit input-output coupling much easier.
• The more fundamental the rule, the better it is to make the model obey it structurally instead of hoping training learns it.