Introduction

In logistic regression, the warning divide by zero encountered in log usually appears when your loss function computes log(p) or log(1 - p) and the predicted probability has become exactly 0 or 1. The model math is still conceptually valid, but the numerical implementation is unstable. The real fix is to make the loss computation numerically safe, not to silence the warning blindly.

Where The Warning Comes From

A common binary cross-entropy formula is:

• '-y * log(p) - (1 - y) * log(1 - p)'

If p == 0, then log(p) is undefined. If p == 1, then log(1 - p) is undefined.

That is enough to trigger warnings and produce inf or nan values.

A naive implementation shows the problem quickly:

1import numpy as np
2
3
4y_true = np.array([1, 0, 1])
5p = np.array([1.0, 0.0, 0.8])
6
7loss = -np.mean(y_true * np.log(p) + (1 - y_true) * np.log(1 - p))
8print(loss)

This can emit the divide-by-zero warning because np.log(0) occurs.

Fix 1: Clip The Probabilities

The most direct fix is to clip probabilities away from the exact endpoints.

1import numpy as np
2
3
4y_true = np.array([1, 0, 1])
5p = np.array([1.0, 0.0, 0.8])
6
7eps = 1e-15
8p = np.clip(p, eps, 1 - eps)
9
10loss = -np.mean(y_true * np.log(p) + (1 - y_true) * np.log(1 - p))
11print(loss)

This is the standard defense when you are implementing logistic regression loss manually.

Why it works:

• 'p is never exactly 0'
• 'p is never exactly 1'
• both logarithm terms stay finite

The clipped values are extremely close to the originals, but they avoid numerical singularities.

Why Predictions Reach Exactly 0 Or 1

In theory, the sigmoid function outputs values strictly between 0 and 1. In practice, floating-point arithmetic can still drive values to the machine limits when logits become very large in magnitude.

For example, if your code computes probabilities from huge logits, rounding can effectively collapse values to endpoints.

That means the warning is often a symptom of one of these issues:

• manual probability computation with unstable formulas
• extremely large positive or negative logits
• overly aggressive optimization steps
• unscaled features that push the model into extreme ranges

Fix 2: Use A Numerically Stable Loss Formula

A better long-term fix is to compute loss from logits with a stable expression instead of calculating sigmoid first and taking logs afterward.

One stable form for binary cross-entropy uses np.logaddexp:

1import numpy as np
2
3
4def stable_logistic_loss(y_true, logits):
5    return np.mean(np.logaddexp(0, logits) - y_true * logits)
6
7
8logits = np.array([8.0, -10.0, 2.0])
9y_true = np.array([1.0, 0.0, 1.0])
10
11print(stable_logistic_loss(y_true, logits))

This avoids the fragile log(sigmoid(...)) pattern altogether.

If you are using a machine learning library, prefer its built-in logistic loss instead of writing the loss manually unless you really need custom training code.

Fix 3: Check Your Feature Scale And Optimization

Sometimes the warning reflects training instability rather than just loss implementation. If features are poorly scaled or the learning rate is too high, logits can become extreme very quickly.

Practical defenses include:

• standardize or normalize features
• lower the learning rate in custom optimization code
• add regularization
• monitor logits or predicted probabilities during training

These do not replace clipping or stable formulas, but they reduce the likelihood of numerically extreme states.

What Not To Do

Do not just disable NumPy warnings and move on. That hides the signal while leaving the unstable computation intact.

Also, do not assume scikit-learn's LogisticRegression is the problem. If you are using the library normally, it already uses stable internals. The warning usually appears when you write the probability or loss computation manually in NumPy.

A Small Safe Helper

If you need manual binary cross-entropy repeatedly, wrap the clipping logic.

1import numpy as np
2
3
4def binary_cross_entropy(y_true, p, eps=1e-15):
5    p = np.clip(p, eps, 1 - eps)
6    return -np.mean(y_true * np.log(p) + (1 - y_true) * np.log(1 - p))
7
8
9print(binary_cross_entropy(np.array([1, 0]), np.array([1.0, 0.0])))

That keeps the stability fix close to the formula and prevents accidental regression later.

Common Pitfalls

• Computing np.log(p) and np.log(1 - p) on probabilities that can reach 0 or 1.
• Silencing the warning instead of fixing the unstable computation.
• Computing loss from probabilities when a logits-based stable formula would be better.
• Ignoring feature scaling and learning-rate issues that create extreme logits.
• Reimplementing logistic regression internals unnecessarily when a library already provides a stable loss.

Summary

• The warning comes from taking a logarithm of 0 in logistic regression loss.
• Clip probabilities away from 0 and 1 if you compute the loss manually.
• Prefer numerically stable logits-based formulas such as those built on np.logaddexp.
• Check feature scaling and optimization settings if logits become extreme.
• Do not hide the warning without fixing the underlying math.