Introduction

Keras can run with float64, but the whole pipeline has to agree on that choice. When double-precision training fails, the usual causes are mixed dtypes, unsupported GPU kernels, or custom code that silently falls back to float32.

Make the Dtype Choice Explicit

If you want double precision, configure it deliberately:

1import numpy as np
2import tensorflow as tf
3
4tf.keras.backend.set_floatx("float64")
5
6x = np.random.randn(100, 8).astype("float64")
7y = np.random.randn(100, 1).astype("float64")
8
9model = tf.keras.Sequential(
10    [
11        tf.keras.layers.Input(shape=(8,), dtype="float64"),
12        tf.keras.layers.Dense(16, activation="relu", dtype="float64"),
13        tf.keras.layers.Dense(1, dtype="float64"),
14    ]
15)
16
17model.compile(optimizer="adam", loss="mse")
18model.fit(x, y, epochs=2, verbose=0)

This is the simplest stable pattern:

• global default set to float64
• input arrays are float64
• layers are explicitly float64

If one part stays in float32, TensorFlow may insert casts or throw dtype mismatch errors.

Why GPU Runs Fail More Often

Many GPUs and kernels are optimized for float32 and lower-precision training. Double precision may be:

• slower
• unsupported for some operations
• poorly supported in a particular driver or package combination

That means a model can work on CPU and fail on GPU for the same code path.

A useful diagnostic step is to force CPU execution:

CUDA_VISIBLE_DEVICES="" python train.py

If the model works on CPU but fails on GPU, the issue is usually kernel or hardware support rather than your model definition.

Watch for Accidental Mixed Precision

A very common bug is mixing float64 inputs with float32 constants or layers:

1import numpy as np
2import tensorflow as tf
3
4x = np.array([1.0, 2.0], dtype=np.float64)
5bias = tf.constant([0.5, 0.5], dtype=tf.float32)

This may not fail immediately, but it creates instability and confusing casts deeper in the graph.

A few quick checks help:

print(x.dtype)
print(model.dtype_policy)

When debugging precision issues, inspect dtypes aggressively instead of assuming TensorFlow guessed correctly.

Custom Layers Need Extra Care

Even if the main model is configured for float64, custom code can break the guarantee:

• hard-coded float32 constants
• 'tf.cast(..., tf.float32) hidden in helper functions'
• third-party ops compiled only for common precisions
• preprocessing code that converts back to float32

So if the obvious layers look correct, inspect the custom or less-traveled parts of the pipeline next.

Decide Whether You Really Need float64

Double precision is not automatically better for deep learning. It uses more memory and often runs slower. Many models train perfectly well in float32, and many modern accelerators are tuned for that path.

Use float64 when you actually need the precision, such as:

• scientific modeling
• numerically sensitive custom losses
• long chains of calculations where accumulated error matters

If there is no strong numerical reason, float32 is usually the better engineering default.

Memory Cost Matters Too

Moving from float32 to float64 doubles tensor storage. So even if dtype support is fine, you can trigger memory errors more easily:

• larger activations
• larger optimizer state
• larger gradients

That means some "double precision fails" cases are really memory-capacity failures in disguise.

Debugging Checklist

When float64 runs fail, check in this order:

1. are the NumPy inputs really float64
2. were layers created after setting the global float type
3. does the failure happen only on GPU
4. do custom layers or helper functions hard-code float32
5. are you actually hitting a memory limit rather than a dtype-support problem

That sequence usually gets to the root cause quickly.

Common Pitfalls

• Setting float64 globally but feeding float32 inputs.
• Assuming every GPU path supports double precision as smoothly as float32.
• Hard-coding float32 constants inside custom layers.
• Forgetting that float64 doubles tensor memory usage.
• Using double precision by habit instead of because the workload truly needs it.

Summary

• Keras can run in double precision, but the full pipeline must agree on dtype.
• Most failures come from mixed dtypes, limited GPU support, or hidden float32 casts.
• Make layer and input dtypes explicit when debugging.
• If GPU execution fails, test the same code on CPU to isolate the problem.
• Use float64 intentionally, not automatically.