TensorFlow
CUDA
XLA
GPU
error-handling

Tensorflow device CUDA0 not supported by XLA service while setting up XLA_GPU_JIT device number 0

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Introduction

This error means TensorFlow tried to use XLA JIT compilation for your GPU, but the current runtime could not create a supported XLA GPU device. In practice, that usually points to one of three categories: the TensorFlow build and CUDA stack do not match, the GPU is visible to TensorFlow but not usable by XLA, or XLA was enabled when the environment was not actually ready for GPU JIT compilation.

What the message is really saying

The key parts are:

  • 'CUDA:0 means TensorFlow sees the first CUDA GPU,'
  • 'XLA_GPU_JIT means XLA JIT compilation was requested,'
  • 'not supported by XLA service means the XLA GPU backend could not use that device.'

That does not always mean "the GPU is broken." It specifically means "the XLA GPU path is not working in this environment."

First check what TensorFlow sees

Start with the basics:

python
1import tensorflow as tf
2
3print(tf.__version__)
4print(tf.config.list_physical_devices('GPU'))
5print(tf.sysconfig.get_build_info())

If no GPU devices appear, the problem is not XLA first. It is the TensorFlow GPU setup itself.

If the GPU appears, but XLA fails only when jit_compile=True or XLA flags are enabled, the issue is much more likely to be in the XLA path specifically.

Common causes

A frequent cause is a mismatch between the installed TensorFlow build and the CUDA or driver stack available on the machine. Another is enabling XLA in an environment where the GPU architecture or supporting libraries are not compatible with the JIT path being requested.

It also happens when code or environment variables force XLA unnecessarily, for example:

  • 'jit_compile=True on a function,'
  • 'TF_XLA_FLAGS enabling JIT globally,'
  • or optimizer settings that turn XLA on for the whole session.

Disable XLA as a quick diagnostic

If you want to confirm that the model itself is fine, temporarily disable XLA and run the workload normally.

python
import tensorflow as tf

tf.config.optimizer.set_jit(False)

Also remove explicit jit_compile=True where present:

python
1@tf.function
2# not @tf.function(jit_compile=True)
3def step(x):
4    return model(x)

If the code runs correctly after disabling XLA, you have confirmed that the failure is in the XLA GPU setup rather than in the general TensorFlow model code.

Check the environment, then re-enable carefully

Once the non-XLA path works, verify the GPU environment:

  • TensorFlow version
  • NVIDIA driver version
  • CUDA runtime availability
  • and whether the installation path matches TensorFlow's supported GPU setup

After that, re-enable XLA only for the narrow section that actually benefits from it. XLA is an optimization tool, not a requirement for ordinary TensorFlow GPU execution.

Common Pitfalls

The biggest mistake is assuming the presence of CUDA:0 means every GPU feature must work automatically. TensorFlow discovering a device is not the same as XLA successfully compiling for it.

Another common issue is debugging only Python code while the real problem is in the installed runtime stack. Driver, CUDA, and TensorFlow compatibility matter more here than small model edits.

Be careful with global environment flags. A setting left over from an earlier experiment can silently force XLA on and make unrelated scripts fail with the same message.

Finally, do not treat disabling XLA as a permanent failure unless you actually need XLA. Many training and inference jobs run perfectly well on GPU without XLA JIT.

Summary

  • The error means TensorFlow tried to use XLA on GPU and the XLA GPU backend could not support the current device setup.
  • First confirm whether TensorFlow sees the GPU at all.
  • If ordinary GPU execution works but XLA fails, disable XLA and verify the environment stack.
  • Check TensorFlow, driver, and CUDA compatibility before re-enabling JIT compilation.
  • XLA is optional optimization, not a requirement for normal TensorFlow GPU use.

Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Basis Lab
No course covers this one yet

Basis Lab writes one for you from a sentence about what you want to be able to do, then teaches it and checks you understood. Your first course is free.

Build my course
ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design