Introduction

A simple PyTorch model that works on CPU can fail on GPU for only a few recurring reasons. In most cases, the model, inputs, labels, or loss-related tensors are not all on the same device, or the local PyTorch build does not actually have CUDA support. The fastest way to debug it is to verify device availability first, then move every participating tensor and module to the same device consistently.

Start With the Device Check

Before looking at model code, confirm that PyTorch can see a CUDA-capable GPU.

1import torch
2
3print(torch.cuda.is_available())
4if torch.cuda.is_available():
5    print(torch.cuda.get_device_name(0))

If torch.cuda.is_available() is False, the problem is not your network architecture. It means the current environment does not have a usable CUDA path for PyTorch.

That could be due to a CPU-only PyTorch install, missing drivers, or a machine with no supported GPU.

Move Model and Data to the Same Device

The most common runtime error is mixing CPU and GPU tensors.

1import torch
2import torch.nn as nn
3
4
5device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
6
7model = nn.Sequential(
8    nn.Linear(10, 32),
9    nn.ReLU(),
10    nn.Linear(32, 2),
11).to(device)
12
13x = torch.randn(8, 10).to(device)
14y = torch.randint(0, 2, (8,)).to(device)
15
16criterion = nn.CrossEntropyLoss()
17output = model(x)
18loss = criterion(output, y)
19loss.backward()

If model is on the GPU and x or y stays on the CPU, PyTorch raises a device mismatch error.

The rule is simple: model parameters, inputs, targets, and any manually created tensors used in the forward or loss computation must all live on the same device.

Watch for Hidden CPU Tensors

A subtle version of the same bug happens when you create new tensors inside the training step without moving them to the device.

mask = torch.ones_like(output, device=device)

This is safer than:

mask = torch.ones(output.shape)

The second line creates a CPU tensor by default unless you specify the device. Hidden CPU tensors are a very common reason a "simple network" suddenly stops working only when moved to the GPU.

Optimizer and Model Initialization Order

Make sure the model is moved to the device before training begins and before you rely on its parameters in the normal training flow.

model = MyModel().to(device)
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)

This is the normal safe pattern. It keeps your parameter references consistent from the start.

GPU Errors Are Sometimes Environment Errors

Not every GPU failure is a coding bug. If CUDA is unavailable or PyTorch was installed without CUDA support, calling .to("cuda") will fail even if the code is otherwise correct.

A quick environment check is:

1import torch
2print(torch.__version__)
3print(torch.version.cuda)
4print(torch.cuda.is_available())

If the reported CUDA support is missing or unavailable, fix the environment before debugging model code.

Minimal Training Loop Pattern

A good structure keeps device movement explicit and centralized.

1for inputs, labels in dataloader:
2    inputs = inputs.to(device)
3    labels = labels.to(device)
4
5    optimizer.zero_grad()
6    outputs = model(inputs)
7    loss = criterion(outputs, labels)
8    loss.backward()
9    optimizer.step()

If you keep the .to(device) calls at the batch boundary, device management becomes easier to audit.

Common Pitfalls

The most common mistake is moving the model to the GPU but forgetting to move the input batch or target labels.

Another issue is creating extra tensors inside the forward pass or loss computation without placing them on the same device.

Developers also sometimes assume that having an NVIDIA GPU automatically means the current PyTorch install supports CUDA. That is not guaranteed.

Finally, do not debug GPU-specific performance or convergence issues until the basic device consistency problem is solved first.

Summary

• Verify CUDA availability before debugging the network itself.
• Move the model, inputs, labels, and auxiliary tensors to the same device.
• Watch for hidden CPU tensor creation inside training code.
• Keep device movement explicit at the batch boundary.
• If .to("cuda") fails outright, fix the PyTorch or driver environment before changing model logic.