Introduction

Adversarial examples are inputs that look almost unchanged to a person but push a model toward the wrong prediction. In PyTorch, the core idea is simple: compute the gradient of the loss with respect to the input, then nudge the input in the direction that increases that loss.

The Simplest Attack: FGSM

The Fast Gradient Sign Method, usually shortened to FGSM, is the standard first attack to implement. It takes one gradient step on the input:

• run a forward pass
• compute the loss against the true label
• backpropagate into the input tensor
• add epsilon * sign(gradient) to the input

For images, the final perturbed tensor is normally clamped back into the valid pixel range, such as 0 through 1.

A Runnable PyTorch Example

The example below trains a tiny classifier on synthetic two-dimensional data and then generates adversarial samples with FGSM. The data is simple, but the attack logic is the same pattern used for real models.

1import torch
2import torch.nn as nn
3import torch.optim as optim
4
5torch.manual_seed(0)
6
7# Synthetic binary classification data.
8x = torch.randn(800, 2)
9y = ((x[:, 0] + 0.8 * x[:, 1]) > 0).long()
10
11model = nn.Sequential(
12    nn.Linear(2, 16),
13    nn.ReLU(),
14    nn.Linear(16, 2),
15)
16
17criterion = nn.CrossEntropyLoss()
18optimizer = optim.Adam(model.parameters(), lr=0.01)
19
20for _ in range(200):
21    optimizer.zero_grad()
22    logits = model(x)
23    loss = criterion(logits, y)
24    loss.backward()
25    optimizer.step()
26
27
28def fgsm_attack(model, inputs, labels, epsilon):
29    adv_inputs = inputs.clone().detach().requires_grad_(True)
30    logits = model(adv_inputs)
31    loss = criterion(logits, labels)
32
33    model.zero_grad()
34    loss.backward()
35
36    perturbation = epsilon * adv_inputs.grad.sign()
37    attacked = adv_inputs + perturbation
38    return attacked.detach()
39
40
41test_points = x[:10]
42test_labels = y[:10]
43
44clean_predictions = model(test_points).argmax(dim=1)
45adversarial_points = fgsm_attack(model, test_points, test_labels, epsilon=0.25)
46adversarial_predictions = model(adversarial_points).argmax(dim=1)
47
48print("true labels       :", test_labels.tolist())
49print("clean predictions :", clean_predictions.tolist())
50print("adversarial preds :", adversarial_predictions.tolist())

Even on this toy problem, some points that were classified correctly before the attack will flip after the perturbation is added.

Why requires_grad_ Matters

Normally, you optimize model weights, not inputs. For adversarial attacks, you temporarily mark the input tensor as differentiable so PyTorch can populate input.grad after loss.backward().

That is the critical shift in perspective. The model parameters stay fixed. The attack changes the data instead.

For image models, the attack function usually looks like this:

1def fgsm_image_attack(model, images, labels, epsilon):
2    images = images.clone().detach().requires_grad_(True)
3    outputs = model(images)
4    loss = nn.CrossEntropyLoss()(outputs, labels)
5
6    model.zero_grad()
7    loss.backward()
8
9    attacked = images + epsilon * images.grad.sign()
10    attacked = torch.clamp(attacked, 0.0, 1.0)
11    return attacked.detach()

The torch.clamp call is important when the input space has known bounds.

Beyond FGSM

FGSM is a one-step attack. Stronger attacks usually repeat the update several times while projecting the sample back into an allowed perturbation region. Projected Gradient Descent, or PGD, is the classic next step because it often finds more reliable failures than one gradient step.

You can also switch from untargeted attacks to targeted ones. An untargeted attack only tries to make the model wrong. A targeted attack tries to push the model toward one specific incorrect class.

For defense experiments, people often combine these attacks with adversarial training, where the model sees perturbed samples during training rather than only at evaluation time.

Common Pitfalls

• Forgetting requires_grad_(True) on the input means the attack has no gradient to use.
• Calling the attack while the model is in training mode can introduce randomness through dropout or batch normalization updates.
• Choosing an epsilon that is too large makes the perturbation unrealistic and the evaluation less informative.
• Not clamping image data back to the legal range creates invalid samples.
• Comparing attacks across models without normalizing input preprocessing can produce misleading results.

Summary

• Adversarial examples are created by perturbing the input, not the model weights.
• FGSM is the easiest PyTorch attack to implement because it needs only one backward pass.
• Mark the input tensor as requiring gradients, compute the loss, then step in the gradient-sign direction.
• Clamp attacked image tensors back to the valid input range.
• Use FGSM to learn the workflow, then move to stronger attacks such as PGD for more realistic robustness testing.