Introduction

In PyTorch, state_dict and parameters() both relate to model state, but they serve different workflows. parameters() is mainly for optimization during training, while state_dict is the canonical object for saving and loading model state. Understanding the difference prevents checkpoint bugs and confusing training behavior.

What parameters() Returns

model.parameters() yields an iterator of learnable tensors, usually weights and biases registered as nn.Parameter.

1import torch
2import torch.nn as nn
3
4model = nn.Sequential(
5    nn.Linear(4, 8),
6    nn.ReLU(),
7    nn.Linear(8, 2),
8)
9
10for p in model.parameters():
11    print(p.shape, p.requires_grad)

This iterator is typically passed to an optimizer:

optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)

Important detail: parameters() does not include non-trainable buffers such as BatchNorm running statistics.

What state_dict Contains

model.state_dict() returns an ordered dictionary mapping string names to tensors representing both parameters and persistent buffers.

sd = model.state_dict()
for name, tensor in sd.items():
    print(name, tensor.shape)

For modules like BatchNorm, the dictionary includes running mean and running variance, which are not returned by parameters(). This is why state_dict is better for checkpointing complete model state.

Saving and Loading Correctly

Standard checkpointing saves state_dict objects, not raw module objects.

1import torch
2
3checkpoint = {
4    "model": model.state_dict(),
5    "optimizer": optimizer.state_dict(),
6    "epoch": 12,
7}
8
9torch.save(checkpoint, "checkpoint.pt")

Loading:

1new_model = nn.Sequential(
2    nn.Linear(4, 8),
3    nn.ReLU(),
4    nn.Linear(8, 2),
5)
6new_optimizer = torch.optim.Adam(new_model.parameters(), lr=1e-3)
7
8ckpt = torch.load("checkpoint.pt", map_location="cpu")
9new_model.load_state_dict(ckpt["model"])
10new_optimizer.load_state_dict(ckpt["optimizer"])
11start_epoch = ckpt["epoch"] + 1

This is robust across process restarts and safer than serializing entire Python objects.

When to Use Each API

Use parameters() when:

• Creating optimizers.
• Inspecting trainable tensors.
• Freezing or unfreezing gradients.

Use state_dict when:

• Saving and loading checkpoints.
• Copying weights between model instances.
• Exporting model state for reproducibility.

Example of freezing by using parameter iteration:

for p in model[0].parameters():
    p.requires_grad = False

Example of partial load by editing state_dict keys:

1pretrained = torch.load("backbone.pt", map_location="cpu")
2model_sd = model.state_dict()
3filtered = {k: v for k, v in pretrained.items() if k in model_sd and v.shape == model_sd[k].shape}
4model_sd.update(filtered)
5model.load_state_dict(model_sd)

Buffers and Why They Matter

Batch normalization and similar modules rely on running statistics. If you only handle trainable parameters and ignore buffers, inference quality can degrade even though weights seem loaded.

Because state_dict includes those buffers, it preserves behavior more faithfully between training and inference environments.

Checkpoint Compatibility Across Code Changes

As projects evolve, layer names and shapes can change. In those cases, strict loading may fail even when much of the model is reusable. A controlled partial load can speed transfer learning while keeping failures visible.

missing, unexpected = model.load_state_dict(ckpt["model"], strict=False)
print("missing:", missing)
print("unexpected:", unexpected)

Treat non-empty missing or unexpected lists as a review item, not something to ignore by default. Explicitly documenting why each mismatch is acceptable keeps experiments reproducible and avoids silent regressions.

Common Pitfalls

A common pitfall is saving only model.parameters() and expecting full model restoration. This loses module structure and non-parameter buffers, making restoration incomplete.

Another issue is loading a checkpoint into a model with different layer names or shapes. Use strict loading first, then controlled partial loading if architecture changed intentionally.

Developers also forget to save optimizer state for resumed training. Without it, momentum-based optimizers restart cold and can change convergence.

Finally, do not mutate tensors from state_dict in-place without understanding reference behavior. Clone tensors when performing manual edits to avoid unintended side effects.

Summary

• parameters() yields trainable tensors and is optimizer-focused.
• state_dict stores named parameters and persistent buffers.
• Use state_dict for reliable save and load workflows.
• Save optimizer state alongside model state for training resumes.
• Handle architecture mismatches explicitly during checkpoint loading.