Introduction

The higher library makes meta-learning code in PyTorch much easier to write because it lets you unroll optimization steps without manually cloning every parameter tensor. One option that often causes confusion is copy_initial_weights, because it changes whether the inner-loop model starts from detached copies of the base weights or from the original differentiable parameters.

What copy_initial_weights controls

Inside higher.innerloop_ctx, higher creates a functional version of your model for inner-loop updates. The question is: where do the initial fast weights come from?

That is exactly what copy_initial_weights decides.

• If copy_initial_weights=True, higher starts the functional model from copies of the current parameters.
• If copy_initial_weights=False, the functional model starts from the original parameters themselves, so gradients can flow back to them through the inner loop.

This distinction matters most in meta-learning, where the outer optimization often needs gradients with respect to the pre-update parameters.

Why MAML usually needs False

In Model-Agnostic Meta-Learning, the outer objective depends on how the model adapts after one or more inner updates. To compute the meta-gradient correctly, the adapted parameters must remain connected to the original model parameters.

That means the usual MAML-style choice is:

copy_initial_weights=False

If you instead use copied initial weights, the functional model starts from detached clones, and the outer loss no longer differentiates back to the original starting point in the same way.

A minimal example

Here is the typical structure:

1import torch
2import torch.nn as nn
3import torch.optim as optim
4import higher
5
6model = nn.Linear(1, 1)
7outer_opt = optim.SGD(model.parameters(), lr=0.01)
8inner_opt = optim.SGD(model.parameters(), lr=0.1)
9loss_fn = nn.MSELoss()
10
11x_train = torch.tensor([[1.0], [2.0]])
12y_train = torch.tensor([[2.0], [4.0]])
13x_val = torch.tensor([[3.0]])
14y_val = torch.tensor([[6.0]])
15
16with higher.innerloop_ctx(
17    model,
18    inner_opt,
19    copy_initial_weights=False
20) as (fmodel, diffopt):
21    train_loss = loss_fn(fmodel(x_train), y_train)
22    diffopt.step(train_loss)
23
24    val_loss = loss_fn(fmodel(x_val), y_val)
25    outer_opt.zero_grad()
26    val_loss.backward()
27    outer_opt.step()

With False, the validation loss can backpropagate through the inner update and reach the original model.parameters().

What happens when it is True

Setting copy_initial_weights=True creates a more isolated inner loop. That can be useful if you want to experiment with local adaptation behavior without making the original initialization part of the differentiation path.

Conceptually, the flow becomes:

1. take current parameter values
2. copy them into functional fast weights
3. update those copied fast weights in the inner loop
4. keep the original model parameters out of that unrolled gradient path

This can reduce surprises if your goal is debugging or if you only need adapted weights for evaluation, not for a meta-gradient.

A good mental model

Think of the original model parameters as the starting checkpoint.

• 'copy_initial_weights=False means the functional model starts directly from that checkpoint in a differentiable way.'
• 'copy_initial_weights=True means the functional model starts from a new snapshot that contains the same numeric values but is not the same gradient-connected starting point.'

The numbers may look identical at the beginning, but the autograd graph is different, and that difference is the entire point.

Related setting: track_higher_grads

This option is often confused with copy_initial_weights, but they solve different problems.

• 'copy_initial_weights controls where the initial fast weights come from.'
• 'track_higher_grads controls whether higher-order gradients are retained through the unrolled optimization.'

You often see them together in meta-learning code, so it is worth checking both when debugging gradient flow.

Common Pitfalls

The biggest mistake is assuming "copy" only affects memory. It also affects the autograd graph, which changes whether the outer loss can update the base model in the intended way.

Another mistake is setting copy_initial_weights=True in MAML-style code and then wondering why the meta-update behaves incorrectly or gradients appear to disappear.

People also forget that matching numeric values do not imply matching gradient connectivity. Two tensors can start with the same data and still participate in completely different computation graphs.

Finally, if gradients still look wrong after choosing the correct flag, inspect track_higher_grads, optimizer state, and any in-place operations inside the model.

Summary

• 'copy_initial_weights decides whether the inner-loop model starts from copied parameters or the original differentiable parameters.'
• Use False when you need gradients to flow back through inner-loop adaptation, as in MAML.
• Use True when you want a more isolated inner loop built from copied starting weights.
• The important difference is not the numeric values but the autograd graph.
• If meta-gradients look wrong, check this flag together with track_higher_grads.