Keras
Adam optimizer
momentum
machine learning
closed-question

Is there a momentum option for Adam optimizer in Keras?

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Introduction

Keras Adam does not expose a separate momentum= argument like SGD does. That is because Adam already contains a momentum-like mechanism through its first-moment moving average, controlled by beta_1.

Adam Already Includes Momentum-Like Behavior

Adam keeps an exponentially decaying average of past gradients. In practical terms, that is the role momentum plays in ordinary SGD.

In Keras, you configure that behavior with beta_1.

python
1from tensorflow import keras
2
3optimizer = keras.optimizers.Adam(
4    learning_rate=1e-3,
5    beta_1=0.9,
6    beta_2=0.999,
7)
8
9print(optimizer)

So the short answer is:

  • no separate momentum parameter on Adam
  • yes, Adam already has momentum-like behavior through beta_1

Compare with SGD Momentum

For SGD, momentum is exposed explicitly:

python
1from tensorflow import keras
2
3optimizer = keras.optimizers.SGD(
4    learning_rate=1e-2,
5    momentum=0.9,
6)

That can make the APIs look inconsistent, but the optimizers are not parameterized in the same way. SGD starts from vanilla gradient descent and optionally adds momentum. Adam is defined around first and second moment estimates from the beginning.

What beta_1 Does

beta_1 controls how strongly Adam remembers past gradients. Higher values mean a longer memory of recent gradient direction, which is analogous to stronger momentum behavior.

A simplified mental model is:

  • larger beta_1 means smoother, more persistent first-moment estimates
  • smaller beta_1 means the optimizer reacts more quickly to the current gradient

That is not identical to classic SGD momentum in every detail, but it is the closest parameter if you are trying to tune the momentum-like part of Adam.

If You Want Nesterov-Like Behavior

If your real question is about adding Nesterov-style momentum to Adam, the relevant optimizer is usually Nadam, not Adam with a hidden momentum switch.

python
1from tensorflow import keras
2
3optimizer = keras.optimizers.Nadam(
4    learning_rate=1e-3,
5    beta_1=0.9,
6    beta_2=0.999,
7)

That is a better fit than looking for a nonexistent momentum= flag on Adam itself.

Tune the Right Thing

If training with Adam feels unstable or too sluggish, the first parameters to consider are often:

  • learning rate
  • 'beta_1'
  • 'beta_2'
  • epsilon

Many optimizer questions are really learning-rate questions wearing optimizer names. So before searching for a momentum flag, check whether the real need is to adjust the learning rate or move to a different optimizer family.

Why the API Is Shaped This Way

Keras exposes Adam according to the optimizer's own math, not according to a generic optimizer interface where every algorithm must accept the same knobs. That is why SGD has momentum= and Adam has beta_1=. They are solving related but not identical update problems, so the parameter names follow the algorithm rather than forcing a misleading uniform vocabulary.

A Practical Tuning Mindset

If a model trains poorly with Adam, switching optimizers can sometimes matter more than trying to make Adam behave exactly like SGD with momentum. Optimizer choice is part of the modeling decision, not just a parameter-search exercise. Sometimes the cleanest answer is simply that you want a different optimizer family rather than a hidden Adam option.

Common Pitfalls

  • Looking for momentum= on Adam just because SGD exposes it that way.
  • Assuming Adam and SGD momentum are identical mechanisms with different names.
  • Changing beta_1 without considering the learning rate at the same time.
  • Expecting Adam tuning to behave exactly like SGD-with-momentum tuning.
  • Using Adam when the training problem might actually benefit from SGD, AdamW, or Nadam instead.

Summary

  • Keras Adam does not have a separate momentum parameter.
  • Adam already includes a momentum-like first-moment estimate controlled by beta_1.
  • If you want classic explicit momentum, use SGD(momentum=...).
  • If you want Nesterov-style Adam, consider Nadam.
  • Tune beta_1 only with a clear understanding that it is part of Adam's own update rule, not a bolt-on SGD momentum switch.

Related reading
Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

All Rights Reserved.