AWS
S3
InvalidToken
ListBuckets
Error Handling

An error occurred InvalidToken when calling the ListBuckets operation The provided token is malformed or otherwise invalid. w/aws s3 ls

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

This AWS CLI error means the credentials currently being used for aws s3 ls include a session token that is missing, expired, malformed, or inconsistent with the access key pair. The key point is that ListBuckets itself is usually not the real problem. The request is failing before S3 authorization logic even begins because the caller's temporary credentials are invalid. The fix is to inspect which credentials the CLI is actually using and then refresh or correct that source.

Why This Happens

The AWS CLI can authenticate with several credential sources:

  • environment variables
  • shared credentials files
  • named profiles
  • SSO sessions
  • assumed-role or STS temporary credentials
  • EC2, ECS, or EKS instance metadata

The InvalidToken error usually appears when one of those sources provides a broken session token or mixes credentials from different sessions.

A common bad state looks like this:

  • 'AWS_ACCESS_KEY_ID from one session'
  • 'AWS_SECRET_ACCESS_KEY from the same session'
  • 'AWS_SESSION_TOKEN from an older or unrelated session'

That combination is invalid even if each value looks syntactically plausible.

Start by Checking What the CLI Is Using

First identify the active credential source.

bash
aws configure list
aws sts get-caller-identity

If get-caller-identity fails with the same token error, that confirms the problem is your current authentication material, not S3 specifically.

Also inspect environment overrides:

bash
env | grep '^AWS_'

Environment variables take precedence over profile settings, so a stale AWS_SESSION_TOKEN in your shell can override otherwise-correct credentials in ~/.aws/credentials.

Temporary Credentials Must Match as a Set

When using STS-issued or assumed-role credentials, three values belong together:

  • access key ID
  • secret access key
  • session token

If any one of them is missing or copied from another session, the token becomes invalid.

For example, these must all come from the same assumed-role response:

bash
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=...

Clearing only one of them can create a broken mixed state.

Refresh the Credentials Cleanly

If you are using temporary credentials, the cleanest fix is usually to refresh the whole session instead of editing one variable manually.

For a named profile, try:

bash
aws sso login --profile myprofile
aws sts get-caller-identity --profile myprofile
aws s3 ls --profile myprofile

If you use assumed-role scripts or external tooling, rerun that workflow so all three values are regenerated together.

If the issue is a stale shell environment, clear the AWS variables and retry with the intended profile:

bash
1unset AWS_ACCESS_KEY_ID
2unset AWS_SECRET_ACCESS_KEY
3unset AWS_SESSION_TOKEN
4aws s3 ls --profile myprofile

That often resolves the error immediately when the shell environment was overriding correct profile settings.

Check for Clock and Expiry Problems

Temporary credentials expire. If a token has passed its validity window, AWS rejects it.

That means two practical checks matter:

  • confirm the credential session is still active
  • confirm the local machine clock is sane

A badly skewed local clock is less common, but it can make otherwise valid temporary credentials look invalid to AWS services.

Containers and CI Make This Easier to Misconfigure

In local shells, the problem is often stale environment variables. In CI, Docker, or Kubernetes, the problem is often mismatched secret injection.

For example, a container may receive only two of the three required temporary-credential fields. Or an old secret may still be mounted after the assumed-role session expired.

In those environments, verify exactly what the process sees:

bash
printenv | grep '^AWS_'

Then compare that with the intended credential source for the job.

Do Not Chase S3 Permissions First

This error is easy to misdiagnose as an S3 IAM policy problem. It usually is not.

If the token is malformed or invalid, the request fails before bucket permissions are meaningfully evaluated. So do not spend time editing s3:ListAllMyBuckets policies until aws sts get-caller-identity works cleanly.

That command is the fastest sanity check because it isolates the identity layer from the service-specific action.

Common Pitfalls

The biggest mistake is fixing only one credential field by hand. Temporary credentials come as a matched set, so partial edits often leave the CLI in an inconsistent state.

Another mistake is forgetting that environment variables override named profiles. A stale AWS_SESSION_TOKEN in the shell can silently break an otherwise-correct profile.

Teams also often debug S3 policies first when the real problem is invalid authentication material. If sts get-caller-identity fails, stop debugging S3 and fix credentials.

Finally, do not overlook expiring SSO or assumed-role sessions. A credential set that worked this morning may simply be expired now.

Summary

  • 'InvalidToken on aws s3 ls usually means the CLI is using broken temporary credentials.'
  • Start with aws configure list and aws sts get-caller-identity.
  • Make sure access key, secret key, and session token all come from the same session.
  • Clear stale environment variables when they override the correct profile.
  • Refresh the whole credential session instead of manually patching one value.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design