boto3
AWS
S3
Python
cloud storage

Listing contents of a bucket with boto3

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

Listing the contents of an S3 bucket with boto3 is straightforward for small buckets, but the details matter once you deal with prefixes, pagination, or large result sets. The main API to know is list_objects_v2, and the main operational rule is that S3 object listings are paginated, so code that assumes one response contains everything will eventually break.

Start With list_objects_v2

The basic boto3 client call looks like this:

python
1import boto3
2
3s3 = boto3.client("s3")
4response = s3.list_objects_v2(Bucket="my-bucket")
5
6for obj in response.get("Contents", []):
7    print(obj["Key"])

This is fine for quick experiments and small buckets, but it returns only up to a limited page of results. That is why it should be treated as a starting point, not as the final production pattern.

Use Prefixes to Narrow the Listing

S3 does not have real directories, but prefixes are often used to simulate them. If you only want a logical folder, specify a prefix.

python
1import boto3
2
3s3 = boto3.client("s3")
4response = s3.list_objects_v2(Bucket="my-bucket", Prefix="logs/2026/")
5
6for obj in response.get("Contents", []):
7    print(obj["Key"])

This is more efficient and easier to reason about than listing the whole bucket and filtering in Python afterward.

Handle Pagination Properly

The correct scalable solution is to use a paginator.

python
1import boto3
2
3s3 = boto3.client("s3")
4paginator = s3.get_paginator("list_objects_v2")
5
6for page in paginator.paginate(Bucket="my-bucket"):
7    for obj in page.get("Contents", []):
8        print(obj["Key"])

This is the pattern you should reach for if the bucket can contain many objects. It also makes the code future-proof, because you are no longer assuming a single response page.

Include More Than Just the Key

The objects in Contents contain metadata that is often useful, such as size and modification time.

python
1import boto3
2
3s3 = boto3.client("s3")
4response = s3.list_objects_v2(Bucket="my-bucket")
5
6for obj in response.get("Contents", []):
7    print(obj["Key"], obj["Size"], obj["LastModified"])

That is often enough for inventory scripts, data-lake inspection, or cleanup jobs.

Handle Empty Buckets and Permissions Explicitly

A listing call can return no Contents key at all when the bucket or prefix has no objects. That is why response.get("Contents", []) is better than indexing directly into the dictionary.

You also need permission to list objects. The IAM action is typically s3:ListBucket on the bucket itself. Without it, the call can fail even if you have object-level read permissions.

That is a common surprise when object downloads work but bucket listing does not.

Resource API Versus Client API

boto3 also offers the resource interface, which can read more naturally for simple scripts.

python
1import boto3
2
3s3 = boto3.resource("s3")
4bucket = s3.Bucket("my-bucket")
5
6for obj in bucket.objects.filter(Prefix="logs/"):
7    print(obj.key)

This is often pleasant for one-off scripts, though many teams prefer the client API for explicitness and closer alignment with AWS service operations.

Common Pitfalls

The biggest mistake is assuming list_objects_v2 returns every object in one call. Another is listing the whole bucket when a prefix would be enough.

Developers also often forget that an empty prefix may mean Contents is absent rather than present as an empty list.

Finally, bucket listing requires the correct IAM permission at the bucket level. If listing fails with access errors, check s3:ListBucket first rather than debugging boto3 itself.

Summary

  • Use list_objects_v2 to list objects in an S3 bucket.
  • Use Prefix to narrow the listing to a logical path.
  • Use a paginator for any bucket that might contain many objects.
  • Read object metadata from Contents when key names alone are not enough.
  • Handle empty results and bucket-level IAM permissions explicitly.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.