Listing contents of a bucket with boto3
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
Listing the contents of an S3 bucket with boto3 is straightforward for small buckets, but the details matter once you deal with prefixes, pagination, or large result sets. The main API to know is list_objects_v2, and the main operational rule is that S3 object listings are paginated, so code that assumes one response contains everything will eventually break.
Start With list_objects_v2
The basic boto3 client call looks like this:
This is fine for quick experiments and small buckets, but it returns only up to a limited page of results. That is why it should be treated as a starting point, not as the final production pattern.
Use Prefixes to Narrow the Listing
S3 does not have real directories, but prefixes are often used to simulate them. If you only want a logical folder, specify a prefix.
This is more efficient and easier to reason about than listing the whole bucket and filtering in Python afterward.
Handle Pagination Properly
The correct scalable solution is to use a paginator.
This is the pattern you should reach for if the bucket can contain many objects. It also makes the code future-proof, because you are no longer assuming a single response page.
Include More Than Just the Key
The objects in Contents contain metadata that is often useful, such as size and modification time.
That is often enough for inventory scripts, data-lake inspection, or cleanup jobs.
Handle Empty Buckets and Permissions Explicitly
A listing call can return no Contents key at all when the bucket or prefix has no objects. That is why response.get("Contents", []) is better than indexing directly into the dictionary.
You also need permission to list objects. The IAM action is typically s3:ListBucket on the bucket itself. Without it, the call can fail even if you have object-level read permissions.
That is a common surprise when object downloads work but bucket listing does not.
Resource API Versus Client API
boto3 also offers the resource interface, which can read more naturally for simple scripts.
This is often pleasant for one-off scripts, though many teams prefer the client API for explicitness and closer alignment with AWS service operations.
Common Pitfalls
The biggest mistake is assuming list_objects_v2 returns every object in one call. Another is listing the whole bucket when a prefix would be enough.
Developers also often forget that an empty prefix may mean Contents is absent rather than present as an empty list.
Finally, bucket listing requires the correct IAM permission at the bucket level. If listing fails with access errors, check s3:ListBucket first rather than debugging boto3 itself.
Summary
- Use
list_objects_v2to list objects in an S3 bucket. - Use
Prefixto narrow the listing to a logical path. - Use a paginator for any bucket that might contain many objects.
- Read object metadata from
Contentswhen key names alone are not enough. - Handle empty results and bucket-level IAM permissions explicitly.
Related reading
- Listing files in a specific folder of a AWS S3 bucket
- Listing just the sub folders in an s3 bucket
- Load S3 Data into AWS SageMaker Notebook
- Load S3 Data into AWS SageMaker Notebook
- Lists Count vs Count
- Lists in ConfigParser
- Local cloud stack for Azure similar to LocalStack for AWS?
- Local replica of RDS database

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.