Listing just the sub folders in an s3 bucket
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
Amazon S3 does not have real directories. It has object keys in a flat namespace, and "folders" are usually just common key prefixes separated by /. To list only the apparent subfolders, you need to ask S3 for common prefixes by using a delimiter.
Use list-objects-v2 with a Delimiter
The key CLI pattern is:
With --delimiter /, S3 groups matching keys and returns CommonPrefixes instead of dumping every object below the prefix.
If you want the top-level pseudo-folders in the bucket, use an empty or omitted prefix:
Example Response Shape
Conceptually, if the bucket contains keys like:
then querying with:
returns common prefixes similar to:
Those are the pseudo-subfolders under photos/.
Return Only the Prefix Strings
To print just the folder-like values, add a query:
This is often the most convenient CLI form for shell scripts.
Boto3 Example
The same idea works in Python with boto3:
This is the normal approach when you need folder-like listing behavior inside an application rather than a shell command.
Remember That Empty Folders Are Not Real
S3 does not store folders as first-class objects unless you explicitly create placeholder keys such as photos/ or photos/2024/. That means "folder existence" is usually inferred from the keys below it.
This matters because:
- a folder appears only if objects under that prefix exist
- deleting the last object under a prefix may make the folder seem to vanish
- folder behavior is a client-side convention, not a storage primitive
Once you internalize that model, the CommonPrefixes response becomes much easier to reason about.
Prefix Choice Controls the Result
If you want direct children under photos/2024/, use:
If you forget the trailing slash and use photos/2024, you may match keys that share the text prefix but are not in the pseudo-folder you intended.
That detail becomes especially important in automation, where one slightly wrong prefix can make a script process a much larger logical subtree than expected. It is worth printing the exact prefix during debugging so the request scope is obvious.
Common Pitfalls
The biggest mistake is expecting aws s3 ls or list-objects-v2 without a delimiter to return only folders. Without --delimiter /, you are listing objects, not grouped prefixes.
Another issue is forgetting that S3 folders are virtual. The result depends on key naming, not on a true directory tree.
A third problem is using the wrong prefix, especially missing the trailing slash when you intend to target one pseudo-folder level.
Summary
- S3 folders are simulated through key prefixes, not stored as real directories.
- Use
list-objects-v2with--delimiter /to list pseudo-subfolders. - Read folder-like results from
CommonPrefixes. - Choose the prefix carefully to get the level you actually want.
- The same prefix-and-delimiter model works in boto3 as well as the AWS CLI.
Related reading
- Load S3 Data into AWS SageMaker Notebook
- Load S3 Data into AWS SageMaker Notebook
- Local cloud stack for Azure similar to LocalStack for AWS?
- Local replica of RDS database
- Locked myself out of SSH with UFW in EC2 AWS
- log to specific logstream in cloudwatch from lambda
- Logs complaining extensions/v1beta1 Ingress is deprecated
- Make a bucket public in Amazon S3

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.