Introduction

Amazon S3 looks like a filesystem when you browse it, but it is really an object store indexed by exact object keys. That distinction matters because S3 does not support true server-side wildcard matching like a shell glob over arbitrary path segments.

What S3 actually supports

The S3 listing API supports prefix-based filtering, not general wildcard search. In practice, that means you can efficiently ask for objects starting with a known prefix such as logs/2025/09/, but you cannot ask S3 itself for "all keys matching logs/*/error-*.json" in one native query.

The usual pattern is:

1. narrow the search with the longest useful prefix
2. list matching objects from S3
3. apply wildcard or regex filtering on the client side

A practical Python example with boto3

Here is a simple way to do that with fnmatch:

1import boto3
2import fnmatch
3
4
5s3 = boto3.client("s3")
6bucket = "my-bucket"
7prefix = "logs/2025/"
8pattern = "logs/2025/*/error-*.json"
9
10paginator = s3.get_paginator("list_objects_v2")
11for page in paginator.paginate(Bucket=bucket, Prefix=prefix):
12    for obj in page.get("Contents", []):
13        key = obj["Key"]
14        if fnmatch.fnmatch(key, pattern):
15            print(key)

This is the normal answer for wildcard-style searching in S3. The prefix keeps the list operation smaller, and fnmatch handles the wildcard logic locally.

Using the AWS CLI

The AWS CLI has include and exclude filters that feel wildcard-like, but they still work after object listing, not as a general server-side wildcard query.

aws s3 ls s3://my-bucket/logs/2025/ --recursive | grep 'error-.*\.json'

Or with cp and filter flags:

1aws s3 cp s3://my-bucket/logs/ ./logs/ \
2  --recursive \
3  --exclude "*" \
4  --include "*/error-*.json"

That is useful operationally, but it does not change the underlying S3 limitation: filtering is still effectively built around listing plus client-side matching.

Choosing a good key structure

If you need frequent wildcard-like lookups, the real optimization is not a cleverer wildcard. It is better object-key design.

For example, these keys are much easier to search efficiently:

logs/2025/09/11/error-001.json
logs/2025/09/11/error-002.json
logs/2025/09/12/error-001.json

because you can use strong prefixes such as logs/2025/09/11/ and avoid scanning unrelated data.

If instead your keys hide important searchable information in the middle or at the end, every lookup becomes more expensive.

Large-scale alternatives

For large buckets, repeatedly listing keys and filtering in application code can become slow and costly. In that case, consider more scalable indexing approaches such as:

• S3 Inventory for regular key exports
• Athena queries over inventory data
• metadata catalogs stored outside S3
• application-maintained indexes in DynamoDB or another database

Those approaches make sense when the bucket is large enough that naive list-and-filter operations are no longer practical.

Common Pitfalls

The biggest mistake is assuming shell-style wildcards are a first-class S3 feature. They are not.

Another issue is using an empty or very broad prefix. That forces your code to enumerate far more objects than necessary and can be slow on large buckets.

It is also easy to forget pagination. list_objects_v2 does not return the whole bucket in one response when many objects exist.

Finally, do not confuse the convenience filters in the AWS CLI with efficient server-side wildcard support. They are useful tools, but they do not eliminate the need for careful prefix selection.

Summary

• S3 supports prefix filtering, not arbitrary server-side wildcard search.
• The normal solution is prefix filtering plus client-side wildcard matching.
• 'boto3 with fnmatch is a practical way to search keys by pattern.'
• Good object-key design makes wildcard-like lookup much cheaper.
• For very large buckets, use inventory or an external index instead of repeated broad scans.