Introduction

To get the last modified object from an S3 bucket using the AWS CLI, use s3api list-objects-v2 with a JMESPath query that sorts by LastModified and selects the final element:

aws s3api list-objects-v2 \
  --bucket my-bucket \
  --query "sort_by(Contents, &LastModified)[-1].{Key: Key, LastModified: LastModified}"

This returns a single JSON object with the key and timestamp of the most recently modified object in the bucket. The rest of this article covers prefix filtering, pagination for large buckets, downloading the result, and scripting the whole workflow.

Prerequisites

Before running these commands, ensure the AWS CLI is installed and configured:

1# Check CLI version (v2 recommended)
2aws --version
3
4# Configure credentials
5aws configure

The IAM principal you authenticate with needs at minimum the s3:ListBucket permission on the target bucket, and s3:GetObject if you intend to download the result.

Step 1: List Objects Sorted by LastModified

The core command uses the --query parameter to apply a JMESPath expression server-side:

aws s3api list-objects-v2 \
  --bucket my-bucket \
  --query "sort_by(Contents, &LastModified)[-1].{Key: Key, LastModified: LastModified, Size: Size}"

Example output:

1{
2    "Key": "logs/2024-03-15/app.log.gz",
3    "LastModified": "2024-03-15T14:22:08+00:00",
4    "Size": 4521984
5}

Understanding the JMESPath Expression

Step 2: Filter by Prefix

Most real-world buckets organize objects under prefixes (pseudo-directories). To find the most recent object under a specific prefix:

1aws s3api list-objects-v2 \
2  --bucket my-bucket \
3  --prefix "logs/production/" \
4  --query "sort_by(Contents, &LastModified)[-1].{Key: Key, LastModified: LastModified}"

The --prefix flag tells S3 to return only objects whose keys start with logs/production/, which dramatically reduces the response size for large buckets.

Step 3: Download the Most Recent Object

To download the object in a single pipeline, capture the key and pass it to s3 cp:

1LATEST_KEY=$(aws s3api list-objects-v2 \
2  --bucket my-bucket \
3  --prefix "backups/" \
4  --query "sort_by(Contents, &LastModified)[-1].Key" \
5  --output text)
6
7aws s3 cp "s3://my-bucket/$LATEST_KEY" ./latest-backup
8echo "Downloaded: $LATEST_KEY"

The --output text flag strips the JSON quotes so the key can be used directly in the s3 cp command.

Handling Large Buckets With Pagination

list-objects-v2 returns a maximum of 1,000 objects per request. For buckets with more objects, you need pagination. The --page-size parameter controls how many objects are requested per API call, and the --max-items parameter limits the total number of objects returned.

However, for finding the most recent object across the entire bucket, you need all objects. Use the --no-paginate flag or let the CLI paginate automatically:

1aws s3api list-objects-v2 \
2  --bucket my-bucket \
3  --query "sort_by(Contents, &LastModified)[-1].{Key: Key, LastModified: LastModified}" \
4  --no-paginate

For very large buckets (millions of objects), this approach becomes slow. A better strategy is to use S3 Inventory or narrow the search with a specific prefix.

Alternative: Using s3 ls With Sort

For a quick-and-dirty approach, you can use s3 ls combined with sort and tail:

aws s3 ls s3://my-bucket/logs/ --recursive | sort | tail -1

This works because s3 ls output includes timestamps in a sortable format. However, it downloads all object listings as text, which is less efficient than the JMESPath approach for large buckets.

Scripting a Complete Workflow

Here is a complete bash script that finds and downloads the latest object, with error handling:

1#!/bin/bash
2set -euo pipefail
3
4BUCKET="my-bucket"
5PREFIX="data/exports/"
6DEST_DIR="./downloads"
7
8# Find the latest object
9LATEST=$(aws s3api list-objects-v2 \
10  --bucket "$BUCKET" \
11  --prefix "$PREFIX" \
12  --query "sort_by(Contents, &LastModified)[-1].{Key: Key, LastModified: LastModified}" \
13  --output json)
14
15if [ "$LATEST" = "null" ] || [ -z "$LATEST" ]; then
16    echo "No objects found under s3://$BUCKET/$PREFIX"
17    exit 1
18fi
19
20KEY=$(echo "$LATEST" | python3 -c "import sys, json; print(json.load(sys.stdin)['Key'])")
21MODIFIED=$(echo "$LATEST" | python3 -c "import sys, json; print(json.load(sys.stdin)['LastModified'])")
22
23echo "Latest object: $KEY (modified: $MODIFIED)"
24
25# Download
26mkdir -p "$DEST_DIR"
27aws s3 cp "s3://$BUCKET/$KEY" "$DEST_DIR/"
28echo "Downloaded to $DEST_DIR/"

Using the Python SDK (boto3) Alternative

When the AWS CLI approach becomes unwieldy, the Python SDK offers more control:

1import boto3
2
3s3 = boto3.client("s3")
4
5paginator = s3.get_paginator("list_objects_v2")
6pages = paginator.paginate(Bucket="my-bucket", Prefix="logs/")
7
8latest = None
9for page in pages:
10    for obj in page.get("Contents", []):
11        if latest is None or obj["LastModified"] > latest["LastModified"]:
12            latest = obj
13
14if latest:
15    print(f"Latest: {latest['Key']} ({latest['LastModified']})")
16    s3.download_file("my-bucket", latest["Key"], "latest-file")

This handles pagination automatically and avoids the JMESPath complexity. It is the better approach for production automation scripts.

Command Reference

Common Pitfalls

Forgetting the --prefix filter on large buckets causes the CLI to enumerate every object, which can take minutes or hours and incur significant LIST API costs. Always scope the search to the narrowest prefix possible.

Assuming list-objects-v2 returns all objects in one call is incorrect for buckets with more than 1,000 objects. Without --no-paginate, the JMESPath query only operates on the first page of results, which may not contain the most recent object.

Using --query with sort_by on an empty prefix returns null when the bucket or prefix contains no objects. Scripts should check for null before attempting to download.

Confusing LastModified with creation time is a subtle issue. S3 updates LastModified when an object is overwritten, so the "most recently modified" object may not be the "newest" object by key name or logical sequence.

Not quoting the bucket name or prefix in scripts can cause word splitting issues in bash. Always use double quotes around variables.

Summary

• Use aws s3api list-objects-v2 with --query "sort_by(Contents, &LastModified)[-1]" to find the most recent object.
• Add --prefix to narrow the search to a specific path within the bucket.
• Use --output text when capturing the key for use in downstream commands like s3 cp.
• For buckets with more than 1,000 objects, ensure pagination is handled with --no-paginate or use the boto3 SDK.
• Always check for empty results (null) in scripts before attempting to process the key.
• Prefer prefix filtering over full-bucket scans to control both latency and API costs.