AWS
Python
Boto3
S3
Cloud Storage

how to copy s3 object from one bucket to another using python boto3

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Copying an object from one S3 bucket to another with boto3 is straightforward once you know the source bucket, source key, destination bucket, and destination key. The main things that trip people up are permissions, metadata behavior, and choosing the right API for large objects.

The Basic Copy Operation

A common approach is to use the S3 resource with the underlying client copy helper. You provide a CopySource description for the original object and a destination bucket and key for the copy.

python
1import boto3
2from botocore.exceptions import ClientError
3
4s3 = boto3.resource("s3")
5
6source = {
7    "Bucket": "source-bucket-name",
8    "Key": "reports/2026-03.csv",
9}
10
11destination_bucket = "archive-bucket-name"
12destination_key = "monthly/2026-03.csv"
13
14try:
15    s3.meta.client.copy(source, destination_bucket, destination_key)
16    print("Copy complete")
17except ClientError as exc:
18    print(f"Copy failed: {exc}")

This is the normal answer when you want a server-side copy inside S3. The data is copied by AWS rather than downloaded to your machine and uploaded again.

Copying Within the Same Bucket

The exact same technique works if the source and destination are in the same bucket and only the key changes.

python
1source = {
2    "Bucket": "my-bucket",
3    "Key": "incoming/photo.jpg",
4}
5
6s3.meta.client.copy(source, "my-bucket", "processed/photo.jpg")

That is useful for workflows that move objects between prefixes such as incoming/, processed/, and archive/.

Why copy Is Often Better Than Manual Download and Upload

A beginner approach is sometimes:

  1. Download the object locally.
  2. Upload it to the destination bucket.

That works, but it is wasteful when S3 can perform the copy internally. Using the S3 copy API is faster, cheaper in local resources, and simpler because your application does not have to stream the object bytes itself.

It is also a better choice for large objects because the helper can use multipart copy behavior under the hood.

Preserving or Replacing Metadata

A copy does not always preserve metadata the way people expect. If you use low-level options that replace metadata, you must explicitly provide the values you want to keep.

Here is an example with copy_from when you want to set new metadata intentionally:

python
1import boto3
2
3s3 = boto3.resource("s3")
4obj = s3.Object("destination-bucket", "docs/report.pdf")
5
6obj.copy_from(
7    CopySource={"Bucket": "source-bucket", "Key": "docs/report.pdf"},
8    Metadata={"owner": "finance"},
9    MetadataDirective="REPLACE"
10)

If you do not intend to replace metadata, be careful with options that change the directive.

Permissions You Need

The credentials used by boto3 must be allowed to read the source object and write the destination object.

In practice, that usually means permissions such as:

  • 's3:GetObject on the source key'
  • 's3:PutObject on the destination key'

If buckets use KMS encryption, you may also need the relevant KMS permissions.

When copy operations fail with AccessDenied, the code is often fine and the IAM policy is the real issue.

Handling Versioned or Special Cases

If the source bucket is versioned and you need a specific version, include VersionId in the copy source description.

python
1source = {
2    "Bucket": "source-bucket",
3    "Key": "reports/summary.csv",
4    "VersionId": "3Lgk...example"
5}
6
7s3.meta.client.copy(source, "dest-bucket", "reports/summary.csv")

That ensures you copy the exact object version you intended rather than the latest one.

Common Pitfalls

A common mistake is confusing the bucket name with the object key. In S3, folders are just prefixes inside the key, so archive/2026-03.csv is still a single key string.

Another pitfall is assuming local file paths are involved. A server-side S3 copy does not require a temporary local file.

Metadata replacement is another common source of surprises. If you specify replacement options, make sure you include all metadata you still want.

Finally, do not ignore region, encryption, and permissions. Most failed copy operations come from AWS configuration issues, not from Python syntax problems.

Summary

  • Use S3 server-side copy through boto3 instead of downloading and re-uploading objects.
  • Provide the source bucket and key in CopySource, then specify the destination bucket and key.
  • 's3.meta.client.copy is a practical default for most copy operations.'
  • Check IAM and KMS permissions when the copy fails.
  • Be careful with metadata and versioned objects if the copy needs exact fidelity.

Course illustration
Course illustration

All Rights Reserved.