Boto
Amazon S3
file upload
Python programming
cloud storage

Boto - Uploading file to a specific location on Amazon S3

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

In S3, the "location" of a file is not a real folder on a filesystem. It is the object key inside a bucket. So uploading to a specific location means choosing the right bucket and the right key, such as reports/2026/january/data.csv, when you call the boto client or resource API.

Understand the S3 path model

An S3 object is identified by:

  • the bucket name
  • the object key

What looks like folders in the console is just key prefix naming. For example, this object:

text
s3://my-bucket/images/avatars/user-42.png

really means:

  • bucket: my-bucket
  • key: images/avatars/user-42.png

So the way to upload to a specific "location" is to pass that full key explicitly.

Use boto3 upload_file with the desired key

In modern Python code, use boto3:

python
1import boto3
2
3s3 = boto3.client("s3")
4
5local_path = "avatar.png"
6bucket = "my-bucket"
7key = "images/avatars/user-42.png"
8
9s3.upload_file(local_path, bucket, key)

That uploads avatar.png into the bucket under the given key. If the prefix does not already appear in the S3 console, that is fine. S3 does not require folder creation first.

Build keys deliberately

It is common to generate the key dynamically from application data:

python
1from pathlib import Path
2import boto3
3
4s3 = boto3.client("s3")
5
6user_id = 42
7local_path = Path("avatar.png")
8key = f"images/avatars/user-{user_id}/{local_path.name}"
9
10s3.upload_file(str(local_path), "my-bucket", key)

This is often safer than string-copying S3 paths all over the codebase. Treat the key as part of your storage design, not as an afterthought.

Add metadata or content type when needed

If you need content type, ACL, or metadata, pass ExtraArgs:

python
1s3.upload_file(
2    "report.csv",
3    "my-bucket",
4    "reports/2026/report.csv",
5    ExtraArgs={
6        "ContentType": "text/csv",
7        "Metadata": {"source": "batch-job"}
8    }
9)

This matters because "uploaded to the right key" is not always enough. Downstream consumers may also depend on correct metadata.

Handle errors instead of assuming the upload worked

Production code should catch AWS and file-system problems explicitly:

python
1import boto3
2from botocore.exceptions import BotoCoreError, ClientError
3
4s3 = boto3.client("s3")
5
6try:
7    s3.upload_file("report.csv", "my-bucket", "reports/2026/report.csv")
8    print("Upload succeeded")
9except FileNotFoundError:
10    print("Local file does not exist")
11except (BotoCoreError, ClientError) as exc:
12    print(f"S3 upload failed: {exc}")

Common failures include missing credentials, insufficient IAM permission, a bad bucket name, or uploading to a region you did not expect.

Remember that prefixes are not directories

This is the S3 misunderstanding that causes the most confusion. You do not create a folder and then upload into it the way you would on a local disk. The key itself defines the apparent path.

So this:

python
key = "backup/2026/01/data.json"

already is the "location." There is no separate folder-creation step needed.

Common Pitfalls

The most common mistake is treating S3 prefixes as real folders and looking for an API call to create them first. Usually you do not need one.

Another common issue is confusing the local filename with the S3 key. The third argument to upload_file is where the object will live in the bucket.

People also forget that successful upload may still require metadata or content-type settings for correct downstream behavior.

Finally, if the upload fails, check IAM permissions and credentials before assuming the key path itself is wrong.

Summary

  • In S3, a specific "location" means a bucket plus an object key.
  • Use boto3.client("s3").upload_file(local_path, bucket, key) to upload to that key.
  • Prefixes such as reports/2026/ are part of the key, not real folders.
  • Build object keys deliberately and pass metadata when needed.
  • Handle IAM, credentials, and local-file errors explicitly in production code.

Course illustration
Course illustration

All Rights Reserved.