AWS S3
Boto3
File Upload
Python
Cloud Storage

uploading file to specific folder in S3 using boto3

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Amazon S3 does not have real folders in the filesystem sense. What looks like a folder in the console is just part of the object key, often called a prefix. That means uploading a file to a "folder" in S3 is really just uploading it with a key such as reports/2026/file.csv. Once that is clear, the Boto3 code becomes straightforward.

Build the S3 Key Correctly

The most important input is the object key. If you want a file to appear under a folder-like path, include that path in the key.

python
bucket_name = "my-example-bucket"
local_path = "data/report.csv"
s3_key = "reports/2026/report.csv"

When the file is uploaded with that key, the S3 console will display it under reports/2026/. No separate folder-creation step is required.

Upload with upload_file

For ordinary file uploads, upload_file is the simplest high-level API.

python
1import boto3
2
3s3 = boto3.client("s3")
4
5s3.upload_file(
6    Filename="data/report.csv",
7    Bucket="my-example-bucket",
8    Key="reports/2026/report.csv"
9)

This reads the local file and uploads it to the specified bucket and key. If the key already exists, the new upload replaces the old object.

Generate Prefixes Dynamically

In real applications, the folder-like path often depends on date, user ID, or document type. Build the key explicitly instead of concatenating strings carelessly.

python
1from pathlib import Path
2
3def make_s3_key(prefix: str, filename: str) -> str:
4    clean_prefix = prefix.strip("/")
5    clean_name = Path(filename).name
6    return f"{clean_prefix}/{clean_name}"
7
8key = make_s3_key("uploads/images", "/tmp/photo.png")
9print(key)

This protects you from malformed keys such as duplicate slashes or accidental leakage of local filesystem paths into the uploaded object name.

Add Metadata or Content Type When Needed

Sometimes the upload needs more than just bytes. You may want to set content type, server-side encryption, or access control settings.

python
1import boto3
2
3s3 = boto3.client("s3")
4
5s3.upload_file(
6    Filename="data/report.csv",
7    Bucket="my-example-bucket",
8    Key="reports/2026/report.csv",
9    ExtraArgs={
10        "ContentType": "text/csv",
11        "ServerSideEncryption": "AES256"
12    }
13)

Use ExtraArgs deliberately. It is easy to ignore metadata during initial development and then discover later that downstream consumers infer the wrong content type.

Prefer upload_fileobj for In-Memory Streams

If the file contents are already in memory or are being streamed, upload_fileobj may be a better fit than writing a temporary local file first.

python
1import boto3
2from io import BytesIO
3
4s3 = boto3.client("s3")
5buffer = BytesIO(b"hello from memory")
6
7s3.upload_fileobj(
8    Fileobj=buffer,
9    Bucket="my-example-bucket",
10    Key="messages/hello.txt"
11)

The S3 "folder" behavior is the same. The only difference is where the bytes come from.

Handle Errors Explicitly

Uploads can fail for reasons that have nothing to do with the key prefix:

  1. Missing credentials.
  2. Wrong bucket name or region assumptions.
  3. Missing s3:PutObject permission.
  4. Local file path errors.

A small error-handling wrapper makes those failures easier to diagnose.

python
1import boto3
2from botocore.exceptions import ClientError, NoCredentialsError
3
4def upload_to_prefix(local_path: str, bucket: str, key: str) -> None:
5    s3 = boto3.client("s3")
6    try:
7        s3.upload_file(local_path, bucket, key)
8    except FileNotFoundError:
9        print("Local file does not exist")
10    except NoCredentialsError:
11        print("AWS credentials are missing")
12    except ClientError as exc:
13        print(exc.response["Error"]["Message"])

That does not replace proper logging, but it makes the failure mode visible during development.

Think in Terms of Keys, Not Directories

S3 does not require empty folders, directory creation, or path traversal rules like a local disk. The object key is the truth. If the key is team/a/report.csv, the object belongs to that logical prefix. If the key is just report.csv, it appears at the bucket root.

Once you shift from "folder upload" thinking to "key construction" thinking, most S3 upload questions become much simpler.

Common Pitfalls

  • Trying to create S3 folders separately even though the prefix is just part of the object key.
  • Building keys with raw string concatenation and producing double slashes or incorrect filenames.
  • Forgetting that uploading to an existing key replaces the previous object.
  • Assuming upload failures are about prefixes when the real issue is credentials or IAM permissions.
  • Ignoring metadata such as content type when the uploaded file will be consumed by other systems.

Summary

  • In S3, the folder-like path is just part of the object key.
  • Use upload_file for local files and include the desired prefix in Key.
  • Build keys carefully so local path details do not leak into object names accidentally.
  • Use ExtraArgs when metadata or encryption settings matter.
  • Debug uploads by separating key construction issues from credential and permission issues.

Course illustration
Course illustration

All Rights Reserved.