S3
boto3
file_upload
put_object
file transfer

What is the Difference between file_upload and put_object when uploading files to S3 using boto3

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

In boto3, the comparison people usually mean is upload_file versus put_object. Both can place an object into S3, but they operate at different levels of abstraction. put_object is the low-level API for sending object content directly, while upload_file is a higher-level managed transfer helper that is usually better for local files and large uploads.

The Low-Level Call: put_object

put_object maps closely to the S3 API. You provide a bucket, a key, and a body, and boto3 sends the request.

python
1import boto3
2
3s3 = boto3.client("s3")
4
5s3.put_object(
6    Bucket="my-bucket",
7    Key="notes/hello.txt",
8    Body=b"hello world",
9    ContentType="text/plain",
10)

This is a good fit when:

  • the content is already in memory
  • you are generating bytes dynamically
  • you want direct control over request parameters

For example, if your application builds a CSV report in memory, put_object is a natural API because there may not be any local file on disk to upload.

The Managed Helper: upload_file

upload_file is designed for uploading a file from disk. You give it a local path, a bucket, and a key, and boto3 handles the file transfer for you.

python
1import boto3
2
3s3 = boto3.client("s3")
4
5s3.upload_file(
6    "report.csv",
7    "my-bucket",
8    "exports/report.csv",
9)

This helper is usually the better choice when:

  • you already have a file on disk
  • the file might be large
  • you want boto3 to manage multipart upload behavior automatically

That last point is the big distinction. Managed transfers can split large files into parts and upload them efficiently without you implementing that logic manually.

Why Large Files Change the Recommendation

For small objects, both approaches can work. For larger files, upload_file usually wins because it is built on boto3's transfer manager.

Benefits of upload_file include:

  • automatic multipart upload when appropriate
  • retry handling for transfer parts
  • less memory pressure because the file can be streamed from disk

With put_object, the Body is a single request payload. That is perfectly fine for small generated content, but it is not the helper most people want for multi-gigabyte local files.

A Practical Side-by-Side Example

Suppose you have two cases.

Case one: create a JSON document in memory and write it to S3.

python
1import boto3
2import json
3
4s3 = boto3.client("s3")
5
6payload = json.dumps({"status": "ok", "count": 3}).encode("utf-8")
7
8s3.put_object(
9    Bucket="my-bucket",
10    Key="status/report.json",
11    Body=payload,
12    ContentType="application/json",
13)

Case two: upload a local video file.

python
1import boto3
2
3s3 = boto3.client("s3")
4
5s3.upload_file(
6    "video.mp4",
7    "my-bucket",
8    "media/video.mp4",
9)

The first case naturally fits put_object. The second naturally fits upload_file.

Metadata and Extra Arguments

Both approaches can attach metadata and headers, but the shape is slightly different.

With put_object, many attributes are top-level keyword arguments:

python
1s3.put_object(
2    Bucket="my-bucket",
3    Key="docs/readme.txt",
4    Body=b"hello",
5    ContentType="text/plain",
6    Metadata={"source": "script"},
7)

With upload_file, extra object settings are passed through ExtraArgs:

python
1s3.upload_file(
2    "readme.txt",
3    "my-bucket",
4    "docs/readme.txt",
5    ExtraArgs={
6        "ContentType": "text/plain",
7        "Metadata": {"source": "script"},
8    },
9)

So upload_file does not prevent advanced configuration. It just wraps the upload path in a transfer-oriented interface.

A Note on upload_fileobj

There is also a middle ground: upload_fileobj. It accepts a file-like object rather than a file path.

python
1import boto3
2import io
3
4s3 = boto3.client("s3")
5buffer = io.BytesIO(b"hello from memory")
6
7s3.upload_fileobj(buffer, "my-bucket", "notes/memory.txt")

This is helpful when you want the managed-transfer behavior but your data is not stored in a named local file.

Choosing the Right API

Use put_object when the payload is already available as bytes or a stream-like body and you want a direct S3 operation.

Use upload_file when the source is a local file and you want boto3 to handle multipart upload logic, retries, and transfer management.

If you find yourself reading a local file into memory just to pass it to put_object, that is often a sign upload_file would be a better fit.

Common Pitfalls

  • Comparing put_object to a method called file_upload. In boto3, the usual helper name is upload_file.
  • Using put_object for large local files when upload_file would handle multipart transfer more cleanly.
  • Reading an entire file into memory unnecessarily before uploading it.
  • Assuming upload_file is less configurable. You can still pass headers and metadata through ExtraArgs.
  • Choosing an API based only on "can it upload" rather than on the source of the data and transfer size.

Summary

  • 'put_object is the lower-level S3 call and is ideal for in-memory content or direct request control.'
  • 'upload_file is the higher-level managed helper for files on disk.'
  • Large file uploads usually favor upload_file because multipart handling is built in.
  • Both APIs can set metadata and content headers.
  • Pick the API that matches where the data comes from and how much transfer management you want boto3 to handle.

Course illustration
Course illustration

All Rights Reserved.