AWS
file upload
cloud storage
local machine
data transfer

Uploading file to AWS from local machine

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

When people say "upload a file to AWS from my local machine", they usually mean uploading to Amazon S3. The most common tools are the AWS CLI for ad hoc transfers and an SDK such as Boto3 when the upload is part of an application or script.

The Simplest Path: aws s3 cp

For a one-off upload, the AWS CLI is usually enough. First configure credentials:

bash
aws configure

Then upload the file:

bash
aws s3 cp ./report.csv s3://my-bucket/reports/report.csv

This command needs:

  • valid AWS credentials
  • permission such as s3:PutObject
  • a bucket that already exists

You can also upload a whole directory:

bash
aws s3 cp ./build s3://my-bucket/build --recursive

If you want directory synchronization instead of blind copying, use sync:

bash
aws s3 sync ./build s3://my-bucket/build

Understanding The S3 Path

An S3 object path is:

  • bucket name: my-bucket
  • key: reports/report.csv

There are no real folders inside S3. The key just uses slashes as naming convention. That matters when you upload, because the exact key name controls where the object appears in the console and how other systems reference it.

Use Boto3 In Python Scripts

If the upload belongs inside an application or automation script, Boto3 is the standard Python choice.

python
1import boto3
2
3s3 = boto3.client("s3")
4
5s3.upload_file(
6    Filename="report.csv",
7    Bucket="my-bucket",
8    Key="reports/report.csv",
9)

This uses the same AWS credential chain as other SDK tools, including environment variables, shared config files, or role-based credentials.

You can attach metadata or content type too:

python
1s3.upload_file(
2    Filename="image.png",
3    Bucket="my-bucket",
4    Key="images/image.png",
5    ExtraArgs={
6        "ContentType": "image/png",
7        "Metadata": {"source": "local-machine"}
8    },
9)

That is useful when the object will be served directly or processed later.

Choose Credentials Carefully

For local development, the most common options are:

  • 'aws configure'
  • environment variables such as AWS_ACCESS_KEY_ID
  • AWS SSO or IAM Identity Center login
  • temporary credentials from assuming a role

Avoid embedding long-lived credentials directly into scripts. If you are working inside an organization, temporary role-based credentials are usually better.

Verify The Upload

After uploading, confirm the object exists:

bash
aws s3 ls s3://my-bucket/reports/

Or inspect it in Python:

python
response = s3.head_object(Bucket="my-bucket", Key="reports/report.csv")
print(response["ContentLength"])

Verification is especially important in automation, because a script can fail partway through or upload to the wrong key prefix.

Common Reasons Uploads Fail

A few errors appear again and again:

  • 'AccessDenied: the identity lacks s3:PutObject or bucket policy access'
  • 'NoSuchBucket: the bucket name is wrong'
  • region mismatch or incorrect endpoint assumptions
  • uploading to a key you did not intend because the destination path was mistyped

If the bucket uses server-side encryption or strict bucket policies, you may also need extra arguments:

bash
aws s3 cp ./report.csv s3://my-bucket/reports/report.csv \
  --sse AES256

Or in Boto3:

python
1s3.upload_file(
2    "report.csv",
3    "my-bucket",
4    "reports/report.csv",
5    ExtraArgs={"ServerSideEncryption": "AES256"},
6)

When Not To Upload Directly

If you are building a user-facing application, you may not want the client machine to have AWS write credentials at all. In that case, generate a presigned URL on a backend and upload through that URL instead.

That approach is safer when:

  • users upload files from browsers
  • you want time-limited permissions
  • you do not want to distribute AWS credentials

Common Pitfalls

  • Assuming "AWS upload" always means EC2 or some generic AWS endpoint instead of S3.
  • Using the wrong S3 key and later thinking the file was lost.
  • Hardcoding long-lived AWS credentials in local scripts.
  • Forgetting required permissions such as s3:PutObject.
  • Using cp --recursive when sync would better match the desired behavior.

Summary

  • For manual uploads, aws s3 cp is the simplest tool.
  • For scripted uploads, Boto3 is the standard Python SDK option.
  • Make sure credentials, bucket name, region, and object key are correct.
  • Verify the upload with aws s3 ls or head_object.
  • For end-user uploads, consider presigned URLs instead of distributing AWS credentials.

Course illustration
Course illustration

All Rights Reserved.