Uploading file to AWS from local machine
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
When people say "upload a file to AWS from my local machine", they usually mean uploading to Amazon S3. The most common tools are the AWS CLI for ad hoc transfers and an SDK such as Boto3 when the upload is part of an application or script.
The Simplest Path: aws s3 cp
For a one-off upload, the AWS CLI is usually enough. First configure credentials:
Then upload the file:
This command needs:
- valid AWS credentials
- permission such as
s3:PutObject - a bucket that already exists
You can also upload a whole directory:
If you want directory synchronization instead of blind copying, use sync:
Understanding The S3 Path
An S3 object path is:
- bucket name:
my-bucket - key:
reports/report.csv
There are no real folders inside S3. The key just uses slashes as naming convention. That matters when you upload, because the exact key name controls where the object appears in the console and how other systems reference it.
Use Boto3 In Python Scripts
If the upload belongs inside an application or automation script, Boto3 is the standard Python choice.
This uses the same AWS credential chain as other SDK tools, including environment variables, shared config files, or role-based credentials.
You can attach metadata or content type too:
That is useful when the object will be served directly or processed later.
Choose Credentials Carefully
For local development, the most common options are:
- '
aws configure' - environment variables such as
AWS_ACCESS_KEY_ID - AWS SSO or IAM Identity Center login
- temporary credentials from assuming a role
Avoid embedding long-lived credentials directly into scripts. If you are working inside an organization, temporary role-based credentials are usually better.
Verify The Upload
After uploading, confirm the object exists:
Or inspect it in Python:
Verification is especially important in automation, because a script can fail partway through or upload to the wrong key prefix.
Common Reasons Uploads Fail
A few errors appear again and again:
- '
AccessDenied: the identity lackss3:PutObjector bucket policy access' - '
NoSuchBucket: the bucket name is wrong' - region mismatch or incorrect endpoint assumptions
- uploading to a key you did not intend because the destination path was mistyped
If the bucket uses server-side encryption or strict bucket policies, you may also need extra arguments:
Or in Boto3:
When Not To Upload Directly
If you are building a user-facing application, you may not want the client machine to have AWS write credentials at all. In that case, generate a presigned URL on a backend and upload through that URL instead.
That approach is safer when:
- users upload files from browsers
- you want time-limited permissions
- you do not want to distribute AWS credentials
Common Pitfalls
- Assuming "AWS upload" always means EC2 or some generic AWS endpoint instead of S3.
- Using the wrong S3 key and later thinking the file was lost.
- Hardcoding long-lived AWS credentials in local scripts.
- Forgetting required permissions such as
s3:PutObject. - Using
cp --recursivewhensyncwould better match the desired behavior.
Summary
- For manual uploads,
aws s3 cpis the simplest tool. - For scripted uploads, Boto3 is the standard Python SDK option.
- Make sure credentials, bucket name, region, and object key are correct.
- Verify the upload with
aws s3 lsorhead_object. - For end-user uploads, consider presigned URLs instead of distributing AWS credentials.
Related reading
- uploading file to specific folder in S3 using boto3
- uri for aws_api_gateway_integration for type AWS integration to DynamoDb
- URL for public Amazon S3 bucket
- Use Azure Machine learning to detect symbol within an image
- Use terraform to set up a lambda function triggered by a scheduled event source
- User-data scripts is not running on my custom AMI, but working in standard Amazon linux
- User is not authorized to perform cloudformationCreateStack
- User is not authorized to perform dynamodbPutItem on resource

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.