Introduction

Slow S3 uploads are usually caused by the upload pattern, not by S3 being generically slow. The common bottlenecks are a single-threaded client, long network distance to the bucket, too many tiny files, or missing multipart upload for large objects.

That is good news because the fixes are concrete. Once you identify whether the problem is file size, latency, or client configuration, S3 uploads usually become much faster with straightforward transfer tuning.

Start by Identifying the Workload

Not all slow uploads are the same. Ask which of these you are doing:

• one or a few large files
• thousands of small files
• uploads from a distant region or remote geography
• uploads with client-side compression or encryption

Large files benefit from multipart upload and concurrency. Small files suffer more from request overhead than from bandwidth limits. Long-distance uploads are sensitive to latency even when raw throughput looks fine.

Use Multipart Upload for Large Files

For large objects, multipart upload is the standard improvement. It splits the file into parts, uploads parts in parallel, and retries failed parts independently.

With boto3, configure the transfer manager explicitly:

1import boto3
2from boto3.s3.transfer import TransferConfig
3
4s3 = boto3.client("s3")
5
6config = TransferConfig(
7    multipart_threshold=8 * 1024 * 1024,
8    multipart_chunksize=8 * 1024 * 1024,
9    max_concurrency=10,
10    use_threads=True,
11)
12
13s3.upload_file(
14    "large-video.mp4",
15    "my-bucket",
16    "uploads/large-video.mp4",
17    Config=config,
18)

This keeps multiple parts in flight and usually uses available bandwidth much better than a single-stream upload.

Tune Concurrency Instead of Guessing

Higher concurrency is helpful only up to the point your machine and network can sustain it. A weak laptop, container CPU limit, or narrow outbound link can actually perform worse if you push concurrency too high.

A practical approach is to start with a moderate value such as 8 or 10, benchmark, and then adjust. The right number depends on latency, bandwidth, file size, and client CPU.

Treat Small Files as a Different Problem

If your workload is many small files, multipart upload is not the main answer. Each file still costs request setup, TLS work, and metadata handling, so per-file overhead dominates.

Useful strategies include:

• bundle files into one archive
• upload several files in parallel
• avoid repeated session setup
• reduce unnecessary HEAD or LIST calls around the upload

For example:

tar -czf reports.tar.gz ./reports
aws s3 cp reports.tar.gz s3://my-bucket/uploads/reports.tar.gz

One larger upload is often much faster than thousands of tiny ones.

Check Region and Network Path

The bucket should be near the system doing the upload. If the application lives in one region and uploads constantly to a faraway bucket, round-trip latency taxes every request.

For geographically distant public clients, S3 Transfer Acceleration can help because traffic enters AWS through edge locations. It is not automatically the cheapest or best option, but it is worth testing when global uploads are consistently slow.

Compare Against the AWS CLI

A quick baseline test is to compare your application with the AWS CLI:

aws s3 cp large-video.mp4 s3://my-bucket/uploads/large-video.mp4

If the CLI is slow too, the problem is probably network path, region placement, or raw bandwidth. If the CLI is much faster, your code path likely needs better transfer settings or less client-side overhead.

Watch Client-Side Processing

Compression, encryption, checksumming, and file packaging can make the whole operation feel like an S3 problem even when the network is fine. Measure the full pipeline, not just the final SDK call.

Also make sure your program is not creating a new client for every file. Reusing the same SDK client and its connection pool is typically more efficient.

Common Pitfalls

The most common mistake is using a basic upload call for large files and expecting it to saturate the network without multipart concurrency.

Another common issue is trying to solve a small-file workload with large-file tuning. If request overhead is the bottleneck, batching is more important than part size.

People also increase concurrency blindly. Past a certain point, the client becomes CPU-bound or socket-bound and throughput stops improving.

Finally, do not ignore bucket region choice. A well-tuned uploader cannot completely hide avoidable latency from sending data to the wrong region.

Summary

• Use multipart upload with moderate concurrency for large S3 objects.
• For many small files, batch them or parallelize carefully to reduce request overhead.
• Keep the bucket close to the uploader or test S3 Transfer Acceleration for distant clients.
• Reuse the SDK client and compare performance against the AWS CLI.
• Measure whether the bottleneck is network, file pattern, or client-side processing before tuning settings.