django
storages
S3 buckets
multiple buckets
cloud storage

django-storages with multiple S3 Buckets

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Using more than one S3 bucket in a Django project is a normal requirement, not an edge case. Teams often want one bucket for static assets, another for user uploads, and sometimes a third for private documents or backups. django-storages supports that cleanly once you stop thinking in terms of one global storage backend and start defining separate storage classes.

Why Multiple Buckets Help

Different file types usually have different operational needs.

  • Static files are often public and aggressively cached.
  • Media uploads are usually private or at least overwrite-sensitive.
  • Internal exports may need tighter IAM rules and shorter retention.

Putting all of those objects in one bucket makes permissions, lifecycle policies, and CDN behavior harder to manage. Separate buckets let you keep the rules simple.

Define Separate Storage Backends

The core pattern is to create a storage class for each bucket. That lets Django choose the right backend per use case instead of relying on one global DEFAULT_FILE_STORAGE.

python
1from storages.backends.s3 import S3Storage
2
3
4class StaticStorage(S3Storage):
5    bucket_name = "myapp-static-assets"
6    location = "static"
7    default_acl = "public-read"
8    file_overwrite = True
9
10
11class MediaStorage(S3Storage):
12    bucket_name = "myapp-user-media"
13    location = "media"
14    default_acl = "private"
15    file_overwrite = False

This is the most readable setup because the bucket choice is encoded in Python instead of being buried in conditionals.

Wire Static Files And Uploaded Files Separately

Static files and uploaded media are configured differently in Django.

python
1# settings.py
2
3STATIC_URL = "https://myapp-static-assets.s3.amazonaws.com/static/"
4STATICFILES_STORAGE = "myproject.storage_backends.StaticStorage"
5
6DEFAULT_FILE_STORAGE = "myproject.storage_backends.MediaStorage"
7MEDIA_URL = "https://myapp-user-media.s3.amazonaws.com/media/"

With this setup:

  • 'collectstatic writes to the static bucket,'
  • model FileField and ImageField uploads use the media bucket.

That covers many projects without any additional complexity.

Use A Dedicated Storage On Specific Models

Sometimes only one model or field should use a different bucket. In that case, pass a storage instance directly to the field.

python
1from django.db import models
2from myproject.storage_backends import MediaStorage
3
4
5private_media_storage = MediaStorage()
6
7
8class Profile(models.Model):
9    avatar = models.ImageField(storage=private_media_storage, upload_to="avatars/")

This is useful when most uploads go to one bucket but a specific document class needs different retention or access rules.

Django 4.2 And The STORAGES Setting

If your project uses newer Django settings conventions, you can define named storage configurations and keep the bucket details centralized.

python
1STORAGES = {
2    "default": {
3        "BACKEND": "myproject.storage_backends.MediaStorage",
4    },
5    "staticfiles": {
6        "BACKEND": "myproject.storage_backends.StaticStorage",
7    },
8}

This does not remove the need for separate storage classes. It just makes Django’s configuration style more explicit.

Credentials And Permissions Still Matter

Multiple buckets do not require multiple credential sets, but they do require IAM permissions that cover every bucket the app touches. A common deployment error is granting access to the media bucket and forgetting the static bucket.

If your app role needs both, the policy must reflect that.

json
1{
2  "Version": "2012-10-17",
3  "Statement": [
4    {
5      "Effect": "Allow",
6      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
7      "Resource": [
8        "arn:aws:s3:::myapp-static-assets/*",
9        "arn:aws:s3:::myapp-user-media/*"
10      ]
11    },
12    {
13      "Effect": "Allow",
14      "Action": ["s3:ListBucket"],
15      "Resource": [
16        "arn:aws:s3:::myapp-static-assets",
17        "arn:aws:s3:::myapp-user-media"
18      ]
19    }
20  ]
21}

The storage code can be correct while the IAM layer still blocks uploads or reads.

A Good Default Architecture

A practical split looks like this:

  • one public bucket behind a CDN for static files,
  • one private bucket for user uploads,
  • optional additional storage classes for reports, exports, or backups.

That design keeps collectstatic, user uploads, and private file delivery from competing for the same S3 policy and lifecycle rules.

Common Pitfalls

  • Using one global bucket because it seems simpler, then struggling with mixed ACL and caching needs.
  • Forgetting that STATICFILES_STORAGE and DEFAULT_FILE_STORAGE often need different backends.
  • Reusing a storage class but changing only location, when the bucket itself should differ too.
  • Granting IAM access to one bucket and assuming the second bucket will work automatically.
  • Letting user uploads overwrite files by leaving file_overwrite = True where uniqueness matters.

Summary

  • Multiple S3 buckets are best handled through separate Django storage classes.
  • Static files and media uploads usually deserve different backends and policies.
  • Attach a storage class globally or directly on individual model fields.
  • Keep bucket access aligned with IAM permissions and deployment roles.
  • A clean multi-bucket setup makes caching, privacy, and lifecycle rules much easier to manage.

Course illustration
Course illustration

All Rights Reserved.