Introduction

Writing a file to S3 from AWS Lambda with Python is a common serverless pattern for reports, transformed payloads, exports, and event archives. The mechanics are simple with boto3, but a good implementation also needs correct IAM permissions, stable bucket configuration, and clear error handling.

In practice, there are two common approaches: upload bytes directly with put_object, or write a temporary file under /tmp and upload that file. The first is simpler for small outputs. The second is useful when a library expects a real file.

Upload Small Content Directly

If you already have the content in memory, put_object is the simplest solution:

1import json
2import os
3import boto3
4
5s3 = boto3.client("s3")
6BUCKET = os.environ["OUTPUT_BUCKET"]
7
8
9def lambda_handler(event, context):
10    key = "reports/hello.txt"
11    body = "Hello from Lambda\n"
12
13    s3.put_object(
14        Bucket=BUCKET,
15        Key=key,
16        Body=body.encode("utf-8"),
17        ContentType="text/plain; charset=utf-8",
18    )
19
20    return {
21        "statusCode": 200,
22        "body": json.dumps({"bucket": BUCKET, "key": key}),
23    }

This is the normal answer for text files, JSON payloads, and other modest outputs that fit comfortably in memory.

Use Environment Variables for Bucket and Prefix

Hardcoding bucket names makes promotion between environments awkward. A better pattern is to read configuration from environment variables:

1import os
2import boto3
3
4s3 = boto3.client("s3")
5BUCKET = os.environ["OUTPUT_BUCKET"]
6PREFIX = os.environ.get("OUTPUT_PREFIX", "exports")
7
8
9def lambda_handler(event, context):
10    key = f"{PREFIX}/result-{context.aws_request_id}.json"
11    payload = b'{"status":"ok"}'
12
13    s3.put_object(
14        Bucket=BUCKET,
15        Key=key,
16        Body=payload,
17        ContentType="application/json",
18    )
19
20    return {"statusCode": 200, "key": key}

That keeps the function reusable across dev, staging, and production.

Use /tmp When You Need a Real File

If a library generates output on disk, write to Lambda's temporary storage and upload the file:

1from pathlib import Path
2import boto3
3import os
4
5s3 = boto3.client("s3")
6BUCKET = os.environ["OUTPUT_BUCKET"]
7
8
9def lambda_handler(event, context):
10    temp_path = Path("/tmp/data.csv")
11    temp_path.write_text("id,value\n1,100\n2,200\n", encoding="utf-8")
12
13    with temp_path.open("rb") as fh:
14        s3.upload_fileobj(fh, BUCKET, "csv/data.csv")
15
16    return {"statusCode": 200}

This is useful for generated CSVs, PDFs, ZIP files, or other file-oriented workflows.

IAM Permissions Are Required

The Lambda execution role must be allowed to write to the target bucket and key prefix:

1{
2  "Version": "2012-10-17",
3  "Statement": [
4    {
5      "Effect": "Allow",
6      "Action": ["s3:PutObject"],
7      "Resource": "arn:aws:s3:::my-output-bucket/exports/*"
8    }
9  ]
10}

Without that permission, the code is correct but the upload will still fail. If the bucket uses encryption or stricter policies, additional permissions may also be needed.

Add Error Handling and Logging

S3 writes should log enough context to debug failures:

1import logging
2from botocore.exceptions import ClientError
3
4logger = logging.getLogger()
5logger.setLevel(logging.INFO)
6
7
8def safe_put(bucket: str, key: str, body: bytes):
9    try:
10        s3.put_object(Bucket=bucket, Key=key, Body=body)
11        logger.info("uploaded bucket=%s key=%s bytes=%d", bucket, key, len(body))
12    except ClientError:
13        logger.exception("failed upload bucket=%s key=%s", bucket, key)
14        raise

That is much more useful in CloudWatch than a generic failure message with no bucket or key information.

Common Pitfalls

The biggest mistake is forgetting S3 write permission on the Lambda execution role. The function then fails even though the Python code is correct.

Another common issue is hardcoding the bucket name and key prefix, which makes deployment across environments fragile.

Developers also sometimes write large files to /tmp without considering Lambda storage limits or cleanup. Temporary storage is useful, but it is not infinite.

Finally, do not return success before the S3 call has actually completed. Wrap the upload path in proper exception handling so failures propagate clearly.

Summary

• Use boto3 with put_object for direct in-memory uploads from Lambda.
• Use /tmp plus upload_fileobj when a workflow needs a real temporary file.
• Store bucket and prefix configuration in environment variables instead of hardcoding them.
• Make sure the Lambda execution role includes the right s3:PutObject permissions.
• Add clear logging and exception handling so S3 failures are easy to diagnose.