Boto3
AWS S3
Python
Cloud Computing
Programming

Open S3 object as a string with Boto3

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

When you fetch an object from Amazon S3 with Boto3, the body is returned as a stream of bytes, not as a Python string. To use the contents as text, you need to read the stream and decode the bytes with the correct character encoding, which is usually UTF-8 for JSON, CSV, and plain text files.

Read the Object Body and Decode It

The direct approach uses get_object, then reads from the Body stream.

python
1import boto3
2
3s3 = boto3.client("s3")
4response = s3.get_object(Bucket="my-bucket", Key="notes/example.txt")
5text = response["Body"].read().decode("utf-8")
6
7print(text)

That is the standard pattern. read() returns bytes, and decode("utf-8") converts those bytes into a normal Python string.

If the object is known to be small, reading it all at once is fine. For very large text files, you may want line-by-line or chunked processing instead of loading everything into memory at once.

Work With Structured Text After Decoding

Once the S3 object is a string, you can pass it to parsers or application code. JSON is a common example.

python
1import boto3
2import json
3
4s3 = boto3.client("s3")
5response = s3.get_object(Bucket="my-bucket", Key="data/config.json")
6text = response["Body"].read().decode("utf-8")
7config = json.loads(text)
8
9print(config["region"])

For CSV, you might decode first and then split lines or pass the string into csv.reader through an in-memory stream.

python
1import boto3
2import csv
3import io
4
5s3 = boto3.client("s3")
6response = s3.get_object(Bucket="my-bucket", Key="reports/users.csv")
7text = response["Body"].read().decode("utf-8")
8reader = csv.reader(io.StringIO(text))
9
10for row in reader:
11    print(row)

The key idea is the same in every case: bytes first, decoded text second, parser third.

Resource API Variant

Boto3 also provides a resource-style API. Some people prefer it because the object lookup reads a little more naturally.

python
1import boto3
2
3s3 = boto3.resource("s3")
4obj = s3.Object("my-bucket", "notes/example.txt")
5text = obj.get()["Body"].read().decode("utf-8")
6
7print(text)

The result is functionally similar. The choice between client and resource style is mostly about the rest of your codebase.

Check Encoding and Content Type

UTF-8 is the usual answer, but not every file in S3 is UTF-8 text. Some files may use another text encoding, and some are binary objects such as images, ZIP files, or Parquet data. In those cases, decoding as UTF-8 will raise an exception or produce garbage text.

A useful habit is to inspect metadata when you are not sure what the object contains:

python
1import boto3
2
3s3 = boto3.client("s3")
4response = s3.get_object(Bucket="my-bucket", Key="notes/example.txt")
5
6print(response.get("ContentType"))
7print(response.get("ContentLength"))

If the object is text but uses a different encoding, decode accordingly, such as latin-1 or utf-16. If it is binary, keep it as bytes and handle it with the correct binary parser.

Handle Errors Explicitly

Two failure modes are common: the object does not exist, or the AWS credentials and permissions are wrong. Catching the exception makes those cases easier to debug.

python
1import boto3
2from botocore.exceptions import ClientError
3
4s3 = boto3.client("s3")
5
6try:
7    response = s3.get_object(Bucket="my-bucket", Key="notes/example.txt")
8    text = response["Body"].read().decode("utf-8")
9    print(text)
10except ClientError as exc:
11    print(f"S3 request failed: {exc}")

This does not replace proper logging, but it does make the failure source clearer than a generic crash in a larger script.

Common Pitfalls

  • Treating the S3 body as a string immediately is incorrect because Body.read() returns bytes.
  • Decoding with UTF-8 when the file uses another encoding can raise errors or silently corrupt text.
  • Loading a very large object fully into memory is inefficient when streaming or chunked processing would be safer.
  • Assuming all S3 objects are text is a mistake. Binary files should remain bytes until handled by the right parser.
  • Debugging parsing logic before checking bucket permissions, key names, and credentials wastes time when the failure is really in the S3 request itself.

Summary

  • Use get_object and read from response["Body"] to access S3 object contents.
  • Convert bytes to a Python string with .decode("utf-8") when the object is UTF-8 text.
  • Decode first, then pass the string to JSON, CSV, or other text parsers.
  • Inspect metadata and encoding assumptions when the decoded output looks wrong.
  • Keep binary objects as bytes and handle large text objects carefully to avoid memory issues.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.