Open S3 object as a string with Boto3
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
When you fetch an object from Amazon S3 with Boto3, the body is returned as a stream of bytes, not as a Python string. To use the contents as text, you need to read the stream and decode the bytes with the correct character encoding, which is usually UTF-8 for JSON, CSV, and plain text files.
Read the Object Body and Decode It
The direct approach uses get_object, then reads from the Body stream.
That is the standard pattern. read() returns bytes, and decode("utf-8") converts those bytes into a normal Python string.
If the object is known to be small, reading it all at once is fine. For very large text files, you may want line-by-line or chunked processing instead of loading everything into memory at once.
Work With Structured Text After Decoding
Once the S3 object is a string, you can pass it to parsers or application code. JSON is a common example.
For CSV, you might decode first and then split lines or pass the string into csv.reader through an in-memory stream.
The key idea is the same in every case: bytes first, decoded text second, parser third.
Resource API Variant
Boto3 also provides a resource-style API. Some people prefer it because the object lookup reads a little more naturally.
The result is functionally similar. The choice between client and resource style is mostly about the rest of your codebase.
Check Encoding and Content Type
UTF-8 is the usual answer, but not every file in S3 is UTF-8 text. Some files may use another text encoding, and some are binary objects such as images, ZIP files, or Parquet data. In those cases, decoding as UTF-8 will raise an exception or produce garbage text.
A useful habit is to inspect metadata when you are not sure what the object contains:
If the object is text but uses a different encoding, decode accordingly, such as latin-1 or utf-16. If it is binary, keep it as bytes and handle it with the correct binary parser.
Handle Errors Explicitly
Two failure modes are common: the object does not exist, or the AWS credentials and permissions are wrong. Catching the exception makes those cases easier to debug.
This does not replace proper logging, but it does make the failure source clearer than a generic crash in a larger script.
Common Pitfalls
- Treating the S3 body as a string immediately is incorrect because
Body.read()returns bytes. - Decoding with UTF-8 when the file uses another encoding can raise errors or silently corrupt text.
- Loading a very large object fully into memory is inefficient when streaming or chunked processing would be safer.
- Assuming all S3 objects are text is a mistake. Binary files should remain bytes until handled by the right parser.
- Debugging parsing logic before checking bucket permissions, key names, and credentials wastes time when the failure is really in the S3 request itself.
Summary
- Use
get_objectand read fromresponse["Body"]to access S3 object contents. - Convert bytes to a Python string with
.decode("utf-8")when the object is UTF-8 text. - Decode first, then pass the string to JSON, CSV, or other text parsers.
- Inspect metadata and encoding assumptions when the decoded output looks wrong.
- Keep binary objects as bytes and handle large text objects carefully to avoid memory issues.
Related reading
- Opening port 80 EC2 Amazon web services
- Openshift service is not available after short inactivity
- Optimize write performance for AWS Aurora instance
- Optional secondary indexes in DynamoDB
- Openface Flask Wrapper Flask seems to be blocking a thread
- Operation on every pair of element in a list
- Order by in DynamoDB using params
- Organizing AWS IAM permissions limit of 10 policies?

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.