Properly catch boto3 Errors
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
With boto3, the safest way to handle AWS failures is to catch the right exception class and then inspect the structured error code that AWS returned. Many examples online either catch Exception too broadly or compare human-readable messages that can change. Robust boto3 code should branch on documented exception types and machine-readable error codes instead.
Catch ClientError for Service Responses
Most AWS API failures that come back from a service are raised as botocore.exceptions.ClientError.
The important data is in err.response, especially the nested Error entry. That structure is much more stable than the formatted exception string.
Separate SDK Problems from Service Problems
Not every boto3 failure is a service response. Some errors come from the SDK or transport layer itself.
This split matters because retry logic and user feedback may differ. A throttling response from AWS is not the same kind of problem as a broken credential provider chain or a connection failure.
Branch on Error Codes, Not Full Messages
When you need special handling, inspect Error.Code.
Re-raising unknown cases is important. Otherwise, broad exception handling can hide real production failures.
Prefer Specific Service Exceptions When Available
Some clients expose service-specific exception helpers.
This can improve readability, but not every case has a clean generated exception class. ClientError with code inspection remains the most generally reliable pattern.
Log Enough Context to Debug
Error handling is not only about catching. You also want enough context to debug without logging secrets. That usually means the AWS operation, the resource identifier, and the error code.
A good handler is therefore selective and observable. It decides which failures are expected, which deserve retries, and which should terminate the workflow.
Add Retry Logic Deliberately
Some boto3 failures are transient and should be retried, while others are permanent. Throttling, temporary network failures, and timeouts often deserve another attempt. AccessDenied, malformed requests, and missing required parameters usually do not.
That means exception handling and retry policy should be designed together. Catching an error is only the first step. You still need to decide whether the call should be retried, surfaced to a caller, or turned into a domain-specific application error.
Test the Failure Path
Error handling code is easy to leave untested because the success path is simpler to reproduce. With boto3, it is worth exercising the unhappy path explicitly so you know your code branches on real AWS-style responses.
This kind of test confirms that your handler is reading structured fields rather than relying on assumptions about message formatting.
Common Pitfalls
- Catching plain
Exceptionand losing the distinction between AWS failures and unrelated bugs. - Matching on the formatted error message instead of the structured error code.
- Swallowing unknown
ClientErrorcases instead of re-raising them. - Treating SDK failures and service responses as if they were the same category.
- Logging credentials or entire request payloads while debugging AWS errors.
Summary
- Catch
ClientErrorfor AWS service-side failures. - Catch
BotoCoreErrorseparately for SDK and transport problems. - Use
err.response["Error"]["Code"]for decision-making. - Re-raise unexpected cases instead of hiding them.
- Log resource and operation context, but avoid leaking secrets.
Related reading
- Properly Configuring Kafka Connect S3 Sink TimeBasedPartitioner
- Pros and cons on utilizing Azure Service Fabric vs Custom Azure Cloud?
- Provision multiple logical databases with Terraform on AWS RDS cluster instance
- Proxy Outbound/Egress Traffic Within Kubernetes
- Proxies with Python 'Requests' module
- psycopg2 insert multiple rows with one query
- Property cannot be found in forward class object
- Property set method not found error during reflection

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.