DynamoDB
Python
Boto3
AWS
Database Connection

How to establish a connection to DynamoDB using python using boto3

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

Amazon DynamoDB is a fully managed NoSQL database service designed to handle large-scale data needs with low latency and scalability. Python developers often use the boto3 library, the official AWS SDK for Python, to interact with DynamoDB. This article provides a comprehensive guide on establishing a connection to DynamoDB using boto3 and interacting with the service.

Prerequisites

Before you begin, ensure that you have the following:

  • AWS Account: You need an active AWS account to use DynamoDB.
  • IAM Permissions: Ensure your AWS Identity and Access Management (IAM) user has the necessary permissions for DynamoDB.
  • Python Environment: Ensure Python and pip are installed on your machine.
  • boto3: Install boto3 using pip if you haven't already: pip install boto3.

Setting Up AWS Credentials

To interact with AWS services, boto3 requires valid AWS credentials. The simplest way to manage these is by using the AWS CLI:

  1. Install AWS CLI: If you don't have AWS CLI installed, follow these instructions.
  2. Configure AWS CLI: Run aws configure and provide your AWS Access Key ID, AWS Secret Access Key, default region name, and default output format.

Credentials can also be stored in the ~/.aws directory as follows:

  • Config file (~/.aws/config):
ini
  [default]
  region = us-west-2
  output = json
  • Credentials file (~/.aws/credentials):
ini
  [default]
  aws_access_key_id = YOUR_ACCESS_KEY
  aws_secret_access_key = YOUR_SECRET_KEY

Establishing a Connection Using boto3

To establish a connection to DynamoDB, you need to create a boto3.client or boto3.resource. Here’s how you can do it:

python
1import boto3
2
3# Creating a DynamoDB client
4dynamodb_client = boto3.client('dynamodb', region_name='us-west-2')
5
6# OR creating a DynamoDB resource which is a higher-level abstraction
7dynamodb_resource = boto3.resource('dynamodb', region_name='us-west-2')

boto3.client vs boto3.resource

  • Client: Provides low-level service access. You must manually handle operations and exceptions.
  • Resource: Provides a higher-level abstraction over clients, ideal for object-oriented operations and easier management.

Basic Operations

Creating a Table

To create a DynamoDB table with a resource, follow this example:

python
1import boto3
2
3dynamodb = boto3.resource('dynamodb')
4
5table = dynamodb.create_table(
6    TableName='Movies',
7    KeySchema=[
8        {
9            'AttributeName': 'year',
10            'KeyType': 'HASH'  # Partition key
11        },
12        {
13            'AttributeName': 'title',
14            'KeyType': 'RANGE'  # Sort key
15        }
16    ],
17    AttributeDefinitions=[
18        {
19            'AttributeName': 'year',
20            'AttributeType': 'N'
21        },
22        {
23            'AttributeName': 'title',
24            'AttributeType': 'S'
25        },
26    ],
27    ProvisionedThroughput={
28        'ReadCapacityUnits': 5,
29        'WriteCapacityUnits': 5
30    }
31)
32
33print("Table status:", table.table_status)

Writing Data to a Table

python
1table = dynamodb.Table('Movies')
2
3table.put_item(
4   Item={
5        'year': 2021,
6        'title': 'Inception',
7        'info': {
8            'director': 'Christopher Nolan',
9            'rating': 8.8
10        }
11    }
12)

Reading Data from a Table

python
1response = table.get_item(
2    Key={
3        'year': 2021,
4        'title': 'Inception'
5    }
6)
7item = response.get('Item')
8print(item)

Querying Data

python
1response = table.query(
2    KeyConditionExpression=Key('year').eq(2021)
3)
4for item in response['Items']:
5    print(item)

Updating an Item

python
1table.update_item(
2    Key={
3        'year': 2021,
4        'title': 'Inception'
5    },
6    UpdateExpression='SET info.rating = :val',
7    ExpressionAttributeValues={
8        ':val': 9.0
9    }
10)

Error Handling

When performing operations on DynamoDB using boto3, it's important to handle exceptions. AWS defines specific exceptions for different scenarios, such as provisioned throughput exceeded. Here’s how to handle them:

python
1import botocore
2
3try:
4    # DynamoDB operations here
5    pass
6except botocore.exceptions.ClientError as error:
7    if error.response['Error']['Code'] == 'ProvisionedThroughputExceededException':
8        print("Throughput limit reached. Consider increasing limits or using appropriate backoff strategies.")
9    else:
10        print("Unexpected error: %s" % error)

Summary

Here’s a summary of key points when using boto3 to connect to DynamoDB:

ConceptDetails
AWS CredentialsStored in ~/.aws/credentials and ~/.aws/config
boto3.clientLow-level service access
boto3.resourceHigh-level, object-oriented access
Table OperationsCreate, Read, Update, Delete operations supported
Error HandlingUse botocore.exceptions.ClientError for handling AWS errors

Conclusion

Connecting to DynamoDB using Python and boto3 is a straightforward process with access to both low-level and high-level abstractions to manage your database interactions. With its extensive functionality and scalability, DynamoDB is an excellent choice for applications requiring fast and predictable performance. Always ensure your IAM user has the correct permissions, and handle exceptions gracefully to build robust applications.


Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.