dynamodb
global secondary index
dynamodb local
database querying
aws dynamodb

Querying a Global Secondary Index in dynamodb Local

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction to DynamoDB Local

Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. However, for testing and development purposes, Amazon offers DynamoDB Local, a downloadable version of DynamoDB that you can run on your computer. This is particularly useful for development activities where developers can test their applications without incurring costs or internet connectivity issues.

DynamoDB Local supports most of the core features of DynamoDB, including Queries and Global Secondary Indexes (GSIs).

Understanding Global Secondary Indexes

What is a Global Secondary Index?

A Global Secondary Index (GSI) in DynamoDB is a powerful feature that allows you to query your data using non-primary key attributes. Unlike the primary key, where you are limited to querying with hash and optional range keys, GSIs enable querying on different attributes for diverse access patterns.

Importance of Global Secondary Indexes

  1. Enhanced Query Flexibility: GSIs allow querying on attributes other than the primary key, expanding query flexibility.
  2. Performance Optimization: By querying on attributes that better suit your query traffic, you can optimize read performance.
  3. Non-blocking Operations: Modifications to GSIs do not block read or write operations, making them ideal for real-time applications.

Setting Up DynamoDB Local

To begin leveraging the power of GSIs in DynamoDB Local, you'll need to download and set it up:

bash
1# Download DynamoDB Local
2wget http://s3.amazonaws.com/dynamodb-local/dynamodb_local_latest.tar.gz
3
4# Extract the package
5tar -xzf dynamodb_local_latest.tar.gz
6
7# Run DynamoDB Local
8java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb

This setup allows you to spin up a local instance of DynamoDB on which you can perform operations without needing an AWS account.

Creating a Table with a Global Secondary Index

To better understand GSIs, let's create a DynamoDB table using the AWS CLI and add a GSI to it in the context of DynamoDB Local.

Creating the Table

Let’s say we are building a simple application to track user activity:

bash
1aws dynamodb create-table --table-name UserActivities \
2    --attribute-definitions AttributeName=UserId,AttributeType=S AttributeName=ActivityId,AttributeType=S \
3    --key-schema AttributeName=UserId,KeyType=HASH AttributeName=ActivityId,KeyType=RANGE \
4    --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 \
5    --endpoint-url http://localhost:8000

Adding a Global Secondary Index

Suppose we want to query the activities by ActivityType. We can fulfill this requirement by creating a GSI:

bash
1aws dynamodb update-table --table-name UserActivities \
2    --attribute-definitions AttributeName=UserId,AttributeType=S AttributeName=ActivityType,AttributeType=S \
3    --global-secondary-index-updates \
4    "[{\"Create\":{\"IndexName\": \"ActivityTypeIndex\", \
5    \"KeySchema\":[{\"AttributeName\":\"ActivityType\",\"KeyType\":\"HASH\"}], \
6    \"Projection\":{\"ProjectionType\":\"ALL\"}, \
7    \"ProvisionedThroughput\": {\"ReadCapacityUnits\": 5, \"WriteCapacityUnits\": 5}}}]" \
8    --endpoint-url http://localhost:8000

Querying the Global Secondary Index

After creating a GSI, you can query it to fetch items based on the ActivityType attribute:

bash
1aws dynamodb query --table-name UserActivities \
2    --index-name ActivityTypeIndex \
3    --key-condition-expression "ActivityType = :activityType" \
4    --expression-attribute-values '{":activityType":{"S":"Sports"}}' \
5    --endpoint-url http://localhost:8000

Explanation of Query Components:

  • Key Condition Expression: Specifies the key value for querying. Here, ActivityType = :activityType is used to fetch all activities of a particular type.
  • Expression Attribute Values: Map of attribute names to values for use in the query.

Additional Considerations

Consistency Models

DynamoDB Local supports two read consistency models:

  • Eventually Consistent Reads: Default and less costly, providing a fast response.
  • Strongly Consistent Reads: Guarantees the most up-to-date data but is constrained to certain regions and incurs more cost—even though it’s locally simulated.

Provisioned Throughput Capacity

For GSIs, you need to provision additional read and write capacity independently of the main table, keeping in mind the expected query load.

Table: Key Points

FeatureDescription
GSI Query FlexibilityAllows querying on attributes other than primary keys, expanding query capabilities.
Performance OptimizationOptimizes read performance by matching GSIs with query demands.
Non-blocking UpdatesModifications to GSIs do not block ongoing read or write operations.
Read ConsistencyOffers both eventually consistent and strongly consistent reads, with trade-offs.
Provisioned CapacityRequires separate provisioning of throughput, based on expected usage for each GSI.
Local Testing with DynamoDBEnables testing and application development without incurring real-world costs or issues.

Conclusion

Using GSIs with DynamoDB Local provides a flexible and efficient way to handle diverse query patterns, optimize resource usage, and enhance data retrieval operations. The local setup allows for comprehensive testing and emulation of AWS DynamoDB features, ensuring a smooth transition to a production environment. As always, consider your application's access patterns and performance requirements when designing your indexing strategies.


Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.