DynamoDB
Bash Scripting
AWS
Database Management
Partition and Sort Keys

Delete all items in a DynamoDB table using bash with both partition and sort keys

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.


Introduction

Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. It supports tables with both partition keys and sort keys, offering flexible querying and data modeling capabilities. However, there comes a time when you may need to delete all items from a DynamoDB table, whether for maintenance, testing, or reconfiguration purposes. This article explores how to achieve this using Bash scripting.

Understanding DynamoDB Terminology

Key Concepts

  • Partition Key: The primary key, also known as a hash key, which uniquely identifies a record within a table.
  • Sort Key: An optional key, also known as a range key, used to store multiple items with the same partition key, enabling more detailed querying.
  • Items: Equivalent to rows in a traditional database.
  • Attributes: The data fields within an item which can be thought of as columns in a database table.

Pre-requisites

Before deleting all items, ensure the following:

  1. AWS CLI Installed: Make sure that the AWS Command Line Interface (CLI) is installed and configured with the necessary permissions.
  2. Table Backed Up: Always back up the data before mass deletion to prevent accidental loss.

Delete All Items from a DynamoDB Table: Step-by-Step Guide

Step 1: List All Items

Initially, list all items in the table using a combination of `scan` operation and AWS CLI. This can traverse through the entire table. The command structure is generally as follows:

  • Scan Operation: Retrieves all items from the table. Adjust the query to match data types (`S` for string, `N` for number, `B` for binary).
  • Looping through Items: We use `jq`, a lightweight command-line JSON processor, to extract and iterate over each partition and sort key pair.
  • Delete Operation: Executes the `delete-item` AWS CLI command, specifying the partition and sort keys.
  • Rate of Deletion: Deleting items one-by-one can be slow depending on table size. Consider parallel processing or even truncating the table when conditions allow.
  • Cost: Both scan and delete operations incur costs. Monitor usage and costs in the AWS Management Console to avoid unexpected charges.
  • Error Handling: Incorporate error handling and logging for production-ready scripts.

Course illustration
Course illustration

All Rights Reserved.