Choosing an AWS Data Store

Topics Covered

Starting From the Workload

Access patterns first

Consistency and transactions

Scale and growth

Latency targets

Query flexibility

Team and operations

Cost shape

AWS offers more than fifteen managed databases, and most of them are good at something. The hard part is not learning each one but choosing among them for a particular workload, and then living with the choice for years. Data outlives code: an application can be rewritten in a quarter, but moving a busy production dataset to a different engine is a project measured in months. That asymmetry is why the choice deserves a deliberate process that starts from the workload, not from the database a team used last time or the one with the most conference talks this year.

Access patterns first

The single most useful input is a list of the ways the application reads and writes data, with rough frequencies. How does it look up a single record, by which identifiers? What lists does it show, filtered and sorted by what? Which reports does it run, and how often? Which writes must happen together? A workload dominated by lookups by key and short lists under one parent looks very different from one dominated by ad hoc filtering across many attributes, and different again from one dominated by aggregations over months of history. Writing the list before choosing an engine exposes which patterns are the hard ones, and it is the same list a key value design would need anyway, so the effort is never wasted.

Frequency matters as much as shape. A query that runs once a night can be slow and expensive without anyone caring, while one that runs on every page view sets the latency and cost of the whole system. Marking each pattern with its expected rate, its latency target and whether it is on the critical path turns the list into a ranking, and the engine is chosen for the patterns at the top of that ranking. Rare patterns can often be served another way, by a nightly export or a secondary store, rather than shaping the primary choice.

Key lookups and per customer lists made up 97 percent of requests. The monthly report ran once, and it should not choose the database.

Consistency and transactions

The next question is what correctness the workload needs. Does a write have to be visible to the next read everywhere, or can readers see slightly old data? Do several records have to change atomically, such as debiting one account and crediting another? Are there invariants that span many rows, such as unique usernames or inventory that must not go negative? Relational databases give rich transactions across any rows. DynamoDB gives single item atomicity and conditional writes everywhere, plus transactions over up to one hundred items. Search and analytics engines are generally eventually consistent and should not hold data that needs these guarantees.

Durability and recovery belong in the same conversation. How much data can the business lose in a failure, measured in seconds of writes, and how long can the store be unavailable? A workload that tolerates losing the last minute of writes has more options than one that tolerates nothing. Point in time recovery, cross region copies and the speed of restoring a large dataset differ between engines, and a store that takes hours to restore may not meet a recovery time objective measured in minutes, however well it performs day to day.

Scale and growth

Size and throughput set hard limits. A relational database on one primary handles a great deal, tens of thousands of transactions a second on a large Aurora instance, and terabytes of data, but writes are bounded by one node. DynamoDB scales reads and writes horizontally with no practical ceiling if keys are well distributed. Analytics engines scale with data volume but not with high rates of small updates. Estimating data size and peak request rates for three years out, then checking them against each candidate's limits, rules out some options early. Most applications are far smaller than they fear, and choosing a distributed store for a workload that fits on one instance adds cost and complexity for no benefit.

Growth estimates are rarely accurate, so it pays to ask what happens if they are off by ten times in either direction. A choice that still works at ten times the estimate, perhaps with a bigger instance or more shards, is robust. A choice that only works within a narrow band forces a migration if the product succeeds, which is exactly when the team has the least time for one.

Latency targets

Each engine has a natural latency range. In memory stores answer in microseconds, DynamoDB in single digit milliseconds at any scale, relational databases in milliseconds for indexed queries and much longer for complex joins, search engines in tens of milliseconds, and analytics engines in seconds. A workload whose critical path needs single digit millisecond reads at the ninety ninth percentile, under heavy load, narrows the field. One that serves dashboards a few times an hour can accept seconds and benefit from engines optimised for scanning rather than lookups.

Latency is also shaped by where the data sits relative to its users. A store in one region adds a round trip of a hundred milliseconds or more for users on another continent. If the workload serves a global audience with tight latency targets, multi region options such as DynamoDB global tables or Aurora DSQL become important, while a regional workload can ignore them. Asking where readers and writers are located, not just how fast the engine is, avoids choosing a store that is fast in a benchmark and slow for half the users.

Query flexibility

Some workloads know their queries in advance and rarely change them. Others need to answer new questions every week, joining and filtering in ways nobody planned. Relational databases and analytics engines excel at the second kind, because SQL and a query planner can answer arbitrary questions. Key value stores excel at the first kind, but every new access pattern may need a new index or a data migration. The more uncertain the future queries, the more valuable a flexible engine becomes, and early stage products, where requirements shift monthly, often benefit from a relational database even when a key value store would scale further.

Team and operations

A database the team understands, can debug at three in the morning and knows how to tune is worth a lot. Managed services remove much of the operational load, but each engine still has its own failure modes, metrics, limits and modelling discipline. Adding a new engine means learning those, and the cost is paid by every engineer who touches it. That is not an argument against ever adopting something new, but it is a cost to weigh alongside features. It also argues for fewer engines used well over many engines used shallowly.

Cost shape

Engines bill differently. Provisioned relational instances cost the same whether busy or idle. DynamoDB on demand bills per request, which is cheap at low volume and expensive at high steady volume. Serverless options scale to near zero. Analytics engines bill for data scanned or for compute time. Storage prices differ by an order of magnitude between S3 and a provisioned database volume. Modelling the expected bill for each candidate at today's size and at the three year estimate often changes the answer, especially for large, cold datasets that a cheaper storage tier could hold.

It helps to separate fixed costs from variable ones when comparing. A provisioned cluster with replicas in three zones has a floor that is paid even at zero traffic, which matters for many small services or environments such as staging and per developer copies. Serverless and per request engines have almost no floor but a steeper slope. A team running fifty small services may find that floors dominate its bill, while a team with one large service is dominated by slope, and the cheapest engine differs between them.

List the patterns before naming an engine

Write down every read and write pattern with its rate and latency target, the transactions and consistency it needs, and three year size and throughput. Choose the engine for the patterns at the top of that list, weigh the team's experience and the bill, and serve rare patterns another way.