DynamoDB
Fulltext Search
Database
AWS
NoSQL

Fulltext Search DynamoDB

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

DynamoDB is excellent for key-based access patterns, but it is not a full-text search engine. If you need stemming, relevance ranking, typo tolerance, or multi-field text queries, the usual answer is to keep DynamoDB as the source of truth and index searchable text somewhere else.

Why DynamoDB alone is not enough

DynamoDB can query efficiently by partition key, sort key, and secondary indexes. What it cannot do well is "find all items whose description contains related forms of this phrase and rank them by relevance." A contains filter is not full-text search. It scans data, does not tokenize text, and does not scale well for search-heavy workloads.

That distinction matters because many teams start by trying to force search into DynamoDB, then discover that the table design gets worse while search quality stays poor.

Common architecture: DynamoDB plus OpenSearch

The standard AWS design is:

  1. write canonical records to DynamoDB
  2. capture table changes with DynamoDB Streams
  3. push searchable fields into Amazon OpenSearch Service
  4. query OpenSearch for search results
  5. optionally hydrate results from DynamoDB if you need the latest full record

This keeps transactional writes simple while giving search its own index and ranking rules.

Indexing DynamoDB changes into OpenSearch

The Lambda below handles DynamoDB Stream events and updates an OpenSearch index. It uses boto3 for AWS credentials and opensearch-py for indexing.

python
1import os
2from boto3.dynamodb.types import TypeDeserializer
3import boto3
4from opensearchpy import OpenSearch, RequestsHttpConnection, AWSV4SignerAuth
5
6region = os.environ["AWS_REGION"]
7host = os.environ["OPENSEARCH_HOST"]
8
9session = boto3.Session()
10credentials = session.get_credentials()
11auth = AWSV4SignerAuth(credentials, region, "es")
12deserializer = TypeDeserializer()
13
14client = OpenSearch(
15    hosts=[{"host": host, "port": 443}],
16    http_auth=auth,
17    use_ssl=True,
18    verify_certs=True,
19    connection_class=RequestsHttpConnection,
20)
21
22def from_ddb(image):
23    return {key: deserializer.deserialize(value) for key, value in image.items()}
24
25def handler(event, context):
26    for record in event["Records"]:
27        event_name = record["eventName"]
28
29        if event_name in ("INSERT", "MODIFY"):
30            item = from_ddb(record["dynamodb"]["NewImage"])
31            doc_id = item["id"]
32
33            client.index(
34                index="articles",
35                id=doc_id,
36                body={
37                    "title": item["title"],
38                    "body": item["body"],
39                    "tags": item.get("tags", []),
40                },
41            )
42
43        elif event_name == "REMOVE":
44            item = from_ddb(record["dynamodb"]["OldImage"])
45            client.delete(index="articles", id=item["id"], ignore=[404])

This pattern gives near-real-time indexing without changing your application write path.

Querying the search index

Once the index exists, search becomes much closer to what users expect:

bash
1curl -X POST "https://my-domain.us-east-1.es.amazonaws.com/articles/_search" \
2  -H "Content-Type: application/json" \
3  -d '{
4    "query": {
5      "multi_match": {
6        "query": "distributed cache invalidation",
7        "fields": ["title^3", "body", "tags"]
8      }
9    }
10  }'

This can search across multiple fields and boost title matches above body matches. That is the kind of relevance behavior DynamoDB does not provide natively.

Alternatives and tradeoffs

OpenSearch is the default answer, but not the only one:

  • Algolia if you want a managed search product with less infrastructure work
  • Meilisearch or Typesense if you control your own search service
  • application-side filtering only when the dataset is small and search quality is not important

The right choice depends on scale and operational appetite. The core idea stays the same: the transactional store and the search index serve different purposes.

Common Pitfalls

  • Using DynamoDB Scan plus FilterExpression and calling it full-text search. It is not.
  • Indexing every attribute instead of only the fields users actually search.
  • Forgetting delete handling, which leaves stale documents in the search index.
  • Assuming OpenSearch is strongly consistent with DynamoDB. There is usually indexing lag.
  • Returning search hits directly when the application really needs to re-read authoritative data from DynamoDB.

Summary

  • DynamoDB is a great key-value and document store, but it does not provide true full-text search.
  • The common production pattern is DynamoDB plus Streams, Lambda, and OpenSearch.
  • Use DynamoDB as the source of truth and keep only searchable fields in the search index.
  • Expect eventual consistency between the table and the search engine.
  • Pick a dedicated search system when you need ranking, stemming, phrase matching, or typo tolerance.

Course illustration
Course illustration

All Rights Reserved.