Introduction

A common DynamoDB issue is trying to search by a nested JSON property from the AWS console and expecting query-like performance. DynamoDB indexing works on top-level key attributes, so nested values inside map attributes are not directly queryable unless modeled intentionally. You can still filter nested values with scans, but that has major cost and latency tradeoffs.

Why Nested Search Feels Limited

DynamoDB supports two main read patterns:

• 'Query, which uses partition and optional sort keys efficiently'
• 'Scan, which reads items broadly and applies filters after reading'

Nested map properties can be referenced in filter expressions, but filters do not reduce read units consumed by a scan. If you need fast lookup by a nested field, that field should usually be copied to a top-level indexed attribute.

Console Behavior Versus API Expectations

In the AWS console, search on nested paths is limited and easy to misinterpret. The console can help inspect data, but production query patterns should be designed around table keys and indexes.

For example, if your item looks like this:

1{
2  "pk": "ORDER#1001",
3  "sk": "META",
4  "details": {
5    "status": "PAID",
6    "channel": "WEB"
7  }
8}

Trying to find all details.status = PAID without a dedicated index generally requires scanning.

Using Filter Expressions on Nested Properties

You can target nested map attributes with expression attribute names in CLI or SDK.

1aws dynamodb scan \
2  --table-name Orders \
3  --filter-expression "details.#st = :v" \
4  --expression-attribute-names '{"#st":"status"}' \
5  --expression-attribute-values '{":v":{"S":"PAID"}}'

This works functionally, but performance may degrade quickly on large tables.

Better Data Model for Frequent Nested Lookups

If nested status is frequently queried, denormalize it into a top-level attribute and add an index.

Example item redesign:

1{
2  "pk": "ORDER#1001",
3  "sk": "META",
4  "status": "PAID",
5  "details": {
6    "status": "PAID",
7    "channel": "WEB"
8  }
9}

Then create a GSI on status plus optional secondary sort dimension.

Query example with SDK:

1import boto3
2from boto3.dynamodb.conditions import Key
3
4db = boto3.resource("dynamodb")
5table = db.Table("Orders")
6
7resp = table.query(
8    IndexName="gsi_status",
9    KeyConditionExpression=Key("status").eq("PAID")
10)
11
12print(len(resp["Items"]))

This gives predictable performance compared with full-table scanning.

Migration Strategy Without Downtime

If table is already in production, migrate in steps:

1. add new top-level query attribute
2. backfill old items in batches
3. create and validate GSI
4. switch reads from scan filter to query
5. remove old scan path when stable

Keep dual-write logic temporarily to maintain consistency during rollout.

Cost and Observability Considerations

Nested filter scans can look fine on dev data and become expensive in production. Monitor:

• consumed read capacity
• p95 and p99 read latency
• throttling events

Also log which code paths still use scans so you can prioritize migrations to indexed access patterns.

When Scan Is Still Acceptable

Scan with nested filters can still be reasonable for low-frequency admin workflows, one-off data audits, or migration verification tasks. In those cases, keep strict usage boundaries and run scans off-peak to reduce impact. If scan use starts appearing in user-facing request paths, treat that as a schema redesign signal and schedule index-oriented remediation.

Common Pitfalls

• Assuming filter expressions provide query-level efficiency.
• Designing nested JSON first and query patterns second.
• Relying on console behavior as performance proof.
• Skipping backfill verification when adding denormalized fields.
• Leaving scan-based fallback logic in critical high-volume paths.

Summary

• Nested map fields can be filtered, but not efficiently queried without key design.
• 'Scan plus filter is functional, not scalable for high-volume access.'
• Frequent nested lookups should be denormalized into indexed top-level attributes.
• Use staged migrations to move from scan filters to key-based queries safely.
• Track read cost and latency to detect model issues early.