AWS
DynamoDB
Command Line
JSON
Data Processing

How to simplify aws DynamoDB query JSON output from the command line?

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

Raw DynamoDB CLI output is verbose because each attribute includes a type wrapper such as S, N, or BOOL. That format is useful for API fidelity but inconvenient for shell scripts and quick analysis. You can simplify the output reliably with JMESPath queries in AWS CLI and optional jq post processing.

Understand Why DynamoDB Output Looks Verbose

DynamoDB represents each field as a typed map so the API can preserve data types across languages. A normal item might include values similar to string, number, and boolean wrappers in one response. For automation, you usually want a flat structure with simple keys and plain values.

Start with a raw query:

bash
1aws dynamodb query \
2  --table-name Orders \
3  --key-condition-expression "pk = :pk" \
4  --expression-attribute-values '{":pk":{"S":"USER#42"}}' \
5  --output json

This output is correct but noisy for downstream shell usage.

Flatten Results with AWS CLI --query

The fastest built in simplification is --query, which applies JMESPath before printing. Select only required fields and unwrap types directly.

bash
1aws dynamodb query \
2  --table-name Orders \
3  --key-condition-expression "pk = :pk" \
4  --expression-attribute-values '{":pk":{"S":"USER#42"}}' \
5  --query 'Items[].{orderId:order_id.S,status:order_status.S,total:total.N}' \
6  --output table

This command produces a readable table and removes unrelated attributes. For machine consumption, change output to json.

bash
1aws dynamodb query \
2  --table-name Orders \
3  --key-condition-expression "pk = :pk" \
4  --expression-attribute-values '{":pk":{"S":"USER#42"}}' \
5  --query 'Items[].{orderId:order_id.S,status:order_status.S,total:total.N}' \
6  --output json

Keep query expressions in one script constant so they are reviewed and versioned.

Convert Numeric Strings with jq

DynamoDB numbers are returned as strings. If later tools expect numeric JSON, pipe the data through jq and cast fields with tonumber.

bash
1aws dynamodb query \
2  --table-name Orders \
3  --key-condition-expression "pk = :pk" \
4  --expression-attribute-values '{":pk":{"S":"USER#42"}}' \
5  --query 'Items[].{orderId:order_id.S,total:total.N,paid:is_paid.BOOL}' \
6  --output json \
7| jq '[.[] | .total |= tonumber]' ``` Now `total` is numeric JSON instead of text, which helps analytics tools and typed scripts. ## Reuse Expressions in a Shell Script For repeatable operations, put your expression and parameters in a script with strict mode. This avoids copy and paste errors in ad hoc commands. ```bash #!/usr/bin/env bash set -euo pipefail TABLE_NAME="Orders" PK_VALUE="USER#42" QUERY_EXPR='Items[].{orderId:order_id.S,status:order_status.S,total:total.N}' aws dynamodb query \ --table-name "$TABLE_NAME" \ --key-condition-expression "pk = :pk" \ --expression-attribute-values "{\":pk\":{\"S\":\"$PK_VALUE\"}}" \ --query "$QUERY_EXPR" \ --output json \ | jq '[.[] | .total |= tonumber]' ``` Save this as `query-orders.sh`, make it executable with `chmod +x query-orders.sh`, and keep it in your operations repository. ## Validate Output Before Using It in Pipelines Before wiring simplified output into production scripts, test with two scenarios: * Existing key that returns items. * Missing key that returns an empty list. Then check exit status and output shape: ```bash ./query-orders.sh | jq 'type, length' ``` An explicit shape check prevents downstream failures when DynamoDB data changes. ## Handle Nested Attributes If your table stores maps or lists, keep flattening logic explicit so consumers know what fields are guaranteed. Combine `--query` selection with `jq` restructuring only for needed attributes. ```bash aws dynamodb query \ --table-name Orders \ --key-condition-expression \"pk = :pk\" \ --expression-attribute-values '{\":pk\":{\"S\":\"USER#42\"}}' \ --query 'Items[].{orderId:order_id.S,city:shipping.M.city.S,sku:items.L[0].M.sku.S}' \ --output json ``` This keeps command output compact while preserving a clear contract for downstream scripts. ## Common Pitfalls * Parsing raw `Items` manually with brittle text tools. Prefer `--query` and `jq` for structured extraction. * Forgetting that numeric values come as strings. Cast with `tonumber` when needed. * Writing different query expressions across scripts. Centralize one expression per use case. * Assuming fields always exist. Handle missing keys defensively in post processing. * Using table output for machine pipelines. Use JSON output for automation. ## Summary * DynamoDB CLI output is verbose by design because values carry explicit types. * Use `--query` to flatten and select only the fields you need. * Use `jq` to cast numeric strings and shape output for downstream tools. * Store reusable expressions in scripts under version control. * Validate output shape before integrating with production automation.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.