Introduction
The AWS CLI returns JSON by default, and filtering specific values from this output requires either the built-in --query flag (which uses JMESPath expressions) or piping through jq. Once filtered, you capture the result in a shell variable using command substitution ($()). The --query approach is portable and requires no extra tools, while jq offers more powerful JSON manipulation. Both are essential for AWS automation scripts.
Basic: --query with JMESPath
The --query flag filters JSON output using JMESPath expressions:
1# Get a specific instance's ID
2INSTANCE_ID=$(aws ec2 describe-instances \
3 --filters "Name=tag:Name,Values=web-server" \
4 --query "Reservations[0].Instances[0].InstanceId" \
5 --output text)
6
7echo "$INSTANCE_ID"
8# i-0abc123def456789
9
10# Get all running instance IDs
11INSTANCE_IDS=$(aws ec2 describe-instances \
12 --filters "Name=instance-state-name,Values=running" \
13 --query "Reservations[].Instances[].InstanceId" \
14 --output text)
15
16echo "$INSTANCE_IDS"
17# i-0abc123 i-0def456 i-0ghi789
The --output text flag converts the JSON result to plain text, making it easier to use in shell variables.
Filtering with --query
1# Get the public IP of a specific instance
2PUBLIC_IP=$(aws ec2 describe-instances \
3 --instance-ids i-0abc123def456789 \
4 --query "Reservations[0].Instances[0].PublicIpAddress" \
5 --output text)
6
7# Get S3 bucket names
8BUCKETS=$(aws s3api list-buckets \
9 --query "Buckets[].Name" \
10 --output text)
11
12# Get the latest AMI ID for Amazon Linux 2
13AMI_ID=$(aws ec2 describe-images \
14 --owners amazon \
15 --filters "Name=name,Values=amzn2-ami-hvm-*-x86_64-gp2" \
16 --query "sort_by(Images, &CreationDate)[-1].ImageId" \
17 --output text)
18
19# Get security group IDs for a VPC
20SG_IDS=$(aws ec2 describe-security-groups \
21 --filters "Name=vpc-id,Values=vpc-0abc123" \
22 --query "SecurityGroups[].GroupId" \
23 --output text)
Using jq for Complex Filtering
1# Install jq (if not present)
2# apt-get install jq or brew install jq
3
4# Get instance ID with jq
5INSTANCE_ID=$(aws ec2 describe-instances \
6 --filters "Name=tag:Name,Values=web-server" \
7| jq -r '.Reservations[0].Instances[0].InstanceId') # Get all running instance IDs as a newline-separated list aws ec2 describe-instances \ --filters "Name=instance-state-name,Values=running" \ | jq -r '.Reservations[].Instances[].InstanceId' # Complex filtering: instances with more than 4 CPUs LARGE_INSTANCES=$(aws ec2 describe-instances \ | jq -r '.Reservations[].Instances[] | select(.CpuOptions.CoreCount > 4) | .InstanceId') # Get a specific tag value NAME_TAG=$(aws ec2 describe-instances \ --instance-ids i-0abc123 \ | jq -r '.Reservations[0].Instances[0].Tags[] | select(.Key=="Name") | .Value') ``` ## Saving Multiple Values ```bash # Save multiple values to separate variables read -r INSTANCE_ID INSTANCE_TYPE STATE <<< $(aws ec2 describe-instances \ --instance-ids i-0abc123 \ --query "Reservations[0].Instances[0].[InstanceId, InstanceType, State.Name]" \ --output text) echo "ID: $INSTANCE_ID, Type: $INSTANCE_TYPE, State: $STATE" # Save to an array (bash 4+) mapfile -t INSTANCE_IDS < <(aws ec2 describe-instances \ --filters "Name=instance-state-name,Values=running" \ --query "Reservations[].Instances[].InstanceId" \ --output text | tr '\t' '\n') echo "Found ${#INSTANCE_IDS[@]} instances" for id in "${INSTANCE_IDS[@]}"; do echo " $id" done ``` ## Common AWS CLI + Variable Patterns ```bash # Wait for an instance to be running INSTANCE_ID=$(aws ec2 run-instances \ --image-id ami-0abc123 \ --instance-type t3.micro \ --query "Instances[0].InstanceId" \ --output text) aws ec2 wait instance-running --instance-ids "$INSTANCE_ID" echo "Instance $INSTANCE_ID is running" # Get the latest CloudFormation stack output API_URL=$(aws cloudformation describe-stacks \ --stack-name my-stack \ --query "Stacks[0].Outputs[?OutputKey=='ApiUrl'].OutputValue" \ --output text) # Get secret value from Secrets Manager DB_PASSWORD=$(aws secretsmanager get-secret-value \ --secret-id prod/db-password \ --query "SecretString" \ --output text) # Get parameter from SSM Parameter Store CONFIG_VALUE=$(aws ssm get-parameter \ --name "/app/config/db-host" \ --with-decryption \ --query "Parameter.Value" \ --output text) ``` ## Error Handling ```bash # Check if the command succeeded INSTANCE_ID=$(aws ec2 describe-instances \ --instance-ids i-0abc123 \ --query "Reservations[0].Instances[0].InstanceId" \ --output text 2>/dev/null) if [ -z "$INSTANCE_ID" ] || [ "$INSTANCE_ID" = "None" ]; then echo "Error: Instance not found" exit 1 fi # Using set -e for strict error handling set -e BUCKET_NAME=$(aws s3api list-buckets \ --query "Buckets[?starts_with(Name, 'prod-')].Name | [0]" \ --output text) ``` ## Common Pitfalls * **Missing `--output text`**: Without it, the result is JSON-formatted (with quotes). A variable set to `"i-0abc123"` (with quotes) causes issues in subsequent commands. Always use `--output text` when saving to a shell variable. * **`None` vs empty string**: When a `--query` path does not match, AWS CLI returns `None` (the string, not null). Check for both empty and `None` values: `if [ -z "$VAR" ] || [ "$VAR" = "None" ]`. * **JMESPath `[]` flattening**: `Reservations[].Instances[]` flattens nested arrays into a single list. `Reservations[*].Instances[*]` preserves nesting. Use `[]` for most variable-capture scenarios. * **Tab-separated multi-value output**: `--output text` separates array elements with tabs, not newlines. Use `tr '\t' '\n'` to convert to newlines for `mapfile` or `while read` loops. * **Quoting variables in subsequent commands**: Always quote `"$INSTANCE_ID"` when using it in later AWS CLI calls. Unquoted variables with spaces or special characters cause argument splitting errors. ## Summary * Use `--query` with JMESPath expressions and `--output text` to extract specific values from AWS CLI output * Capture results in variables with `VAR=$(aws ... --query "..." --output text)` * Use `jq` for complex JSON filtering that exceeds JMESPath capabilities * Always check for empty or `None` results before using the variable in subsequent commands * Use `--output text` with `tr '\t' '\n'` to convert multi-value results into newline-separated lists for shell loops