Introduction

When ExclusiveStartKey fails in DynamoDB, the bug is usually not in pagination logic itself. The usual cause is a mismatch between the key structure you pass back and the API layer you are calling, because the boto3 resource API and low-level client API expect different key formats.

What ExclusiveStartKey Is Supposed to Be

DynamoDB pagination works like this:

1. you run Query or Scan
2. DynamoDB returns LastEvaluatedKey if another page exists
3. you pass that exact value back as ExclusiveStartKey

The safest implementation rule is simple: do not reconstruct the key by hand unless you truly have to. Reuse the value DynamoDB already returned.

Resource API Versus Client API

This is the source of most bugs.

With the high-level boto3 resource API, keys are ordinary Python values.

1import boto3
2from boto3.dynamodb.conditions import Key
3
4resource = boto3.resource("dynamodb")
5table = resource.Table("events")
6
7response = table.query(
8    KeyConditionExpression=Key("user_id").eq("u-100"),
9    Limit=10,
10)
11
12start_key = response.get("LastEvaluatedKey")
13print(start_key)

If you use table.query, you pass that key back in the same resource-style shape.

1if start_key:
2    next_page = table.query(
3        KeyConditionExpression=Key("user_id").eq("u-100"),
4        Limit=10,
5        ExclusiveStartKey=start_key,
6    )

The low-level client API is different. It expects DynamoDB's typed JSON format.

1import boto3
2
3client = boto3.client("dynamodb")
4
5response = client.query(
6    TableName="events",
7    KeyConditionExpression="user_id = :uid",
8    ExpressionAttributeValues={
9        ":uid": {"S": "u-100"}
10    },
11    Limit=10,
12)
13
14start_key = response.get("LastEvaluatedKey")
15print(start_key)

If you use client.query, the ExclusiveStartKey must stay in that typed format.

Do Not Cross the Formats

This is the classic failure pattern:

• use table.query
• build an ExclusiveStartKey like {"user_id": {"S": "u-100"}}
• get a validation error

Or the reverse:

• use client.query
• pass plain Python values such as {"user_id": "u-100"}
• get another validation error

The fix is to keep each key in the format required by the API layer that produced it.

Composite Keys Must Be Complete

If the table uses both a partition key and a sort key, ExclusiveStartKey must include both. Partial keys are not enough for pagination.

For example, if the table key schema is:

• partition key: user_id
• sort key: created_at

then a valid start key must contain both values.

That is why passing LastEvaluatedKey through directly is so reliable. DynamoDB already knows the exact key shape that belongs to the last evaluated item.

Serialization Problems in Python

Another source of trouble is serializing the key to JSON for storage or transport. DynamoDB numeric values often interact with Python Decimal objects, and naive JSON conversion can change types.

For example, a sort key that should stay numeric can accidentally become a string during a custom JSON round trip. When that corrupted key comes back as ExclusiveStartKey, pagination breaks.

If you need to persist the token outside the immediate request flow, preserve the exact shape carefully and restore it with the same type semantics.

Use DynamoDB Serializers Only When Necessary

If you genuinely need to translate plain Python dictionaries into low-level DynamoDB typed format, use boto3's serializers instead of hand-writing the conversion.

1from boto3.dynamodb.types import TypeSerializer
2
3serializer = TypeSerializer()
4python_key = {
5    "user_id": "u-100",
6    "created_at": 1720992000,
7}
8client_key = {k: serializer.serialize(v) for k, v in python_key.items()}
9print(client_key)

This is much safer than manually building {"S": ...} and {"N": ...} structures everywhere.

Common Pitfalls

Mixing resource-style keys and client-style typed keys is the biggest source of this problem.

Rebuilding ExclusiveStartKey manually instead of reusing LastEvaluatedKey also creates unnecessary failure points.

Forgetting the sort key on a composite-key table is another common issue.

Finally, careless JSON serialization can corrupt number types and make a previously valid pagination token unusable.

Summary

• 'ExclusiveStartKey must match the exact key format expected by the API you are calling'
• reuse LastEvaluatedKey directly whenever possible
• resource API uses plain Python values, while client API uses DynamoDB typed JSON
• composite-key tables require the full key, not only the partition key
• if you must serialize keys, preserve types carefully or use DynamoDB serializers