Introduction

With Amazon Bedrock, "async" can mean two different things, and mixing them up causes confusion. One meaning is application-level concurrency, where your program issues multiple requests without blocking its own event loop. The other is Bedrock's service-level asynchronous inference API, where you start a job, let AWS process it in the background, and later poll for completion.

Know which async model you need

If you want standard text generation requests to run concurrently inside a Python service, you can keep using the Bedrock runtime client and offload blocking SDK calls from the event loop. If you need a true background inference job with persisted output, Bedrock provides async invocation APIs such as start_async_invoke and get_async_invoke.

Those are different tools for different workloads:

• request concurrency for many normal online calls
• job-style async processing for long-running or background tasks

Application-level async with Python

The Python AWS SDK is typically synchronous, so a common pattern is to call it from worker threads while still coordinating work with asyncio.

1import asyncio
2import json
3import boto3
4
5client = boto3.client("bedrock-runtime", region_name="us-east-1")
6
7def invoke_once(prompt: str):
8    body = json.dumps({
9        "inputText": prompt,
10        "textGenerationConfig": {"maxTokenCount": 128}
11    })
12
13    response = client.invoke_model(
14        modelId="amazon.titan-text-express-v1",
15        body=body,
16        contentType="application/json",
17        accept="application/json",
18    )
19    return response["body"].read().decode("utf-8")
20
21async def invoke_async(prompt: str):
22    return await asyncio.to_thread(invoke_once, prompt)
23
24async def main():
25    results = await asyncio.gather(
26        invoke_async("Summarize this text."),
27        invoke_async("Write a short heading."),
28    )
29    print(results)
30
31asyncio.run(main())

This does not turn the AWS API itself into a magical async protocol. It simply lets your application handle multiple blocking calls concurrently.

Service-level async invocation

Bedrock also supports true asynchronous invocation for supported workloads through start_async_invoke. In that pattern, you submit the job and provide an output location, commonly in S3.

1import boto3
2
3client = boto3.client("bedrock-runtime", region_name="us-east-1")
4
5response = client.start_async_invoke(
6    modelId="amazon.nova-reel-v1:0",
7    modelInput={
8        "taskType": "TEXT_VIDEO",
9        "textToVideoParams": {"text": "A calm sunrise over a lake"}
10    },
11    outputDataConfig={
12        "s3OutputDataConfig": {
13            "s3Uri": "s3://my-bedrock-output-bucket/jobs/"
14        }
15    },
16)
17
18invocation_arn = response["invocationArn"]
19print(invocation_arn)

Then poll status later:

status = client.get_async_invoke(invocationArn=invocation_arn)
print(status["status"])

This is the right model when the output may take long enough that you do not want to hold an HTTP request open.

Choose the right pattern for the workload

Use concurrent normal calls when:

• requests are short-lived
• you need low-latency interactive responses
• your own service just needs better throughput

Use Bedrock async jobs when:

• the workload is long-running
• output should land in S3
• the caller can tolerate polling or callback-style completion handling

Trying to force every use case into one pattern usually makes the architecture worse.

In other words, the right answer depends on where you want the waiting to happen. You can hide blocking inside your application with concurrency, or you can move the whole job lifecycle into Bedrock and treat completion as a separate event.

Common Pitfalls

The biggest mistake is assuming asyncio alone makes the boto3 client non-blocking. Without wrapping the call in threads or another concurrency mechanism, the SDK call still blocks.

Another mistake is confusing concurrent online invocation with Bedrock's async job APIs. They solve different operational problems.

Developers also forget that service-level async workflows usually need S3 output configuration and job-status handling. They are not just invoke_model with a different function name.

Finally, be careful with rate limits and concurrency. Async code can increase throughput quickly, but it can also increase request pressure just as quickly.

Summary

• Bedrock async work can mean concurrent client calls or true background inference jobs.
• For normal request concurrency in Python, wrap blocking SDK calls with asyncio.to_thread or similar techniques.
• For long-running supported workloads, use start_async_invoke and later get_async_invoke.
• Pick the pattern based on latency, output handling, and operational needs.
• Async improves throughput only when the surrounding architecture handles concurrency and job tracking correctly.