Parallel execution of a loop that uses async
Master System Design with Codemia
Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.
Introduction
When a loop calls asynchronous work, the important question is whether you want each iteration to wait before starting the next one. If the iterations are independent, the normal pattern is to start all the tasks first and then await them together, rather than awaiting inside the loop body one item at a time.
Sequential await Versus Concurrent Scheduling
This loop is asynchronous, but not concurrent:
Each call waits for the previous one to finish, so the total time is roughly the sum of all delays. If each iteration is independent and mostly I/O-bound, that is usually slower than necessary.
The concurrent pattern is:
Now the operations overlap, so the total time is closer to the longest individual task instead of the sum of all tasks.
async Is Concurrency, Not Automatic CPU Parallelism
This distinction matters. async helps most when the loop body spends time waiting on network, disk, database, or other non-CPU work. It does not magically make CPU-heavy loops run faster on multiple cores.
If the body is CPU-bound, use:
- multiprocessing
- threads where the runtime allows it
- native vectorized or compiled code
For I/O-heavy workloads, though, async concurrency is often the right tool.
Limit Concurrency When Needed
Launching every task at once is not always safe. If the loop can create hundreds or thousands of requests, you may need a semaphore or worker pool to limit concurrency:
This still overlaps work, but it does not overload the remote service or local runtime by opening too many operations at once.
Preserve Order or Handle Results as They Finish
asyncio.gather preserves the order of the original task list. That is useful when output order matters. If you want to process results as soon as each task completes, use asyncio.as_completed instead:
That is useful for streaming results, partial progress updates, or reducing memory pressure when each result can be handled immediately.
Error Handling Still Matters
If one async task fails, you need to decide whether the whole loop should fail or whether errors should be collected and handled individually. gather can do either:
This is often better than letting one failed iteration cancel the entire batch unexpectedly.
Common Pitfalls
The most common mistake is writing await directly inside the loop and assuming the loop is now running in parallel. It is asynchronous, but still sequential unless tasks are scheduled before waiting.
Another pitfall is using async for CPU-bound work and expecting speedups. Async overlaps waiting; it does not replace real parallel computation for heavy CPU tasks.
It is also easy to create too many tasks at once. Concurrency without limits can overload memory, saturate sockets, or trigger rate limits upstream.
Finally, do not ignore error policy. A concurrent loop needs a clear rule for what should happen when one task fails while others are still running.
Summary
- '
awaitinside a loop is usually sequential unless tasks are started first.' - For independent I/O-bound iterations, schedule tasks and await them together.
- Use semaphores or worker limits when unbounded concurrency would be unsafe.
- Use
gatherfor ordered results andas_completedfor processing results as they finish. - Async improves overlap for waiting tasks, not CPU-heavy computation by itself.

