How to limit the maximum number of parallel tasks in C

multithreading

parallel programming

task management

concurrency

How to limit the maximum number of parallel tasks in C

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Start Practicing Learn More

Introduction

In C#, the three main ways to limit parallel tasks are SemaphoreSlim for async/await patterns, Parallel.ForEach with MaxDegreeOfParallelism for CPU-bound work, and Channel<T> or ActionBlock<T> for producer-consumer patterns. SemaphoreSlim is the most flexible and works with async code. Parallel.ForEach is simpler but blocks the calling thread. Choose based on whether your workload is I/O-bound (use SemaphoreSlim) or CPU-bound (use Parallel.ForEach).

SemaphoreSlim (Async/Await)

The most common approach for limiting concurrent async operations:

csharp

1using System;
2using System.Collections.Generic;
3using System.Net.Http;
4using System.Threading;
5using System.Threading.Tasks;
6
7public async Task ProcessUrlsAsync(IEnumerable<string> urls, int maxConcurrency = 5)
8{
9    var semaphore = new SemaphoreSlim(maxConcurrency);
10    var tasks = new List<Task>();
11
12    foreach (var url in urls)
13    {
14        await semaphore.WaitAsync();  // Wait if at max concurrency
15
16        tasks.Add(Task.Run(async () =>
17        {
18            try
19            {
20                using var client = new HttpClient();
21                var response = await client.GetStringAsync(url);
22                Console.WriteLine($"Downloaded {url}: {response.Length} chars");
23            }
24            finally
25            {
26                semaphore.Release();  // Always release, even on exception
27            }
28        }));
29    }
30
31    await Task.WhenAll(tasks);
32}

SemaphoreSlim(5) allows at most 5 tasks to execute concurrently. When a task finishes and calls Release(), the next waiting task proceeds.

Parallel.ForEach with MaxDegreeOfParallelism

For CPU-bound synchronous work:

csharp

1var items = Enumerable.Range(1, 100).ToList();
2
3var options = new ParallelOptions
4{
5    MaxDegreeOfParallelism = 4  // At most 4 threads
6};
7
8Parallel.ForEach(items, options, item =>
9{
10    // CPU-bound work
11    var result = HeavyComputation(item);
12    Console.WriteLine($"Processed {item}: {result}");
13});
14
15// All items processed when ForEach returns

MaxDegreeOfParallelism limits the number of threads used by the Parallel.ForEach loop. Set it to Environment.ProcessorCount to match the number of CPU cores.

Parallel.ForEachAsync (.NET 6+)

Combines Parallel.ForEach with async support:

csharp

1var urls = new List<string> { "https://example.com", /* ... */ };
2
3await Parallel.ForEachAsync(urls, new ParallelOptions
4{
5    MaxDegreeOfParallelism = 5
6}, async (url, cancellationToken) =>
7{
8    using var client = new HttpClient();
9    var html = await client.GetStringAsync(url, cancellationToken);
10    Console.WriteLine($"Downloaded {url}: {html.Length} chars");
11});

This is the cleanest approach in .NET 6+ — it handles semaphore-like limiting internally and supports CancellationToken.

ActionBlock from TPL Dataflow

ActionBlock<T> from the System.Threading.Tasks.Dataflow NuGet package provides built-in concurrency limiting:

csharp

1using System.Threading.Tasks.Dataflow;
2
3var block = new ActionBlock<string>(async url =>
4{
5    using var client = new HttpClient();
6    var html = await client.GetStringAsync(url);
7    Console.WriteLine($"Downloaded {url}");
8}, new ExecutionDataflowBlockOptions
9{
10    MaxDegreeOfParallelism = 5
11});
12
13foreach (var url in urls)
14{
15    await block.SendAsync(url);
16}
17
18block.Complete();
19await block.Completion;

Channel-Based Producer-Consumer

csharp

1using System.Threading.Channels;
2
3var channel = Channel.CreateBounded<string>(new BoundedChannelOptions(100)
4{
5    FullMode = BoundedChannelFullMode.Wait
6});
7
8// Start limited number of consumers
9var consumers = Enumerable.Range(0, 5).Select(async _ =>
10{
11    await foreach (var url in channel.Reader.ReadAllAsync())
12    {
13        using var client = new HttpClient();
14        await client.GetStringAsync(url);
15        Console.WriteLine($"Processed {url}");
16    }
17}).ToArray();
18
19// Producer: write items to the channel
20foreach (var url in urls)
21{
22    await channel.Writer.WriteAsync(url);
23}
24
25channel.Writer.Complete();
26await Task.WhenAll(consumers);

Exactly 5 consumer tasks process items from the channel concurrently.

Comparison

Approach	Async Support	Best For	.NET Version
`SemaphoreSlim`	Yes	I/O-bound async work	All
`Parallel.ForEach`	No	CPU-bound sync work	All
`Parallel.ForEachAsync`	Yes	I/O-bound (simplest API)	.NET 6+
`ActionBlock<T>`	Yes	Producer-consumer pipelines	NuGet
`Channel<T>` + consumers	Yes	Custom backpressure	.NET Core 3.0+

With Cancellation Support

csharp

1var cts = new CancellationTokenSource();
2cts.CancelAfter(TimeSpan.FromSeconds(30));  // Timeout after 30s
3
4var semaphore = new SemaphoreSlim(5);
5var tasks = new List<Task>();
6
7foreach (var url in urls)
8{
9    await semaphore.WaitAsync(cts.Token);  // Respects cancellation
10
11    tasks.Add(Task.Run(async () =>
12    {
13        try
14        {
15            using var client = new HttpClient();
16            var html = await client.GetStringAsync(url, cts.Token);
17        }
18        finally
19        {
20            semaphore.Release();
21        }
22    }, cts.Token));
23}
24
25try
26{
27    await Task.WhenAll(tasks);
28}
29catch (OperationCanceledException)
30{
31    Console.WriteLine("Operation was cancelled");
32}

Common Pitfalls

Forgetting to Release the semaphore: If an exception occurs before semaphore.Release(), the semaphore count decreases permanently, eventually deadlocking the application. Always release in a finally block.
Using Parallel.ForEach with async lambdas: Parallel.ForEach does not await async lambdas — it treats them as fire-and-forget. Use Parallel.ForEachAsync (.NET 6+) or SemaphoreSlim for async work.
Setting MaxDegreeOfParallelism too high for I/O: For I/O-bound tasks (HTTP calls, database queries), more parallelism does not always mean faster. It can overwhelm the target server or exhaust connection pools. Start with 5-10 and tune based on results.
Task.WhenAll with no semaphore: Task.WhenAll(items.Select(ProcessAsync)) launches ALL tasks immediately with no concurrency limit. If you have 10,000 URLs, all 10,000 HTTP requests fire at once. Always use a semaphore or Parallel.ForEachAsync to limit concurrency.
Disposing SemaphoreSlim while tasks are waiting: Disposing the semaphore while tasks are blocked on WaitAsync() throws ObjectDisposedException. Wait for all tasks to complete before disposing the semaphore.

Summary

Use SemaphoreSlim with WaitAsync/Release for async concurrency limiting
Use Parallel.ForEach with MaxDegreeOfParallelism for synchronous CPU-bound work
Use Parallel.ForEachAsync (.NET 6+) for the simplest async concurrency limiting API
Always release semaphores in finally blocks to prevent deadlocks
Choose concurrency limits based on the workload type — CPU cores for CPU-bound, lower limits for I/O-bound