Introduction

If you need to limit how many threads run at the same time, the usual answer is not to create threads freely and then try to stop them. The right design is to submit work to a bounded executor or thread pool and let that pool enforce the concurrency limit.

Limit Concurrency At The Scheduler, Not Inside Every Task

A common anti-pattern looks like this:

• spawn a thread for each task
• have each thread check whether too many others are running
• block or sleep until a slot is available

That design creates exactly the resource pressure you were trying to avoid. If you already created 5,000 threads, the damage is done even if 4,900 of them are sleeping.

The better design is:

1. represent work as tasks
2. submit tasks to a queue
3. let a fixed-size pool run only N tasks concurrently

Use A Thread Pool

A thread pool limits concurrency naturally because it owns a fixed number of worker threads.

1from concurrent.futures import ThreadPoolExecutor
2import time
3
4
5def work(item):
6    print(f"start {item}")
7    time.sleep(1)
8    print(f"end {item}")
9    return item * 2
10
11
12with ThreadPoolExecutor(max_workers=3) as executor:
13    results = list(executor.map(work, range(6)))
14
15print(results)

Even though six tasks are submitted, only three run at once because max_workers=3 is the concurrency limit.

This is usually the cleanest solution in Python, Java, C#, and most modern runtimes.

Semaphores Are Useful When Work Is Already Threaded

If threads already exist and you need to restrict access to a limited resource, a semaphore is a good fit.

1import threading
2import time
3
4limit = threading.Semaphore(2)
5
6
7def work(name):
8    with limit:
9        print(f"{name} entered")
10        time.sleep(1)
11        print(f"{name} leaving")
12
13
14threads = [threading.Thread(target=work, args=(f"t{i}",)) for i in range(5)]
15for t in threads:
16    t.start()
17for t in threads:
18    t.join()

At most two threads hold the semaphore at once.

This is not a replacement for a thread pool when task submission is under your control, but it is useful when concurrency must be limited around a specific section such as database access or API calls.

Queue Length Matters Too

Limiting active threads is only part of the problem. If tasks are submitted faster than workers can finish them, the queue may grow without bound.

That means a production-quality design often needs both:

• a fixed number of workers
• a bounded queue or backpressure strategy

For example, a server might reject new work, slow producers down, or drop low-priority tasks once the queue is full.

Without that second control, memory usage can still explode even though thread count is capped.

Pick The Limit Based On The Workload

There is no universal "correct" number of threads.

For CPU-bound work, the limit is often close to the number of available CPU cores. More threads than cores usually increases context switching rather than throughput.

For I/O-bound work, a higher limit can make sense because many threads spend time waiting on network or disk operations.

So the right question is not "what is the maximum safe number of threads," but "what is the throughput and latency target for this workload on this machine?"

Measure with realistic load rather than copying a rule of thumb blindly.

Prefer Higher-Level Concurrency APIs

Languages usually provide better tools than manual thread management.

In Java, prefer ExecutorService or a bounded ThreadPoolExecutor.

1ExecutorService pool = Executors.newFixedThreadPool(4);
2for (int i = 0; i < 10; i++) {
3    int taskId = i;
4    pool.submit(() -> System.out.println("task " + taskId));
5}
6pool.shutdown();

The same principle applies in other languages: use the runtime's executor abstraction instead of manually creating raw threads whenever possible.

Common Pitfalls

The most common mistake is creating one thread per task and trying to limit execution afterward. That does not actually control resource creation.

Another mistake is using a semaphore without thinking about queue growth. You may cap active work and still run out of memory.

Developers also often choose pool sizes without considering whether the workload is CPU-bound or I/O-bound.

Finally, if the task is mostly waiting on external I/O, you may need asynchronous I/O instead of more threads.

Summary

• Use a fixed-size thread pool to limit concurrent task execution.
• Use semaphores when you need to guard a specific shared resource.
• Control queue growth, not just active thread count.
• Size pools based on CPU-bound versus I/O-bound behavior.
• Prefer executor abstractions over manual thread creation.