Python
Semaphores
Named Semaphores
Concurrency
Multithreading

Named semaphores in Python?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Python's standard library gives you semaphores for threads and related processes, but it does not expose a portable, high-level API for OS-level named semaphores. If two completely independent processes need to coordinate through a shared system object identified by name, you usually need a platform-specific solution.

What a Named Semaphore Is

A regular semaphore is just a counter with acquire and release. A named semaphore adds one important property: the synchronization object lives in the operating system and can be opened by name from unrelated processes.

That matters when the processes do not share a common parent and cannot inherit Python objects directly. For example:

  • one script starts from a scheduler,
  • another starts from a terminal,
  • both need to limit access to the same device or file-processing slot.

A named semaphore is a good fit there because both processes can agree on the same name.

What the Standard Library Can Do

The threading and multiprocessing modules include semaphore types:

python
from multiprocessing import Semaphore

sem = Semaphore(2)

That object works well when it is created in one process and shared with child processes through multiprocessing. It is not the same thing as a globally named semaphore that any arbitrary process can open later by a string name.

So if the question is "does Python have built-in named semaphores," the practical answer is no, not as a portable standard-library feature.

Using posix_ipc on Unix-Like Systems

On Linux and other Unix-like systems, a common approach is the posix_ipc package. It exposes POSIX named semaphores directly.

python
1import posix_ipc
2import time
3
4sem = posix_ipc.Semaphore("/image-import-lock", flags=posix_ipc.O_CREAT, initial_value=2)
5
6try:
7    sem.acquire()
8    print("entered critical section")
9    time.sleep(3)
10finally:
11    sem.release()

A second independent process can open the same semaphore by name:

python
1import posix_ipc
2
3sem = posix_ipc.Semaphore("/image-import-lock")
4sem.acquire()
5print("another process entered")
6sem.release()

Both programs coordinate through the same OS object because the name matches.

Cleanup Matters

Named semaphores can outlive the creating process. That is useful, but it also means cleanup matters. On POSIX systems, you often want to unlink the named semaphore when the application is truly done with it.

python
1import posix_ipc
2
3sem = posix_ipc.Semaphore("/image-import-lock", flags=posix_ipc.O_CREAT, initial_value=1)
4sem.unlink()

Unlinking removes the name from the system namespace. Existing handles may still function until they close, but future opens by name will fail.

Alternatives When You Need Portability

If your application must run on multiple operating systems, a named semaphore may not be the easiest cross-platform abstraction. Depending on the problem, one of these may be simpler:

  • a file lock,
  • a local TCP service that grants slots,
  • Redis-based locking,
  • a database row used as a coordination primitive,
  • 'multiprocessing semaphores if all workers share the same parent process.'

In many applications, the real need is "only let two workers run at once," not specifically "use a named semaphore." Once you phrase it that way, other coordination tools may be more practical.

When to Use a Named Semaphore

A named semaphore is appropriate when:

  • the processes are independent,
  • the coordination scope is one machine,
  • you need lightweight counting semantics,
  • you are comfortable with OS-specific behavior.

It is less attractive when you need portability, distributed locking, or visibility into ownership and timeouts across many services.

Common Pitfalls

The most common mistake is assuming multiprocessing.Semaphore is automatically system-wide and name-addressable. It is not. It is designed for processes that already share Python-managed state.

Another problem is forgetting cleanup. A stale named semaphore can remain in the system after a crash, which may confuse later runs if your naming and initialization logic are sloppy.

Developers also underestimate platform differences. A solution based on POSIX semaphores is fine on Unix-like systems, but it is not a portable answer for every environment.

Summary

  • Python's standard library does not offer a portable named-semaphore API for unrelated processes.
  • 'multiprocessing.Semaphore works for related processes, not arbitrary named access.'
  • On Unix-like systems, posix_ipc is a practical way to use POSIX named semaphores.
  • Cleanup and naming conventions matter because named objects can outlive the creating process.
  • If portability matters more than OS-level primitives, consider higher-level locking strategies instead.

Course illustration
Course illustration