With statement and python threading
Interview Questions practice on Codemia
Over 8,000 real interview questions from top companies, searchable by company and role.
Introduction
Python's with statement is a context manager that ensures resources are properly acquired and released, even when exceptions occur. In threading, with is used to acquire and release locks, semaphores, and conditions automatically. Writing with lock: is equivalent to lock.acquire() followed by a try/finally block with lock.release(). This pattern prevents common threading bugs like forgetting to release a lock after an exception, which would deadlock all other threads waiting for that lock.
Using with for Thread Locks
The with statement guarantees the lock is released even if an exception occurs inside the block.
Thread-Safe Counter Example
RLock (Reentrant Lock)
An RLock allows the same thread to acquire the lock multiple times without deadlocking.
Use RLock when a function that holds a lock calls another function that also needs the same lock.
Semaphore
A Semaphore limits the number of threads that can access a resource simultaneously.
Condition Variable
A Condition lets threads wait for a specific state change.
Event
An Event is a simple flag for thread signaling. It does not use the with statement (it has no acquire/release) but is commonly used alongside locks.
Custom Context Manager for Threading
Common Pitfalls
- Forgetting to release a lock without
with: If code betweenlock.acquire()andlock.release()throws an exception, the lock is never released and all other threads deadlock. Always usewith lock:instead of manual acquire/release. - Holding a lock during I/O or sleep: Acquiring a lock and then performing network I/O, file I/O, or
time.sleep()inside thewithblock starves other threads. Keep the critical section as short as possible — only protect shared state access, not I/O operations. - Deadlock from acquiring multiple locks in different orders: Thread A acquires lock1 then lock2; Thread B acquires lock2 then lock1 — deadlock. Always acquire multiple locks in a consistent global order, or use
threading.RLockwhen nesting is unavoidable. - Using
LockwhenRLockis needed: If a function that holds aLockcalls another function that also tries to acquire the sameLock, the thread deadlocks on itself. UseRLockfor reentrant locking scenarios. - GIL misconception: Python's Global Interpreter Lock (GIL) does not make your code thread-safe. The GIL prevents concurrent Python bytecode execution but does not prevent race conditions on multi-step operations (e.g.,
counter += 1is multiple bytecodes). You still need explicit locks for shared mutable state.
Summary
- Use
with lock:to automatically acquire and release threading locks safely Lockfor simple mutual exclusion,RLockfor reentrant (nested) lockingSemaphorelimits concurrent access to a fixed number of threadsConditionlets threads wait for state changes withwait()andnotify()- Keep critical sections short — do not hold locks during I/O or sleep
Related reading
- With WPF, is there a method to detect method calls that are blocking GUI updates?
- Wordpress blocks async REST calls in registerBlockType edit function
- Working of physical clock synchronization in distributed systems
- Would Asyncronous TCP networking be a better option for a dedicated server with up to 32 players playing at the same time in C?
- Working of labelEncoder in sklearn
- Working outside of request context error with Celery background task
- WPF background operations using Asynchronous Workflows
- WPF updating the itemssource in a ListBox with an async method
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
Interview Questions practice on Codemia
Over 8,000 real interview questions from top companies, searchable by company and role.