I would like to make/have a scala like 'future' async API for python
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
If you want a Scala-like Future API in Python, the real goal is usually composable asynchronous work with callbacks, mapping, and error propagation. Python already has the primitives for that in asyncio and concurrent.futures, but the API style is different, so building a thin wrapper can give you the same fluent feel without fighting the runtime.
Decide Which Python Concurrency Model You Want
Scala Future is often used for non-blocking workflows built around composition. In Python, there are two common foundations:
- '
asyncio, which is best when your application already has an event loop' - '
concurrent.futures, which is best for thread-pool or process-pool execution'
If you want Scala-like chaining in modern Python applications, asyncio is usually the better base because cancellation and composition are built in.
Build a Small Wrapper Around asyncio.Future
A practical approach is to wrap an awaitable and expose methods like map, flat_map, and recover. The wrapper does not replace asyncio; it packages it into a more functional API.
This gives you a familiar composition surface while still using the standard event loop underneath.
Compose Work in a Scala-Like Style
Once you have the wrapper, chaining reads naturally.
For dependent async steps, use flat_map.
That separates value transformation from asynchronous sequencing in the same way Scala users expect.
Keep Cancellation and Backpressure in Mind
A fluent API is not enough by itself. Real async systems also need cancellation, timeouts, and bounded concurrency. If your wrapper hides those concerns too deeply, it becomes harder to operate in production.
For example, timeouts should still be explicit:
This is one place where staying close to asyncio pays off. Your Scala-like API should complement the runtime, not replace its operational tools.
When concurrent.futures Is the Better Base
If the work is blocking and you want background execution from threads or processes, you can wrap concurrent.futures.Future instead. That is common for CPU-heavy or legacy blocking code.
In that world, you can still add a map-style wrapper, but remember that thread-based futures and asyncio futures are not interchangeable without adapters.
Common Pitfalls
One mistake is recreating a Scala-style API but forgetting Python's execution model. Threads, event loops, and coroutines have different tradeoffs, so the wrapper must stay honest about what it is built on.
Another issue is mixing blocking I/O into an asyncio-based future chain. That stalls the event loop and defeats the purpose of a non-blocking API.
It is also easy to over-design the abstraction. If all you need is await, asyncio.gather, and a few helper functions, adding a custom future type may increase complexity more than it helps.
Summary
- A Scala-like future API in Python is best implemented as a thin wrapper over existing primitives.
- '
asynciois the best foundation for composable non-blocking workflows.' - Expose methods like
map,flat_map, andrecoverwithout hiding cancellation and timeouts. - Use
concurrent.futuresonly when the workload is thread-pool or process-pool based. - Keep the wrapper small so it works with Python's runtime instead of fighting it.
Related reading
- ICE - How to cast implementation to proxy?
- IDE breakpoint in TensorFlow Dataset API mapped py_function?
- Idempotency and Race Condition on REST API in a Distributed System
- Implementation consistent replica in peer-to-peer application
- ICommand.CanExecute async
- If a communication paradigm is asynchronous, is it also time-uncoupled?
- ''id'' is a bad variable name in Python
- Identifying the dependency relationship for python packages installed with pip

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.