With our read and write paths clearly defined, we can begin sketching the high-level design. There’s no need to start with an overly complicated diagram. Begin with something simple and refine it as you optimize your design. A straightforward starting point often involves a basic setup: a client making a request to a service, which then interacts with the database for both the read and write paths. This basic design provides clarity and allows for gradual enhancements as you address specific challenges and requirements.

In our Short URL service, users submit a URL to the URL Generation Service, which stores the mapping in a database. Our initial design could looks as follows:

This design basically captures both the read path (redirecting users to the original URL) and the write path (storing a new URL mapping). However, as we scale and handle more traffic, this setup will encounter scalability challenges, particularly given that the service is read-heavy.

To address potential bottlenecks and improve performance, we introduce two key system design components: an API Gateway and a Cache.

API Gateway

The API Gateway acts as a centralized entry point for client requests, streamlining operations and enhancing scalability. Its key functions include:

• Routing: Directs requests to the appropriate service, whether it's URL generation or redirection.
• Rate Limiting: limits excessive usage.

Cache

Introducing a cache significantly reduces database load by storing frequently accessed short-to-long URL mappings. The cache improves performance by:

• Serving redirect requests directly when the mapping exists in the cache, avoiding database queries.
• Supporting low-latency, high-throughput operations with technologies like Redis.
• Using efficient eviction policies (e.g., LRU) to manage storage and prioritize popular mappings.

The enhanced system design incorporates these components to handle high traffic more effectively:

This updated design improves scalability by reducing direct database queries and managing traffic more efficiently. The API Gateway ensures robust request handling, while the cache delivers faster responses for popular URLs.

This should be sufficient for high level design of Short URL Service with room for deep dive topics later on. Let's move on to Twitter's high level design next.

The initial design of a Twitter service might look something like this:

In this design, the Home Feed Service fetches data directly from the database and computes the live feed for each user on-the-fly. While simple, this approach presents significant scalability challenges. As the user base grows, computing the feed in real time becomes computationally expensive and introduces high latency, resulting in poor user experience.

To address these issues, we need a more scalable approach.

A common pattern in systems with high computational costs is to precompute data that needs to be fetched later. This pattern reduces the need for real-time computation by offloading expensive operations to asynchronous processes.

Real-world Examples:

• Video Streaming Services: Transcode video streams for different resolutions and formats ahead of time to avoid real-time delays.
• Social Media Notifications: Precompute notification lists (e.g., likes, hashtag, comments, mentions) instead of generating them dynamically for millions of users.
• Advertising Platforms: Precompute bidding results, ad rankings, and user segmentation for fast delivery of targeted ads.

The same reasoning applies to the Twitter home feed. Generating the feed dynamically for each request is prohibitively expensive, so we can precompute it.

The pattern usually goes like this:

Client post some data by calling some service. This service writes data to the database, then sends the data as an event to message queue for async processing. A separate service will then pick up the event from the queue and process it. After processing, it will store the data into a cache or database or both depending on the trade offs we are willing to make. The client can then fetch the precomputed data without waiting for the service to process it in real time.

Using this pattern, in our evolved system the client will make a request to Tweet service, the tweet service will store the data to database and then send it as an event to the message queue. A separate service, which in this case is the Home Feed Generator will pick up this event from the queue and process it. It will then store it to both the database and cache. The client will then fetch home feed via the Home Feed Service without the need to compute it in real time.

Below is our upgraded design:

Notice how we added an API Gateway here as well! By reusing a component we know works effectively, we leverage its proven benefits and apply them to the Twitter service.

As mentioned in our social media notifications example, we can also do pre-computation for elements such as hashtag, mentions and etc. Let's create a tweet analyzer service for that.

Identifying scalability challenges and resolving it during an interview is something we can practice and learn.

These patterns are not limited to specific services but can be generalized and adapted to solve various other problems. For example, the same pattern used here for Twitter’s home feed could also apply to Facebook’s feed or other recommendation systems. This means if you can solve this problem you can solve other similar types of problems too!

At the staff level or higher, you're unlikely to be tasked with solving common system design problems. Instead, staff level engineers will often face challenges that are variations of well-known problems, with unique twists or constraints. In such scenarios, it becomes even more important to develop the ability to recognize underlying patterns and effectively apply them to novel situations.