System requirements

Functional Requirements:

The system must recommend the most relevant 5 to 10 keywords in response to a user's search input.

Non-Functional Requirements:

• Speed: The system must deliver these suggestions instantly, with a response time not exceeding 200 milliseconds after the user inputs their query.
• Reliability: The system must continue to offer recommendations even if one or more of its components fail.
• Capacity for Growth: The system must be able to accommodate a growing user base over time.

Capacity estimation

Based on the assumption of processing 3.5 billion queries akin to Google's volume, with 2 billion of these being unique and requiring storage. Given that the average query length is 15 characters, and each character occupies 2 bytes, the calculation for daily storage needs is as follows: 2 billion queries multiplied by 15 characters, further multiplied by 2 bytes, equals 60 billion bytes or 60GB for all daily queries. Therefore, the annual storage requirement is calculated as 60GB per day multiplied by 365 days, totaling 21.9 terabytes per year.

Bandwidth Estimation:

Incoming Bandwidth:

The daily influx consists of 3.5 billion queries, each query averaging 15 characters at 2 bytes per character, summing up to 105 billion bytes or 105GB per day. Breaking this down further into seconds gives us a rate of approximately 1.2GB per second, or equivalently, 10 gigabits per second (10Gb/s).

Outgoing Bandwidth:

Considering the system provides 5 to 10 suggestions per query, the maximum outgoing bandwidth could reach up to 10 times 1.2GB per second, which equates to 12GB/s.

API design

GET /suggestions/{prefix}

This endpoint retrieves suggestions for queries that begin with the specified "prefix." It returns a JSON object with an array of suggestions, each containing an "id" and the "suggestion" itself. For example:

POST /add/{query}

This endpoint is used to insert a "query" into the database if it qualifies as trending, meaning it has reached a specific popularity threshold through repeated searches.

Database design

• Node table:
• Node ID: Integer representing the unique identifier for each node.
• Node Value: String representing the value of the node.
• Words: Array of Strings representing the words associated with the node.

• Edge table:
• Edge ID: Integer representing the unique identifier for each edge.
• Parent Node: Integer representing the parent node in the graph.
• Children Nodes: Array of Integers representing the children nodes connected to the parent node.

High-level design

• User Input Initiation: The journey begins with the user typing in the search box on the UI. The frontend captures these keystrokes in real-time.
• Frontend Debouncing: To optimize the number of requests sent to the backend, the frontend implements debouncing. This technique waits for a short break in typing before sending a request, thus reducing backend load and enhancing system performance.
• Backend API Communication: Following the debouncing, the frontend communicates with the backend via an API Endpoint, transmitting the user's current input for suggestions.
• Request Distribution via Load Balancing: In scalable systems, an initial load balancer distributes the incoming requests to backend servers, ensuring efficient resource use and quick responses.
• Suggestion Retrieval Process: The Suggestion Service checks the Distributed Cache for existing suggestions related to the input. If found (cache hit), it immediately returns these suggestions, speeding up the response.
• Cache Miss and Database Query: In the absence of cached suggestions (cache miss), the service queries the Database, which holds a structured index (like a Trie) for generating suggestions.
• Data Ranking and Response: The service applies a Ranking Algorithm to the database results to prioritize suggestions based on criteria such as popularity and relevance. It then prepares these suggestions for the response.
• Caching and Response Optimization: Prior to responding to the frontend, the Suggestion Service may cache the new suggestions to expedite future requests with similar inputs.
• Displaying Suggestions on the UI: The frontend receives the suggestions from the API Endpoint and displays them below the input field, allowing the user to select from a dropdown list.
• System Health Monitoring: An ongoing monitoring system tracks performance metrics and alerts the operations team about any detected issues, ensuring the system remains efficient and reliable.

Request flows

• User Input Initiation: The journey begins with the user typing in the search box on the UI. The frontend captures these keystrokes in real-time.
• Frontend Debouncing: To optimize the number of requests sent to the backend, the frontend implements debouncing. This technique waits for a short break in typing before sending a request, thus reducing backend load and enhancing system performance.
• Backend API Communication: Following the debouncing, the frontend communicates with the backend via an API Endpoint, transmitting the user's current input for suggestions.
• Request Distribution via Load Balancing: In scalable systems, an initial load balancer distributes the incoming requests to backend servers, ensuring efficient resource use and quick responses.
• Suggestion Retrieval Process: The Suggestion Service checks the Distributed Cache for existing suggestions related to the input. If found (cache hit), it immediately returns these suggestions, speeding up the response.
• Cache Miss and Database Query: In the absence of cached suggestions (cache miss), the service queries the Database, which holds a structured index (like a Trie) for generating suggestions.
• Data Ranking and Response: The service applies a Ranking Algorithm to the database results to prioritize suggestions based on criteria such as popularity and relevance. It then prepares these suggestions for the response.
• Caching and Response Optimization: Prior to responding to the frontend, the Suggestion Service may cache the new suggestions to expedite future requests with similar inputs.
• Displaying Suggestions on the UI: The frontend receives the suggestions from the API Endpoint and displays them below the input field, allowing the user to select from a dropdown list.
• System Health Monitoring: An ongoing monitoring system tracks performance metrics and alerts the operations team about any detected issues, ensuring the system remains efficient and reliable.

Detailed component design

• User Interaction and Popularity Assessment:
• When a user engages with a suggestion by selecting it or conducting a search, this activity is captured by the frontend and communicated back to the backend. This action serves as an indicator of user interest in specific queries.
• The backend system, which may be a distinct service or integrated within the Suggestion Service, monitors the popularity of each query by logging how often and frequently it's searched.
• Initiating a Trending Query Addition:
• Upon identifying a query that surpasses a set benchmark of popularity or engagement, the backend sends a POST request to the /add/{query} endpoint. This can be an internal process or configured to accept submissions for adding trending searches. The request carries the query recognized as sufficiently popular to join the trending suggestions list.
• Popularity Verification:
• The backend employs the /add/{query} endpoint to verify the query's popularity against established criteria, ensuring only truly trending searches are added. This step is crucial to filter out fleeting or insignificant trends, maintaining the integrity of trending suggestions.

Database Incorporation:

• Verified popular queries are then added to the database, updating the data structures (such as a trie) that facilitate the generation of typeahead suggestions. This inclusion allows the suggestion mechanism to dynamically evolve in response to shifting trends and user preferences, enriching the pool of suggestions.

Suggestion Service Efficiency:

• Upon receiving a suggestion request, the Suggestion Service initially consults the Distributed Cache for pre-existing suggestions matching the input. A cache hit leads to an immediate return of suggestions, greatly enhancing the response speed. The process of retrieving and presenting suggestions remains consistent, relying on the Distributed Cache to offer prompt and relevant suggestions.

Trade offs/Tech choices

Redis for Caching:

• Pros: Redis is an excellent choice for caching as it provides low latency access to frequently accessed data, improving the overall performance of the real-time suggestion system.
• Cons: While Redis is fast and efficient for caching, it could introduce complexity to the system architecture. Additionally, Redis is an in-memory database, so there might be limitations on the amount of data that can be cached.
• Improvement: Consider implementing data eviction policies and strategies for handling cache misses to ensure optimal performance.

NoSQL Database:

• Pros: NoSQL databases offer scalability, flexibility in schema design, and fast write capabilities, which are beneficial for handling dynamic data in real-time suggestion systems.
• Cons: NoSQL databases might lack support for complex queries and transactions compared to traditional SQL databases. Also, ensuring data consistency in a distributed NoSQL environment can be challenging.
• Improvement: Utilize sharding and replication techniques to improve scalability and fault tolerance in the NoSQL database setup. Implement data partitioning strategies to distribute data efficiently across nodes.

Failure scenarios/bottlenecks

Addressing these challenges involves implementing strategies to maintain system performance and reliability:

Mitigating Database Overload

• Batch Processing: Accumulate updates and perform them in batches to reduce the frequency of write operations, minimizing the impact on the database.
• Read-Write Separation: Use separate databases for read and write operations to alleviate lock contention. This can be achieved through replication, where writes are handled by the primary database and reads are distributed among replicas.
• Asynchronous Processing: Offload intensive data processing tasks to asynchronous systems to prevent blocking critical operations, thereby reducing load on the primary database system.

Resolving Cache Invalidation Issues

• Dynamic Cache Invalidation: Implement a more sophisticated cache invalidation strategy that quickly identifies and updates only the affected portions of the cache, reducing the latency in reflecting trending queries.
• Debounce Cache Updates: Introduce a mechanism to debounce cache updates, allowing for the aggregation of changes before they are applied. This can help in managing rapidly changing trends with fewer cache invalidations and rebuilds.
• Cache Segmentation: Segment the cache based on query popularity or other metrics to isolate frequent updates from stable data, minimizing cache thrashing.

Overcoming Performance Bottlenecks

• Trie Optimization: For the trie data structure, consider implementing a more efficient variant or compression techniques to reduce its size and depth, enhancing performance for insertions and searches.
• Efficient Query Design: Optimize query algorithms to reduce complexity, particularly for trend analysis and suggestion retrieval. Utilize indexing and consider leveraging full-text search capabilities of modern databases.
• Scalable Architecture: Design the system with scalability in mind, using load balancers, horizontal scaling (adding more servers), and microservices architecture to distribute the load and reduce bottlenecks.

These strategies aim to ensure the system remains responsive, scalable, and capable of handling the dynamic nature of user queries and trending analysis without compromising on performance or user experience.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?