My Solution for Design a Language Translation Service with Score: 8/10

by john_chen

System requirements


Functional:

User interface for translating language

Able to translate inputted language into target language

Supports multiple languages

Able to detect input language


Non-Functional:

Able to serve thousands of request per second.

Able to support offline translation mode.


Capacity estimation


  • Concurrency: Let's assume the service needs to support 50,000 concurrent users at peak times.
  • Throughput: Each user submits an average of 150 words for translation, and the system can process 15 words per second per user.
  • Memory Usage: We estimate that each translation request requires 10MB of memory for processing.
  • Processing Time: The system can translate 1,000 words in 1 minute.


Let's calculate the necessary system capacity based on these parameters:


  • Concurrency:
  • 50,000 concurrent users
  • Throughput:
  • Words submitted per request = 50,000 users * 150 words / 15 seconds = 500,000 words per request
  • Memory Usage:
  • Memory required per request = 50,000 users * 10MB = 500,000 MB or 5 GB per request
  • Processing Time:
  • Words translated in 1 minute = 50,000 users * 1,000 words = 50,000,000 words (1 GB ~ 1 million words)


API Design


translate(words, targetLanguage)


The translate APIs will take in an array of words and target language and returns the translated array of words.



Database design

We will assume that we have an existing translation engine that would perform the translation for us. The main place where we would use a store would be to cache common translations. We can store frequently translated phrases or sentences in a data store, and improve response time by quickly retrieving commonly requested translations.



High-level design


Client

User interface for user to submit their input for translation. This could be web or mobile.


Load Balancer

The load balancer is responsible for balancing request between different instances of the backend service.


Backend service

This is where the backend service take the request and direct it to the appropriate service. This is essentially a BFF.


Translation Service

Translation service is where it looks up the cache for existing translation or direct the request to the translation engine.


Translation Engine

The translation engine is responsible for translation on the fly. This could be a third party service such as amazon translate or google translation service.


Redis Cache

The translation engine decides whether to store the response into the Redis cache. The translation service grabs data from the Redis cache is it's able to find it.





Request flows


  • User: Initiates the translation process by inputting text.
  • System: Receives the input text, and forwards the text along with language preferences to the TranslationAPI.
  • TranslationAPI: Processes the text using machine translation algorithms and returns the translated text to the System.
  • System: Displays the translated text to the User in the selected target language.



Detailed component design


The most interesting component is probably the translation engine and translation service.


If we are using third party solution such as amazon translation or google translation then the translation engine would be pretty trivial as we would implement integrate with these services. However if we are rolling our solution we would have incorporate advanced technologies and techniques such as machine learning to translate different languages.


Translation Engine


Workflow of the Translation Engine:

  1. Input Processing:
  2. The translation engine receives the input text or speech data in the source language that needs to be translated.
  3. It may involve preprocessing steps such as tokenization, normalization, and cleaning to prepare the input for translation.
  4. Language Detection:
  5. The engine might perform language detection to identify the source language if not provided.
  6. Language detection will help determine the appropriate translation model to use for converting the text to the target language.
  7. Translation Models:
  8. Utilizing machine translation algorithms like Neural Machine Translation (NMT) or Statistical Machine Translation (SMT), the engine converts the input text into the target language.
  9. These models learn the patterns and structures of languages to generate accurate translations.
  10. Context and Meaning:
  11. The translation engine takes into account the context, grammar, and semantics of the input text to produce meaningful and contextually appropriate translations.
  12. Quality Evaluation:
  13. The translated output may undergo quality assessment processes to evaluate the accuracy and fluency of the translation using metrics like BLEU score or human evaluation.
  14. Post-Processing:
  15. Additional post-processing steps like fluency adjustments, grammar corrections, or context-based optimization can be applied to refine the translated output.
  16. Output Generation:
  17. The final translated text or speech output is generated and presented to the user in the desired target language through the user interface or delivery mechanism.



  • Google Translate leverages NMT models, which are deep neural networks trained to directly map input text in one language to the desired output text in another language.
  • NMT models consider the context of the entire sentence rather than translating it word-by-word, leading to more accurate and fluent translations.


Offline translation


  • We can have an offline translation engine or incorporate pre-trained language models within the client-side application to perform translations locally without the need for a network connection.
  • Cache frequently used translations or store recently translated phrases locally on the user's device to allow for quick retrieval and offline access without requiring re-translation.


This also means that the client would first utilize offline translation whenever it can and only make request when offline translation isn't sufficient.


Translation Service


Circuit Breaker with Retry Pattern for Primary-Failover Translation Service:

  1. Primary Provider:
  2. Send translation requests primarily to the main third-party API provider (Primary Provider) for efficient translation processing.
  3. Failure Detection and Circuit Breaker:
  4. Monitor the responses from the Primary Provider for errors or timeouts. Utilize the Circuit Breaker Pattern to detect and handle failures from the Primary Provider.
  5. Failover Mechanism:
  6. Upon detecting a failure with the Primary Provider, trigger the Failover mechanism to switch translation requests to the secondary API provider (Fall-back Provider).
  7. Retries and Resilience:
  8. Implement the Retry Pattern for translation requests that fail with the Primary Provider before initiating the Failover to improve resilience.
  9. Apply exponential backoff strategies for retry attempts to prevent overwhelming the Primary Provider during transient failures.
  10. Fallback Translation Service:
  11. Configure the translation service to utilize the Fall-back Provider for translations when the Primary Provider is unavailable. Route requests to the Fall-back Provider until the Primary Provider recovers.
  12. Circuit Breaker Cooldown and Recovery:
  13. After a period, attempt to half-open the Circuit Breaker for the Primary Provider to re-establish communication. Gradually transition back to the Primary Provider if it becomes available.


Advantages of Circuit Breaker and Retry Pattern for Primary-Failover Setup:

  • Service Continuity: Ensure translation services remain functional even if the Primary Provider experiences disruptions.
  • Fault Isolation: Prevent cascading failures by routing translation requests to alternative providers during Primary Provider outages.
  • Resilience and Recovery: Enhance the service's ability to recover from failures and maintain reliable translation capabilities.