System requirements

Functional:

• System must expose APIs for validating the humanness of the user

Non-Functional:

• The system must scale 2 millions of users
• The system must accurately detect human users

Capacity estimation

• 2 millions users per day
• We assume 2 million requests per day which averages about 23 requests per second. Typical server (AWS T2) can easily handle 200 req / second so we can have 3 servers split across availability zones, to ensure resilience against 2 AZ outage.
• We can have 3 extra servers, in same AZs on standby for auto scaling
• We need to store about 3gb (10kb * 100000 = 1gb where each capta on everage requires 10kb memory and we have total of 3 replicated data)

API design

• get_human_test() returns a captcha test to render on the FrontEnd. The test is generated (i.e. there's a library that will generate using sourced data). The test would somehow get loaded on the user's FrontEnd Additionally it returns a session_id which will bind to a specific version of this test and track if it's been previously ran or not.
• is_human(session_id, test, test_results) returns True or False. The session_id would indicate the test being run

Database design

We won't try to design multiple DB. Rather, lets just focus on a single captcha design and we can assume other captcha's will have their own implementations

• Lets have 2 DB types, blob storage and metadata storage.
  • Blob contains the base image and we will use S3
  • Metadata contains data about the captchas, captcha strength, type, preferred function to apply on top of the corresponding captcha
• We will have a CDN which will store pre-generated mdoified captcha ( a captcha is modified so that bots can't use a dictionary attack. by applying a simple filter or transform, we can programmatically obtain the results based off of initial solution and captcha)
• All DBs except CDN are replicated in each AZ

High-level design

• We will have a dedicated caching service that pre-generates CAPTCHAs, using base CAPTCHA (e.g. images, etc...) and stores those in CDN. There will be several CAPTCHA systems/types and the service will cache several types and also, derive the captcha's too from base captcha, to ensure bots cannot dictionary attack the captcha.
• We will have a frontend captcha that uses a shared session cache across all of its stateless instances (it's actually stateful in that requests can use cookies
• The frontend routes requests to the backend captcha systems. When a new request is made, it can redirect the user to a captcha in CDN. The frontend will apply locks whenever accessing a CDN captcha, so that no two captchas are reused. We can use Redis for distributed lock and base it off the CAPTCHA name. The CAPTCHA may not be reused. Furthermore, the session will track success/failure attempts and reattempts are strictly prohibited. When a user finishes attempting the captcha, the user provided input to that test along with session are sent to the front end. The front end forwards the captcha id (i.e. obtained from session cookie), the user input and responds pass/fail. Front end service marks the test as attempted and returns status.
• To scale to millions of users the cache shared across the multiple frontend service will be partitioned using consistent hashing. Furthermore the frontend remains stateless and can be scaled up using dynamic scaling.
• A load balancer will serve as the entry point for the requests. It will rate limit based off of the number of available frontend pods. While the frontend pods will support dynamic scaling, the number of total requests it can manage will be based on healthy pods.
• All frontend pods and backend pods (each with a dedicated Captcha integration) will have 3 instances and auto-scaling enabled.
• We have a dedicated attack bot, which attempts to pass the captcha tests in automated way. This incurs development and processing costs and overhead, but ensures the quality of our system.
• Frontend requests are required to be done over TLS

Request flows

• Request flow is largely described in the high level design and wont be repeated here.

Detailed component design

• High level diagram goes over the component design too.

Trade offs/Tech choices

• We assume high load, which is why CDNs and session cookie caches are used.
• Though costly we ensure no reuse of Captchas. This requires setting up a per captcha redis lock, but increases confidence that the captcha are truly filtering bot users.
• We partition all backend services across 3 AZ hosts for ensuring high availability and reliability.
• All data at rest and in transport is encrypted, which slows performance but ensures our security standards are up to par.

Failure scenarios/bottlenecks

• If there is a spike of requests, our load balancers may refuse new requests until the backend pods have scaled up. Once they have scaled up, our system will still only support an upper bound. We should be able to scale this to very large workloads though so should not be a major concern. Our distributed cache too should scale to large loads.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?