Requirements

Functional Requirements:

• Record ad clicks from users across websites in real time, including client metadata (cookies, IP addresses, browser string), ad_ID, and timestamp.
• Aggregate click data by time window (second, minute, hour) to provide click counts per ad.
• Provide audience info (age, gender, geo location) and showing where users engage, like specific websites or platforms, breakdowns in reports for advertisers.

Non-Functional Requirements:

• Availability for view reports, 99.99%, 52 minutes downtime in a year.
• Consistency eventually for click counts
• Hot key: hot websites
• idempotency for ads clicks, e.g. # of clicks in a second from the same user counted as 1
• Low latency: Ensure aggregation completes within a few minutes.
• Scalability: able to serve 1B clicks per day, and grows 30% per year
• Fault tolerance: Recover from failures using snapshots and replaying events.
• Realtime dashboard: able to see clicks in real time

API Design

collect clicks from FE

POST /v1/track/clicks: request:{clickEvent}

view clicks reports

Report query

GET /v1/reports/ads/{adId}/clicks?start={}&end={}...etc.

GET /v1/reports/ads/{adId}/clicks?campaignId={}&breakdown=age,geo,gender&channel=website

Query Parameters:

- start: start timestamp (inclusive)

- end: end timestamp (exclusive)

- granularity: minute | hour | day

- metrics: clicks, impressions, ctr, conversions

- breakdown: age, geo, gender, device

- campaignId: optional filter

- channel: optional filter (e.g. website, app)

- limit / cursor: for pagination

These endpoints are used

above GET all need JWT as advertiser/admin login as authentication, and rule-based authorization. advertiser can access their own ads/campaigns/..

admin has broader access scope

High-Level Design

User Click

↓

Frontend SDK / Click Redirect

↓

API Gateway / LB

↓

Tracking Collector (thin, stateless)

↓

Kafka (raw click topic)

↓

Flink / Stream Aggregator

- dedup

- enrich

- window aggregation

- rollup by ad/campaign/device/geo/time

↓

Druid (near real-time OLAP)

↓

Report Backend

↓

Advertiser Dashboard / API

In parallel:

Kafka / Flink output

↓

Batch ETL (hourly / daily / reconciliation / backfill)

↓

Relational DB / Warehouse

↓

Report Backend

↓

Redis cache (optional)

On the write path, frontend click events are sent to a thin tracking collector behind the gateway.

The collector should stay stateless and only validate minimal fields, then append events asynchronously to Kafka.

A stream processing layer such as Flink consumes raw click events, performs deduplication and windowed pre-aggregation, and writes near real-time aggregates into Druid for low-latency OLAP queries.

For the read path, a report backend serves advertiser queries, handles authentication and authorization, and reads from Druid for near real-time reporting.

In parallel, the pipeline also feeds a batch layer that produces hourly or daily finalized reports and stores them in a relational database. Redis can be added as a cache for hot reporting queries.

Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.

deal with 10times QPS, scale up

failover