Design TikTok
0%
Editorial Solution
System requirements
Functional
Non-Functional
Why This Differs From Instagram
Scope Control for the Interview
What to Not Design For Day One
Read/Write Asymmetry
Two Distinct User Loops
Capacity estimation
Traffic
Daily Active Sessions
Storage
Bandwidth
Why This Matters
Compute Budget for Ranking
Why Not Store Everything in RAM
Working Set and Cache Efficiency
Cost Breakdown
API design
Upload
Feed
Engagement
Social graph
Search
Moderation
Auth
Rate Limits and Abuse Protection
Why These Choices
Pagination Semantics Matter
Why gRPC Internally But REST at the Edge
Versioning
Error Handling
Database design
Video metadata: sharded MySQL / Vitess
Social graph: sharded MySQL with denormalized follow tables
Engagement counters: Redis with async Cassandra flush
Feed candidate cache: Redis
Feature store: hybrid online/offline
Search: Elasticsearch
Analytics: ClickHouse or Druid
Notifications: Cassandra with TTL
Sessions and Auth: Redis
Object storage: S3, with Glacier tiering
Keeping the Hot Path Consistent Enough
Glacier Tiering in Practice
High-level design
The four planes
Why four planes instead of one service mesh diagram
Key shared infrastructure
Service-Level Ownership
Statelessness at the Edge
Why a Model Registry Is a First-Class System
Request flows
Upload flow
FYP read flow
Swipe prefetch
Engagement Write Flow
Follow Flow
Detailed component design
Candidate Generator (retrieval)
Ranker
Feature Store
Feature Categories
Transcoder
Content Moderation
Notification Service
Search Service
Analytics Service
Trade-offs and tech choices
Recommendation-first vs subscription-first feed
Pull-based vs push-based feed generation
HLS vs DASH vs MP4 progressive
Candidate cache invalidation strategy
Monolithic ranker vs multi-task vs cascade
Redis vs Memcached
MySQL (Vitess) vs NewSQL (Spanner, CockroachDB)
Following Feed Design
Bandwidth-Adaptive Streaming Ladder
Kafka vs Per-Event REST
Synchronous vs Asynchronous Moderation
CDN: Commercial vs Owned
Failure scenarios and bottlenecks
Viral video hot key
Ranker latency spike
Thundering Herd on Candidate Cache
Candidate pool poisoning
Moderation backlog
Database Shard Hotspot
Kafka Lag
Transcoding queue overflow
Region Outage
Feature Store Drift
Cold Start After a Long Absence
Future improvements
Multi-modal embeddings
On-device ranking
Live streaming integration
Ads Personalization
TikTok Shop and e-commerce
Regional models and regulations
Creator Monetization Platform
Trust and Safety Infrastructure
Edge Compute for Personalization
Table of Contents
TikTok looks like Instagram or YouTube on the surface, but it is a fundamentally different system. The home feed is not assembled from who you follow. It is assembled by a recommendation engine that scores every candidate video against you, individually, on every single request. Forget that distinction and you will design the wrong architecture.
Functional
- Upload short-form video: 15 to 60 second vertical clips, with background music, effects, filters, and captions baked in before upload.
- For You Page (FYP): a personalized, infinite-scroll feed of ranked videos. This is the default landing surface and drives roughly 90 percent of watch time.
- Following feed: a secondary tab showing reverse-chronological posts from followed accounts. This is the social-graph path and behaves more like classic Instagram.
- Swipe-to-next playback: vertical swipe loads the next video instantly. The system prefetches the next few candidates so the transition is seamless.
- Social graph: follow, unfollow, like, comment, share. These drive the feedback signals the ranker consumes, but they do not directly compose the home feed.
- Content moderation: every uploaded video must pass an automated moderation pipeline before it enters the candidate pool. High-risk videos go to human review before wide distribution.
Non-Functional
- Sub-second video startup: tap a video and the first frame plays in under 500ms. Anything longer and users swipe away. This drives HLS with 2-second segments, aggressive prefetch, and edge caching of the first chunk.
- Scale: 500M DAU, 100B video views per day, 100M uploads per day. Feed ranking must serve around 1.2M requests per second sustained, with peaks 3-5x higher.
- Feed ranking latency under 100ms: the ranker is the critical path on every feed request. Budget: candidate retrieval 50ms, feature lookup 20ms, model inference 30ms.
- Globally distributed CDN: video bytes must travel less than 50ms from the nearest edge. The origin is a fallback, not a primary serving path.
- Scalable ML inference: the ranker runs on every feed request. That is billions of DNN inferences per day. The inference tier has to scale horizontally like any other stateless service.
- Eventual consistency for engagement: like counts, view counts, and comments can lag a few seconds. The follow graph is strongly consistent because users immediately check whether the Follow button flipped.
The core insight that separates a TikTok design from an Instagram design: the home feed does not look at who you follow. It looks at a personalized candidate pool scored by an ML model. Lead with this in the interview, it frames every downstream decision.
Why This Differs From Instagram
Instagram fans out posts from accounts you follow to your precomputed timeline. TikTok does not. Instead, a candidate generator produces 500 to 1000 candidate videos for every feed request, a ranker scores them against your features, and the top-N scored videos become your feed. The social graph is a signal, not the skeleton. This pull-based, per-request ranking is why TikTok can hook new users with zero follows and why competitors struggle to replicate the experience by bolting ranking onto a subscription-first architecture.
Mid-level: identify the core services (Upload, Feed, Ranker, Social, Moderation) and explain why the FYP uses pull-based per-request ranking instead of fan-out on write.
Senior: design the upload-to-playable pipeline (presigned S3, Kafka-driven transcoder, HLS ladder), the candidate cache plus ranker serving path under 100ms, and the CDN strategy at 500 PB per day of egress.
Staff: go deep on the two-stage recommendation (two-tower retrieval with ANN plus multi-task DNN cascade), feature store with training-serving skew and point-in-time joins, multi-modal moderation with throttle-don't-block on medium confidence, and the economics that flip best-technology choices at TikTok scale (Vitess over NewSQL, owned CDN over commercial).
Scope Control for the Interview
A real 45-to-60 minute interview cannot cover live streaming, TikTok Shop, ads personalization, creator payouts, and regional content compliance all in depth. Pick the core: the For You Page ranking pipeline (retrieval, features, ranking, diversity), the upload-to-playable path (presigned upload, transcoding, moderation, CDN origin), and the feedback loop that closes the ML system. Mention the rest as future improvements so the interviewer knows you are aware of them but do not dilute the deep discussion by spreading thin.
What to Not Design For Day One
It is also useful to name explicit non-goals so the interviewer knows the scope is deliberate, not accidental. Day-one TikTok does not need: ads personalization (a separate and substantial subsystem), creator monetization flows (payments infrastructure), live streaming (a different media pipeline), regional ranker variants (comes later once the global ranker proves out), device-specific on-device ranking (an optimization on top of the server ranker), or ecommerce integrations. Explicitly naming what is out of scope signals that you understand the product surface and are prioritizing the core technical challenge.
Read/Write Asymmetry
The read-to-write ratio on views versus uploads is 1000:1. On any given day, for every new video created there will be roughly a thousand views against the catalog. This single ratio dictates that the serving path must be massively more parallel, massively more cached, and massively more CDN-offloaded than the ingest path. Every later design decision flows from it.
Two Distinct User Loops
Users interact with TikTok in two very different modes, and the system needs to serve both cleanly.
The viewer loop is by far the dominant mode, a user opens the app, swipes through the FYP, occasionally interacts (like, comment, share, follow). This loop drives the vast majority of traffic and is the loop the ranker is optimized for. A single session might consume 50-200 videos. The system is designed to make this loop near-instantaneous on every swipe.
The creator loop is smaller in volume but critical in value. A creator records, edits, and uploads a video, watches engagement metrics, iterates on content. This loop touches the Upload Service, the Moderation pipeline, the Analytics Service, and the Notification Service. It is not latency-critical at the seconds scale (nobody expects analytics to update in under a second) but it is quality-critical (creators need accurate numbers or they lose trust).
Keeping these loops cleanly separated in the architecture makes it easier to reason about failure modes. A failure that degrades the viewer loop affects billions of sessions. A failure that degrades the creator loop affects thousands of power users, fewer but much higher per-user value.
These numbers are not window dressing. They determine which subsystems get built in-house, which go to the CDN, and which parts of the design would collapse under the real load.
Traffic
- DAU: 500M active users per day.
- Uploads: 100M new videos per day. 100M / 86400 gives roughly 1,160 uploads per second sustained, with event-driven peaks of 5-10K per second.
- Views: 100B views per day. 100B / 86400 is roughly 1.16M views per second sustained. Peak (evenings, viral moments) is 3-5x higher, so plan for 3-5M views per second.
- Engagement: at a 5 percent like rate, 5B likes per day, or roughly 58K likes per second. Comments at 1 percent is 1B per day or 11K per second.
- Feed requests: approximately equal to views at steady state, since each swipe consumes one pre-ranked item from a batch. At a fetch-batch of 10, that is roughly 120K feed-service requests per second.
The read-to-write ratio is 1000:1 on views versus uploads. This is identical to YouTube but at 20x the volume.
Daily Active Sessions
500M DAU with an average of 8-10 sessions per day works out to roughly 4B-5B sessions per day. Average session length is about 90 minutes across all active users, though this is heavily skewed, many users check in for a few minutes, heavy users spend hours. Session length directly drives the ranker's training signal quality, longer sessions mean more engagement events per user per day, which means a richer training distribution. Keeping average session length high is not just a product metric, it is an ML infrastructure requirement.
Storage
- Raw uploads: 100M per day times 5 MB average = 500 TB per day ingress into S3. Over three years with multi-rendition storage at 2x, you are looking at around 1 EB of hot video storage.
- Transcoded renditions: each video is encoded at 360p, 540p, 720p, 1080p in both H.264 and AV1. That is roughly 8 variants at a combined 2-3x the original size, so plan for 1-1.5 PB per day of transcoded output.
- Metadata: 100M new video rows per day at 2 KB each gives 200 GB per day of metadata, modest in absolute terms but a firehose into the video metadata DB.
- Engagement log: 100B view events plus 5B likes plus 1B comments equals roughly 106B events per day. At 200 bytes per event, that is 21 TB per day into the event bus and analytics warehouse.
- Feature store: hundreds of features per user plus hundreds per video, kept in Redis for online serving. At 500M users times 2 KB per user feature blob, that is 1 TB of online user features.
- Moderation review queue: at any time there are typically hundreds of thousands of videos pending in the human-review queue (medium-confidence classifier outputs from the last 24 hours). This queue is stored in a priority queue service (Redis sorted set or a dedicated queue service) keyed by predicted view velocity so high-risk content is reviewed first.
- Ranker model artifacts: each deployed ranker is a few hundred MB. With several production versions plus canaries plus older versions retained for rollback, the model registry holds a few tens of GB of model weights at any time. Cheap relative to everything else, but versioned and audited carefully.
Bandwidth
- Egress: 100B views times 5 MB average = 500 PB per day of egress. That is roughly 50 Tbps sustained, 200-250 Tbps at peak. This alone is why TikTok owns PoPs inside major ISPs, it would be prohibitively expensive as pure commercial CDN egress.
- Ingress: 500 TB per day divided by 86400 is under 50 Gbps sustained. Compared to egress, ingress is a rounding error.
- Internal RPC: each feed request fans out to candidate retrieval, feature store lookup, ranker inference, video metadata hydration. Roughly 10 internal RPCs per feed request, so 1.2M external QPS becomes 12M internal QPS inside the cluster.
Why This Matters
The egress number is the single most expensive line on the infrastructure bill. Every millisecond of extra startup latency increases abandonment, which is good for the bill but bad for the product. Every inefficient codec choice costs petabytes. A 10 percent reduction in bitrate via AV1 saves roughly 50 PB per day of egress, which at even 1 cent per GB is 500K dollars per day. This is why codec and CDN strategy are not afterthoughts, they are the business.
Compute Budget for Ranking
Size the ranker compute. At 1.2M feed QPS and roughly 500 candidates per request, the ranker scores 600M candidates per second. A single GPU (say an A100) can score a few hundred thousand candidates per second on a medium-sized DNN with dynamic batching. So the ranker fleet needs thousands of GPUs at minimum. This is a material capex number, which is exactly why the cascade (cheap gate model then full DNN) is not a premature optimization, it is a capacity planning requirement.
Why Not Store Everything in RAM
A naive "just put all metadata in Redis" would require terabytes of in-memory storage for the full catalog of billions of videos. At typical cloud RAM prices that would cost tens of millions of dollars per month. The actual design only keeps the hot working set in RAM (candidate cache, online features, counters) and serves the long tail from disk-backed stores, which is orders of magnitude cheaper.
Working Set and Cache Efficiency
Of the billions of videos in the catalog, only a small fraction is actively viewed on any given day. Pareto-style distributions dominate: the top 1 percent of videos by popularity typically account for 60-80 percent of views. This distribution is what makes caching effective, a few hundred GB of the most popular video metadata covers the overwhelming majority of requests. Estimating the working set size tightly and sizing the caches accordingly is how the system achieves 98+ percent hit rates on cost-effective hardware.
Cost Breakdown
At this scale the cost mix is dominated by bandwidth (CDN egress), followed distantly by compute (GPU inference and transcoding), then storage (S3 plus Glacier). A representative breakdown in percentages might be roughly: bandwidth 50-60 percent, compute 20-25 percent, storage 10-15 percent, everything else 5-10 percent. This mix is useful to keep in mind when proposing optimizations, a new feature that adds compute is not as expensive as one that increases egress, so the cost question is "what is the bandwidth impact" before anything else.
In an interview, always sanity-check optimizations against the dominant cost line. Saving 20 percent of GPU is worth roughly 5 percent of the total bill. Saving 20 percent of egress is worth 10-12 percent of the total bill, twice the impact. This is why the AV1 migration, the 2-second HLS segments, and the owned-CDN decision all show up first in real TikTok engineering priorities, not because they are technically interesting but because bandwidth is where the money is.
REST over HTTPS for client APIs. gRPC for internal service-to-service calls. Every endpoint requires OAuth2 bearer tokens. The API gateway handles auth, rate limiting, abuse detection, and geo routing.
Upload
POST /api/v1/upload/presign: client requests a presigned S3 URL. Server returns anuploadUrl,videoId, andexpiresIn(typically 30 minutes). This lets the client push multi-MB video bytes directly to S3 without touching app servers.POST /api/v1/videos: commits the metadata after the S3 upload finishes. Body includesvideoId,caption,soundId,effects[],hashtags[], and anidempotencyKeyso a retry does not create a duplicate record. Returns a video inprocessingstate.GET /api/v1/videos/{videoId}: fetches metadata and current processing state (uploading,processing,moderating,ready,blocked).
Feed
GET /api/v1/feed/foryou?cursor={opaque}&batch=10: returns the next batch of personalized video descriptors. The cursor is an opaque token that encodes session state on the server (candidate pool version, diversity state, ranker experiment). Not a simple offset.GET /api/v1/feed/following?cursor={opaque}&batch=10: returns reverse-chronological posts from followed accounts. Simpler path, more like classic social feeds.POST /api/v1/feed/foryou/feedback: client reports implicit signals back to the server: watch-time, skip, replay, share. These drive the ranker's next batch.
Engagement
POST /api/v1/videos/{videoId}/like: idempotent. Repeated calls do not toggle, they simply confirm the liked state.DELETE /api/v1/videos/{videoId}/like: explicit unlike.POST /api/v1/videos/{videoId}/comments: body{text, parentCommentId?}. Returns the created comment.GET /api/v1/videos/{videoId}/comments?cursor={opaque}: paginated comments, ranked by engagement.POST /api/v1/videos/{videoId}/share: records a share event for the ranker.
Social graph
POST /api/v1/users/{userId}/follow: creates an edge. Strong consistency: the button state must flip before the response returns.DELETE /api/v1/users/{userId}/follow: removes the edge.GET /api/v1/users/{userId}: profile data, follower count, video grid.
Search
GET /api/v1/search?q={query}&type={videos|users|sounds|hashtags}: backed by Elasticsearch. Sub-200ms latency target. Returns ranked results within each entity type.
Moderation
POST /api/v1/moderation/reports: users report videos. Body{videoId, reason}.POST /api/v1/admin/moderation/decisions: internal-only endpoint for human moderators to approve, block, or demote a video. Guarded by role-based auth.
Auth
POST /api/v1/auth/signup: create an account (email, phone, or OAuth via Google/Apple/Facebook). Returns access and refresh tokens.POST /api/v1/auth/login: authenticate and return tokens.POST /api/v1/auth/refresh: exchange a refresh token for a new access token. Access tokens are short-lived (15 minutes). Refresh tokens are long-lived (30 days) and rotated on every refresh to defeat stolen-token reuse.POST /api/v1/auth/logout: revoke the refresh token and invalidate server-side session state.
Rate limits are aggressive on the auth endpoints (5 requests per minute per IP for login, 1 per hour for signup per device fingerprint) because credential stuffing and account creation abuse are constant threats at TikTok scale.
Rate Limits and Abuse Protection
Every endpoint has a per-user rate limit and a per-IP rate limit. The API Gateway tracks limits in a distributed token-bucket implementation (Redis-backed, with local caching on gateway instances to avoid hammering Redis on every request). Abusive traffic patterns, such as a single IP hitting POST /like at 100 requests per second, trigger a soft block that returns 429 with a retry-after header. Persistent abuse gets escalated to a longer block and flagged for analysis. Abuse detection also feeds the trust-and-safety pipeline, where patterns of coordinated fake-engagement trigger an investigation.
Why These Choices
The upload flow uses presigned URLs for the same reason as Instagram or YouTube, app servers should never proxy multi-MB binary transfers. The feed endpoints use opaque cursors instead of numeric offsets because the candidate pool and ranker version can change mid-session, an opaque cursor lets the server encode all that state without exposing it to the client. And every write endpoint accepts an idempotency key because mobile networks lose connections constantly and clients retry, without idempotency you would see duplicate posts and double-counted likes on a daily basis at 500M DAU.
Pagination Semantics Matter
Interviewers sometimes challenge the opaque cursor with "just use a timestamp." A timestamp does not work for the FYP because the feed is not time-ordered, it is ranker-ordered. A timestamp also cannot encode which video IDs the user has already been shown in the current session (needed for dedup) or which experiment arm they are in (needed to keep the ranker consistent for the session). Opaque cursors are the only pagination primitive that cleanly supports all three.
Why gRPC Internally But REST at the Edge
REST over JSON at the edge gives the mobile and web clients a simple, debuggable protocol that is firewall-friendly and cacheable. Internally, gRPC over HTTP/2 with Protocol Buffers cuts per-request payload by around half and serialization overhead by even more. At 12M internal QPS the CPU savings from gRPC dwarf the development effort. The split is also a clean ownership boundary: the API Gateway is the protocol translation layer between the two worlds.
Versioning
Every client-facing endpoint is versioned via the /api/v1/ prefix. This is not just tradition, it is how you roll out breaking changes without forcing every mobile app on every version to update simultaneously. Old clients continue talking to /api/v1/, new clients talk to /api/v2/, and the Feed Service maintains both contracts until old-version traffic drops below a threshold. At TikTok's scale, even a version deprecation takes months because the long tail of outdated clients is substantial.
Internal gRPC services use Protocol Buffer schema evolution rules: add new fields as optional, never reuse field numbers, never remove a field without a cross-team migration. This lets internal services evolve independently without lockstep deploys.
Error Handling
Every client response follows a consistent error schema: HTTP status code for the class of error (400 for client errors, 500 for server errors), a stable machine-readable error code in the body, and a human-readable message for logging. Client retry logic is based on the error code, 5xx means retry with backoff, 4xx means do not retry. This consistency makes it possible for a mobile client to have a unified error handling path across dozens of endpoints.
When interviewers push on pagination for the FYP, explain that offset pagination is actively harmful here, the ranker re-scores on every request, so the same item can appear at position 3 and then position 17 across two consecutive fetches. Only an opaque cursor that encodes the server-side diversity and dedup state keeps the feed coherent.
TikTok uses aggressive polyglot persistence. Each data class has wildly different access patterns, and forcing them into one store would either blow up cost or collapse performance. The rule: pick the store that makes the dominant access pattern cheap, then accept the work of keeping the rest consistent.
Video metadata: sharded MySQL / Vitess
- Tables:
videos(video_id PK, owner_id, caption, sound_id, duration, status, created_at),video_renditions(video_id, resolution, codec, url, bitrate). - Sharded by
video_id(range-based Vitess shards). This spreads write load across shards evenly since video_id is roughly time-ordered with randomization. - Why SQL: video records have referential integrity with renditions. Joins are cheap within a shard. Point lookups by video_id are the dominant read, and SQL primary key lookups serve those in under 2ms.
Social graph: sharded MySQL with denormalized follow tables
- Tables:
follows(follower_id, followee_id, created_at),followers_by_user(user_id, follower_id, created_at),following_by_user(user_id, followee_id, created_at). - Sharded by
user_id. Two denormalized tables (followers and following) so both directions of the query are a single-shard lookup. - Celebrities with 50M+ followers get bucket partitioning:
followers_by_user(user_id, bucket, follower_id), where bucket is a hash slice. Without bucketing, a single celebrity's partition would become a terabyte-sized hotspot. - Why SQL, not a graph DB: TikTok's graph queries are one-hop. Who follows me, who do I follow. Graph databases shine for multi-hop traversal (friend-of-friend, shortest path), which is not the dominant pattern here.
Engagement counters: Redis with async Cassandra flush
- Like counts, view counts, share counts, comment counts all live in Redis keyed by
video:{id}:likes,video:{id}:views, etc. - Redis
INCRhandles the hot-write path at 58K+ likes per second and 1.2M+ views per second without lock contention. - A Kafka-driven flusher writes counter snapshots to Cassandra every few seconds for durability and cold-read fallback.
- The raw engagement event log (every individual like, view, comment) is written to Kafka and consumed into Cassandra for analytics and the feature store.
Feed candidate cache: Redis
- Key pattern:
candidates:{user_id}-> a serialized list of 500-1000 candidatevideo_idplus precomputed embedding-retrieval metadata. - TTL of a few minutes. The candidate generator refreshes it on a cadence or when a strong new signal arrives (a major follow, a long watch).
- Without this cache, every feed request would do expensive ANN retrieval from scratch, 1.2M retrievals per second at ANN cost would need roughly 10x the fleet.
Feature store: hybrid online/offline
- Online (Redis): per-user features (recent watch history embeddings, demographic buckets, engagement stats) and per-video features (creator features, content tags, engagement velocity). Written on every event, read on every feed request.
- Offline (BigQuery or Hive on Parquet): historical features computed via batch Spark jobs, used for training new ranker versions. Point-in-time correctness is enforced so training does not leak future features into historical rows.
Search: Elasticsearch
- Separate indexes for users, videos (title, caption, captions-to-text from ASR), sounds, hashtags.
- Kept isolated from the primary write path, search lag of a few seconds is fine.
- Index updates flow from Kafka consumers that read the new-video event stream and the metadata-update event stream. A separate consumer handles deletion and moderation blocks, removing videos from the index when they are blocked.
- Full-text ranking combines BM25 lexical relevance with engagement signals (view count, recency, creator reputation). The ranker model for search is a smaller, simpler model than the FYP ranker because search traffic is far lower and the relevance task is more constrained.
Analytics: ClickHouse or Druid
- Aggregates watch-time, retention curves, creator dashboards.
- Columnar storage handles the kind of wide analytical queries (group by country, creator, sound over 30 days) that would crush MySQL.
Notifications: Cassandra with TTL
The notification feed per user is stored in Cassandra with user_id as partition key, ordered by timestamp. Each row has a TTL so old notifications auto-expire. The table schema is essentially: notifications(user_id, ts, kind, actor_id, target_id, aggregated_count) where aggregated_count supports the batched "X and N others" pattern. Redis caches the unread count per user for instant UI badge updates. On read, the service merges server-side notification records with client-side read state to present a coherent feed.
Sessions and Auth: Redis
Active session tokens live in Redis as a key-value map from access token (or its hash) to a small session record (user_id, issued_at, permissions). This gives the API Gateway a sub-millisecond auth lookup on every request. Revocation is instant, deleting the key invalidates the session immediately. Refresh tokens are stored in a durable store (MySQL) with hashed values, because they have long lifespans and need audit history for security incident response.
Object storage: S3, with Glacier tiering
- Raw uploads and transcoded renditions both live in S3 initially.
- After 30 days with low view velocity, a lifecycle policy migrates cold videos to Glacier. TikTok videos have extremely long-tail access, most of a video's lifetime views happen in the first 72 hours, so cold tiering saves substantial cost.
Polyglot persistence is not a fashion choice, it is a capacity argument. No single database handles this workload, Redis is picked for atomic counters at 1.2M writes per second, MySQL/Vitess for transactional metadata with one-hop joins, Cassandra for append-only event streams that dwarf any relational system, ClickHouse for columnar analytics, S3 for unstructured media, Elasticsearch for inverted-index search. Each store is chosen because its access pattern matches the data. An interview answer that puts everything in one database fails on the capacity math before anyone looks at consistency.
Do not store the engagement event log in MySQL. At 106B events per day, no single SQL deployment can ingest this. Kafka plus Cassandra (or the analytics warehouse) is the default, and every ML signal the ranker uses is built by consuming from the event log, not from transactional tables.
Keeping the Hot Path Consistent Enough
The video metadata database is the source of truth for caption, creator, and moderation status. It is also one of the most-read tables in the system. To keep reads fast without hammering the primary shards, every query goes through a read-through Redis cache with a 5 to 10 minute TTL plus explicit invalidation on any write. Since video metadata is effectively immutable after publish (with rare edits for caption or takedown), stale reads from cache are safe and the cache hit rate is extremely high, regularly above 98 percent.
Glacier Tiering in Practice
A TikTok video has a steep engagement decay. Roughly 80 percent of a typical video's lifetime views happen in the first 72 hours. After 30 days, most videos are watched at single-digit views per day. Moving renditions to Glacier after 30 days of low engagement velocity saves something like 80 percent of storage cost for the long tail, at the cost of a 3-to-5-hour retrieval delay if someone does want to watch the cold video. A simple policy is: if a cold video is requested, fire an asynchronous rehydration job, serve a cached thumbnail plus "this video is being prepared" UI state, and push-notify the requester when it is ready. In practice this edge case is vanishingly rare.
TikTok is best understood as four parallel planes that share storage and a message bus, not as a classic monolith-plus-services diagram. Mixing the planes in your head is the fastest way to draw a confused architecture.
The four planes
1. Ingest plane (write path)
Client app -> API Gateway -> Upload Service -> S3 (raw) -> S3 event -> Kafka -> Transcoder pool -> S3 (renditions) -> CDN origin.
In parallel: Upload Service writes metadata to the Videos DB (sharded MySQL) in processing state. A Moderation pipeline consumes from Kafka, runs classifiers, and updates the video's moderation status. Only videos with status=ready become visible to the Feed Service.
2. Serve plane (read path)
Client app -> API Gateway -> Feed Service -> Candidate Generator -> Ranker -> Feature Store -> Video Metadata -> HLS manifests on CDN.
Engagement endpoints (like, comment, share, follow) hit their own services but share the same API Gateway and auth path.
3. ML plane (inference + training)
Online inference: Candidate Generator (ANN over video embeddings in HNSW or ScaNN) and Ranker (multi-task DNN) are deployed as stateless inference services behind a serving framework like TorchServe or TensorFlow Serving, with model versioning, canary rollout, and shadow traffic.
Offline training: a Spark-driven pipeline reads the event log from Cassandra and the offline feature store, constructs training examples, trains new model versions on GPU clusters, and registers them in the model registry. A separate evaluation pipeline gates promotion from staging to production on offline metrics plus shadow A/B.
4. Feedback plane (closed loop)
Every client event (watch-time tick, skip, like, comment, share, report) flows into Kafka -> stream processors (Flink) -> online feature store update and the event log. This is the tight loop that lets the ranker react to user behavior within seconds.
Why four planes instead of one service mesh diagram
The planes have fundamentally different SLAs. Ingest can tolerate 30-second latency because the user expects processing. Serve must respond in 100ms. ML inference must respond in 30-50ms per model. Feedback must propagate to online features in under a second. Drawing them as one big service mesh hides these constraints. Separating them makes the ownership and scaling responsibilities obvious, and it also clarifies which failures are recoverable (feedback lag) versus user-visible (serve latency).
Key shared infrastructure
- API Gateway: stateless layer for auth, rate limiting, geo routing, header-based experiment assignment. Global load balancer (anycast or DNS-based) routes each client to the nearest region.
- Kafka: the backbone of asynchronous processing. Separate topics for upload events, engagement events, moderation events, and analytics. MirrorMaker replicates across regions.
- Service mesh (Envoy + Istio or similar): handles mTLS, retry, circuit breaking, load balancing between services inside a region.
- Model registry + feature registry: the ML plane's governance layer. Every deployed model and feature is versioned and traceable.
- Observability stack: distributed tracing (Jaeger or equivalent), metrics (Prometheus), logs (Elasticsearch or a managed log service). At this scale, tracing is what lets engineers understand a slow feed request that touches 10 services.
- Deployment pipeline: GitOps-style with progressive rollout (canary, 1 percent, 5 percent, 25 percent, 100 percent) gated on health metrics at each stage. A bad deploy rolls back automatically in minutes.
In the interview, draw the four planes as four horizontal tracks that share storage on the left and the message bus in the middle. This layout makes it obvious why the ranker has a tighter latency budget than the transcoder, they are on different planes with different SLAs.
Service-Level Ownership
Each service in the diagram has a single-owner team. The Feed Service owns the FYP read path and its fallback strategy. The Candidate Generator team owns retrieval quality and ANN index health. The Ranker team owns model quality and inference performance. The Feature Store team owns training-serving consistency. The Transcoder team owns the upload-to-playable SLA. The Moderation team owns classifier quality plus human-review throughput. Clear ownership boundaries matter because the ML plane and the serve plane share critical interfaces (the feature store, the model registry) that break subtly when no single team is responsible for the contract.
Statelessness at the Edge
Every service in the serve and ML planes is stateless. Session state lives in the opaque feed cursor (client-side) or in Redis (server-side). This lets the service mesh route any request to any instance, which is how autoscaling actually works at scale. Stateful services are dangerous at this scale, they force session stickiness, complicate blue-green deploys, and add an entire class of failure modes around session draining. Keep state in the data plane, not the service plane.
Why a Model Registry Is a First-Class System
At TikTok's scale there are dozens of models in production simultaneously: retrieval tower, ranker, moderation classifiers, feature pipelines, embedding encoders. Each has versions, canary rollout stages, and performance metrics. A model registry is not just a filesystem for model weights, it is the governance layer: which model is serving which percentage of traffic, what offline metrics it was gated on, which experiment arm it belongs to, and how to roll it back. Without this, a bad model ships with no audit trail and no safe rollback path.
Three flows carry almost the entire system: upload, FYP read, and swipe prefetch. Get these right and the rest of the API surface is mostly glue.
Upload flow
- Client calls
POST /api/v1/upload/presign. Upload Service generates a presigned S3 URL and avideoId, returning both within 50ms. The server has done no heavy work yet. - Client uploads the encoded video bytes directly to S3 via the presigned URL. A 5MB file over LTE takes 5-10 seconds, and none of those bytes ever touch the API server.
- S3 emits an
ObjectCreatedevent to Kafka on theuploadstopic. - A pool of Transcoder workers (GPU-accelerated, running FFmpeg) consumes the event. Each worker produces HLS renditions at 360p, 540p, 720p, and 1080p in H.264 (primary) and AV1 (progressive migration). Segments are 2 seconds, which is the sweet spot for fast startup without excessive request overhead.
- Transcoder writes renditions back to S3 in a
renditions/{videoId}/prefix and publishes atranscodedevent. - A CDN origin warmer pre-pushes the first 1-2 segments and the HLS manifest to edge PoPs. This is crucial: on the first playback, the edge already has the bytes.
- In parallel, the Moderation pipeline consumes the
transcodedevent, runs classifier ensembles (nudity, violence, copyright), and either marks the videoreadyor sends it to a human review queue. - Once moderation passes, the Videos DB status flips to
ready, and the video enters the candidate pool index (ANN index in the retrieval service). - Client receives a push notification: "your video is live."
End-to-end, a typical video is live within 30-60 seconds of upload completing, assuming auto-moderation passes.
Note a subtle thing about step 4: moderation is not blocked on transcoding in all paths. For simple moderation signals (known-bad audio hashes, known-bad visual fingerprints) the raw upload is enough, so those checks can start earlier. More sophisticated classifiers (scene-level violence detection, frame-level nudity detection) do need the transcoded HLS segments because they are designed around standardized input. Running both in parallel minimizes the end-to-end moderation latency for the common case.
Also note step 5: the CDN origin warmer is an optimization, not a correctness requirement. If it is skipped, the first viewer's request for a new video still works, it just hits origin instead of edge, which adds a few hundred milliseconds of latency. For most videos this is fine. For videos predicted to go viral (by the creator's historical viewership or by early signals) the warmer is worth the cost.
FYP read flow
- Client calls
GET /api/v1/feed/foryou?cursor={opaque}&batch=10. - API Gateway authenticates, routes to the regional Feed Service.
- Feed Service decodes the cursor to recover session state (already-seen video IDs, diversity bucket counters, experiment arm).
- Feed Service checks the Candidate Cache in Redis (
candidates:{user_id}). On cache hit, it has 500-1000 pre-retrieved candidate video_ids. On miss, it synchronously calls the Candidate Generator. - Candidate Generator runs ANN retrieval (HNSW or ScaNN) over video embeddings using the user's embedding as the query. It returns roughly 500 candidates in under 50ms. It blends several retrieval streams: content-based (nearest neighbors of recently watched videos), trending (high-velocity videos in the user's geo/language), social (videos from followed creators), and sound-based (videos using sounds the user engaged with).
- Feed Service filters the candidate list against the session's already-seen set and any hard blocks (blocked creators, region restrictions).
- Feed Service calls the Feature Store in Redis to fetch user features and per-candidate video features in a batched read (one pipelined command, under 10ms).
- Feed Service calls the Ranker with the enriched candidate list. The Ranker is a multi-task DNN that outputs per-candidate probabilities for like, comment, share, watch-time, and skip, then combines them into a calibrated score.
- Feed Service applies a diversity filter: limit per-creator video density in the batch, interleave content categories, avoid repeating sounds. This step prevents the ranker from collapsing onto a single obsession.
- Feed Service hydrates the top-10 with HLS manifest URLs, creator info, caption, and engagement counts (from Redis counter reads), and returns the batch.
- Feed Service writes an updated cursor encoding the new session state and returns it in the response.
Total budget: under 150ms including network to client. Typical p50 is 60-80ms.
Worth calling out explicitly: step 4 (candidate retrieval) is where most of the budget can blow up if cold. A cache miss on the candidate cache forces a synchronous ANN retrieval, which can take 30-50ms. Combined with the rest of the budget, a cache miss is still under 150ms, but it is the edge case to watch. The steady-state flow with a warm cache finishes the entire pipeline in 60-80ms because the expensive retrieval is amortized across multiple requests.
Step 9 (diversity filter) deserves special attention because it is the least glamorous but most important quality step. Without it, a ranker that is convinced a user loves soccer will show them eight soccer videos in a row, and the user will churn. The diversity filter enforces category caps (no more than 2 of any one category per batch of 10), creator caps (no more than 2 videos from one creator per batch), sound caps (same), and in some cases hashtag caps. This is typically a constraint-satisfaction step with a tight time budget, measured in single milliseconds.
Swipe prefetch
The killer feature is instantaneous swipe-to-next. This is achieved entirely on the client plus CDN edge caching.
- Client receives batch of 10 feed items. It starts playback on item 0 immediately.
- Client also kicks off a prefetch of the first segment (2 seconds of video) for items 1, 2, and 3. On wifi, prefetch extends to items 4 and 5. On cellular, prefetch reduces to just item 1.
- Each prefetch request hits the nearest CDN edge. For popular videos, the segment is cached locally and returns in 20-50ms. For long-tail videos, the edge pulls from a regional parent cache and then from origin, taking 100-300ms on first-request, after which it is cached.
- When the user swipes, item 1 plays from the prefetched segment in under 100ms. While item 1 plays, item 4 gets prefetched.
- If the user swipes rapidly past items without watching, the client cancels in-flight prefetches to avoid wasting bandwidth.
- When the batch is half-consumed (5 items watched), the client fires the next
GET /feed/foryoucall so the next batch arrives before the current batch runs out.
The prefetch count is a bandwidth-personalization knob. Prefetching too aggressively on cellular wastes user data and tanks goodwill. Prefetching too conservatively on wifi creates visible loading spinners and hurts session length. Most production systems adapt prefetch depth to the device's measured throughput, not a hard-coded constant.
Engagement Write Flow
A user taps Like on a video. The client calls POST /api/v1/videos/{videoId}/like. The Like Service does three things in parallel. First, it issues a Redis INCR on the (sub-)counter keys for video:{id}:likes for instant UI confirmation. Second, it writes a durable event (user_id, video_id, kind=like, ts) to Kafka on the engagement topic. Third, it returns success to the client, typically in under 10ms. Downstream consumers of the Kafka event include: the video_likes table writer (for "who liked this" queries), the online feature store updater (the user just liked something, their affinity features change), the notification dispatcher (the creator gets a notification), the analytics pipeline (feeding trending detection), and the training data logger (this is a positive engagement signal for the model). All of these run asynchronously and independently.
Follow Flow
A user follows a creator. POST /api/v1/users/{userId}/follow hits the User Service. The service writes two rows atomically to the user's shard: one into followers_by_user (for the followee) and one into following_by_user (for the follower). It publishes a follow event to Kafka. It invalidates the follower's following:{user_id} cache in Redis. Consumers: the feature store (follow count is a feature), the candidate generator (the following-feed candidate stream for this user just changed), the notification dispatcher (the creator gets notified), and analytics. The response returns after the SQL write is committed, because the UI must reflect the state change before the user's next tap.
Five components carry the system's complexity: the Candidate Generator, the Ranker, the Feature Store, the Transcoder, and the Content Moderation pipeline. Each deserves its own diagram in a real interview.
Candidate Generator (retrieval)
The retrieval problem: given a user embedding, return 500-1000 candidate videos from a catalog of billions, in under 50ms. Exact nearest-neighbor search at this scale is impossible. The solution is Approximate Nearest Neighbor (ANN) indexes.
- Algorithm: HNSW (Hierarchical Navigable Small World) or ScaNN. Both deliver sub-10ms retrieval over hundreds of millions of vectors.
- Two-tower model: a user tower and a video tower are trained to produce embeddings such that dot-product approximates engagement probability. The video tower is computed offline when a video is uploaded and inserted into the index. The user tower is computed online from the user's current features.
- Blended retrieval streams: the full candidate set is not from one ANN call but a blend: 200 candidates from content similarity, 150 from trending in the user's geo/language, 100 from followed creators, 50 from sound affinity, 100 from exploration (recent videos not yet evaluated).
- Freshness: videos uploaded in the last 24 hours get a recency boost. This ensures new content gets shown to enough users to collect enough engagement signal for the main ranker.
- Index sharding: the ANN index is sharded by embedding-space partitioning (k-means centroids). Each query fans out to the most relevant 5-10 shards rather than all of them.
The Candidate Generator is a stateless service behind a load balancer, calling into a sharded index cluster. It autoscales on QPS and retrieval latency.
Index update path: when a new video lands and is approved for distribution, an embedding is computed via the video tower (usually on a GPU worker at ingest time) and upserted into the ANN index. The index supports online inserts but full rebuilds happen offline on a daily cadence to reclaim deletion tombstones and recompute centroids as the distribution shifts. Users with strongly evolving preferences get their user tower re-computed online from recent watch history, typically every few minutes.
Freshness vs quality: ANN indexes trade recall (how many of the true nearest neighbors you find) for latency (how fast the query returns). TikTok tunes for recall around 90-95 percent because feed quality is more sensitive to missed good candidates than to a few extra milliseconds of retrieval latency. This is a tunable knob per-shard at serving time.
Exact nearest-neighbor search over a billion-vector catalog is not just slow, it is prohibitively expensive. A brute-force scan at 1.2M QPS would need tens of thousands of CPUs just for retrieval, which would dwarf the ranker fleet. HNSW (or ScaNN) buys back that compute by trading 5-10 percent of recall for 100-1000x speedup. The interview lesson is that retrieval quality is a tunable knob, not a goal, recall above 95 percent rarely moves feed metrics but always costs money. Knowing when to stop tuning is part of the design.
Ranker
The ranker converts roughly 500 candidates into a scored ordering. This is the single largest compute sink in the serve plane.
- Architecture: multi-task deep neural network. The shared encoder produces a joint representation from user features, video features, and context features (time of day, session state, network quality). Task-specific heads predict:
- P(like)
- P(comment)
- P(share)
- E(watch-time-fraction)
- P(skip)
- Final score: a weighted combination of task scores, calibrated to watch-time as the primary north star. The weights are themselves tuned via offline A/B.
- Inference: served via TorchServe or TensorFlow Serving with dynamic batching (collect candidates into batches of 256 for GPU efficiency). Batching cuts per-inference cost by 10x with under 2ms of extra latency.
- Cascade: at very high scale, a cheap gate model (linear or small DNN) scores all 1000 candidates first, and only the top 200 go to the full DNN. This saves roughly 80 percent of inference cost with minor quality loss.
- Model versioning: every deployment is gated by offline metrics, then shadow traffic, then a small percentage A/B rollout. A bad model version is rolled back via the serving layer in seconds without touching client code.
- Calibration: raw DNN outputs are not calibrated probabilities by default. A separate calibration layer (Platt scaling or isotonic regression) converts logits into calibrated probabilities, which matters when you are combining multiple task heads into a score because the weights assume comparable scales.
The Ranker is the most instrumented service in the system. Every request logs feature vectors, model scores, final ranking, and downstream engagement (filled in after the user acts) so the training pipeline has ground truth for every served impression. This instrumentation is not free, it adds tens of milliseconds of per-request logging overhead and terabytes per day of log volume, but it is the only way the model quality keeps improving.
Feature Store
The Feature Store is the ML plane's contract with reality. It serves two masters: the online ranker and the offline training pipeline.
- Online tier (Redis):
user:{id}:featuresandvideo:{id}:featureskeys. Updated on every engagement event via Flink stream processors. Read latency under 500us at 1.2M QPS. - Offline tier (BigQuery or Hive on Parquet): historical feature snapshots, partitioned by day. Used to construct training examples with point-in-time correctness, critical to avoid training-serving skew where the model sees features during training that were not available at serving time.
- Feature registry: every feature has a spec (name, type, owner, freshness requirement, source). This governance layer lets teams collaborate without stepping on each other and lets the training pipeline guarantee the same feature logic is used in both online and offline paths.
Training-serving skew is the silent killer of ML systems. If the online and offline implementations of feature X disagree, the model learns something at training time that is not true at serving time, and quality silently regresses. A shared feature library plus point-in-time joins at training time are not optional at this scale.
Feature Categories
The actual feature set in production is broad. A non-exhaustive taxonomy:
- User features: demographic buckets, language, geo, device class, connection type, historical watch-time histogram by category, embedding of recently watched videos, follow-list embedding, last-session timestamps.
- Video features: duration, codec, uploader ID, creator embedding, audio fingerprint, hashtag embedding, ASR transcript embedding, engagement velocity, moderation score, age since upload.
- Context features: time of day, day of week, session length so far, videos watched this session, position in this feed batch, network throughput estimate.
- Cross features: similarity between user embedding and video embedding, historical engagement between this user and this creator, whether the user has followed creators in this video's hashtag space.
Each category has its own update cadence. User features update on every event (Flink stream). Video features update slowly (initial values on upload, engagement velocity every minute). Context features are computed per request (free). Cross features are derived per request from user and video features.
Transcoder
The transcoder is not a single box, it is a GPU-backed worker pool consuming from a Kafka queue.
- Input: raw upload in S3 (various container and codec mixes).
- Output: HLS with 2-second segments at 360p, 540p, 720p, and 1080p, encoded in H.264 (baseline for compatibility) and AV1 (progressive migration for bandwidth savings). A master manifest stitches the variants together for ABR streaming.
- Stack: FFmpeg with NVENC or QuickSync on GPU workers. Kubernetes autoscales the pool on queue depth.
- Parallelism: each video is transcoded in parallel across resolutions by splitting the job. A 60-second video is typically ready across all renditions within 30-90 seconds.
- Priority queue: videos from creators with high recent engagement get priority over low-engagement creators. This ensures trending content is distribution-ready faster.
- Thumbnail generation: a small pipeline step extracts representative frames from the video and runs them through a small ranking model to pick the most engaging thumbnail. The chosen thumbnail is what users see in the feed before playback starts, and it meaningfully affects tap-through rate.
- Error handling: if transcoding fails (rare, typically due to corrupted upload), the event is retried with exponential backoff and eventually moved to a dead-letter queue for human inspection. The video row status flips to
failedand the creator is notified with a clear error explanation.
Content Moderation
Moderation is the hardest ML problem in the system because mistakes are public and costly.
- Classifier ensemble: video frames are run through image classifiers for nudity, violence, and hate symbols. Audio is transcribed via ASR and run through text classifiers for policy violations. Audio is also fingerprinted (like ContentID) against copyrighted music.
- Full moderation stack: the ensemble actually has many specialized heads:
- Visual NSFW classifier on sampled frames (one every 2 seconds).
- Violence and graphic content classifier, a separate vision model.
- Hate-symbol detector, vision-based.
- ASR for audio transcription to feed text-based policy classifiers.
- Audio fingerprinting against the copyright hash database.
- Face detection for known-bad-person matching (region-specific, where legally required).
- Offline coordinated-behavior detector for bot networks, runs at a different cadence.
- Three-way decision:
- High confidence violating: auto-block. Video never enters distribution.
- High confidence safe: auto-approve. Video enters the candidate pool.
- Medium confidence: send to human review with SLA tiers. Videos from high-follower accounts get 1-hour SLA, from new accounts get 24-hour SLA. While pending, distribution is throttled but not blocked.
- Feedback loop: human reviewer decisions become labeled training data for the next classifier version.
- Appeals: creators can appeal a block. Appeals go to a separate human review queue with a higher bar (often two reviewers).
- Client-side pre-check: a small on-device model runs a basic NSFW check before the upload even starts. Obvious violations get an immediate error in the app, which saves server resources and shaves tens of seconds off the worst-case creator experience. The server still runs full moderation on every accepted upload, the client check is defense-in-depth, not a substitute.
Do not auto-block on medium-confidence signals. False positives here are a PR disaster, a legitimate video mistakenly blocked is a creator lost, and creators broadcast these incidents loudly. Medium-confidence should throttle (reduce distribution) and escalate, not kill.
Notification Service
The Notification Service reads from Kafka topics for like, comment, follow, and mention events and delivers via APNS (iOS), FCM (Android), and in-app WebSocket. Two things make this non-trivial at TikTok scale.
First, aggregation. A video going viral generates tens of thousands of likes per minute. Sending 10K individual push notifications to the creator would drain their battery and produce an unreadable stream of alerts. The service batches notifications in tumbling windows (typically 60 seconds) and collapses them: "X and 9,999 others liked your video."
Second, device fan-out. A single user often has multiple devices. The notification platform stores a device-token list per user and fans out to each active device, while also invalidating stale tokens reported by APNS and FCM as dead.
Idempotency is enforced via event IDs, Kafka delivers at-least-once so the Notification Service has to dedupe. A per-event Redis set with a short TTL catches duplicates.
Search Service
Search traffic is roughly 5-10 percent of feed traffic (users search far less than they scroll) but the service still has real engineering content because search intent is explicit and the quality bar is different from recommendation. The primary store is Elasticsearch, sharded by hashtag + user + video-id, with ASR (automatic speech recognition) transcripts indexed alongside titles and captions. ASR matters: it surfaces videos whose titles do not mention the user's query but whose spoken content does, which is most of TikTok's catalog since titles are short and decorative.
Autocomplete is served from a prefix index in Redis, refreshed every hour from aggregated query logs, giving sub-50ms response. Trending topics and hashtag lists are maintained by a Flink job consuming the engagement firehose with a 5-minute sliding window, so trending keywords update near-real-time without hammering Elasticsearch aggregations.
A deliberate architectural choice: search uses its own lightweight ranker (BM25 relevance plus creator reputation plus view-count recency) rather than the heavy FYP DNN. Two reasons. First, search intent is explicit and constrained (the user told you exactly what they want), so personalized ranking adds little on top of lexical match. Second, search QPS is low enough that paying FYP-ranker cost per query is an unforced cost. Index refreshes flow asynchronously via Kafka consumers. A few seconds of lag is acceptable because search users tolerate freshness delays that feed users would abandon.
Analytics Service
Creator analytics (views, retention curves, audience demographics, traffic sources) are served from ClickHouse, deliberately kept off the OLTP path. This separation is a classic OLTP-vs-OLAP boundary, worth naming explicitly in an interview. Analytical queries are wide, they aggregate across days and tens of thousands of videos per creator, often grouped by geography or cohort. Running those queries against MySQL would saturate an OLTP shard and ruin serving latency for everyone sharing that shard.
ClickHouse is sharded by creator-id for per-creator dashboards and by date partition for global trending queries. Materialized views pre-aggregate the common slices (views per day, retention curves per video cohort) so the dashboard UI does not recompute them on every page load. Ingest is append-only from the Kafka engagement firehose via ClickHouse's Kafka engine table, batched into parts every few seconds. End-to-end freshness from engagement event to creator dashboard is roughly 1-2 minutes, which is well inside what creators expect.
The trade-off worth naming: columnar storage is great for analytical scans but poor for point lookups, which is why ClickHouse is an OLAP sidecar and never the primary serving store. The same engagement event is written to Redis (for online counters with millisecond reads), Cassandra (for durable per-video totals), and ClickHouse (for analytical scans), and each store is chosen for its access pattern rather than forcing one store to do all three jobs.
Every architectural decision is a trade. Here are the ones an interviewer will probe, with the reasoning behind each choice.
Recommendation-first vs subscription-first feed
Subscription-first (Instagram, Twitter): feed is a function of who you follow. Simple cold-start for users with many follows, complicated cold-start for new users. Easy to explain to users, easy to moderate because each item's distribution is bounded by follower count.
Recommendation-first (TikTok, our choice): feed is a function of ML scoring over a personalized candidate pool. Excellent cold-start because the ranker has generic prior features even for zero-follow users. Long-tail creators can go viral without a built-in audience. But requires substantial ML infrastructure and a moderation pipeline that assumes any video could suddenly be shown to millions.
Why TikTok chose recommendation-first: it directly unlocks new-creator virality, which drives catalog growth, which drives recommendation quality, which drives engagement. The flywheel does not spin on a subscription-first architecture.
Pull-based vs push-based feed generation
Push (fan-out on write): works for subscription-first feeds. Precompute each follower's timeline when a creator posts. Writes amplify with follower count. Breaks on celebrities.
Pull (rank on read): TikTok's model. Candidates are pulled and ranked at request time. Zero write amplification. But every read runs the ranker, which is expensive.
Our choice: pull. At TikTok's scale, push would require a hybrid with extensive celebrity exceptions and would still fail the core goal of real-time personalization against just-watched signals.
HLS vs DASH vs MP4 progressive
MP4 progressive: simplest. One file, no manifest, no segments. Works for short clips but has no ABR, so users on poor networks get stuck buffering.
DASH: fully open standard, flexible, slightly better compression options, more complex server and client implementations.
HLS: Apple-originated, now industry standard. Excellent player support on iOS and everywhere else via hls.js. 2-second segments enable fast startup and ABR.
Our choice: HLS for primary playback. Everyone supports it, tooling is mature, and the 2-second segment size matches TikTok's sub-second startup target with minimal per-request overhead.
Candidate cache invalidation strategy
The TTL discussion was covered earlier (2-5 minutes in steady state). What belongs here is the invalidation trade-off: passive TTL alone is not enough because it forces every recommendation to be up to 5 minutes stale, and 5 minutes is forever on a platform where the trending set shifts minute-to-minute. TikTok combines passive TTL with three explicit invalidation triggers: a major engagement burst on a candidate the user just watched (invalidate to let the ranker respond), a profile-level signal change (follow, interest toggle, geo shift), and a feature-store delta crossing a threshold. The cost of aggressive invalidation is amplified retrieval load on the candidate generator. The cost of passive-only invalidation is staler recommendations and measurable session-length drop. The current balance is mostly passive with targeted active invalidation on high-value signals.
Monolithic ranker vs multi-task vs cascade
Monolithic (single output): simple deployment, no flexibility in re-weighting objectives.
Multi-task: separate heads for each behavior prediction. Flexibility to re-weight, plus shared-representation quality gains.
Cascade (cheap gate + full DNN): cost optimization. Only the top candidates from the cheap model go to the expensive one.
Our choice: multi-task plus cascade. Multi-task for product flexibility and training signal quality, cascade to keep GPU costs manageable at 1.2M QPS.
Redis vs Memcached
Redis wins on functionality (data structures, atomic operations like INCR, Lua scripting) at a modest cost in memory overhead. Memcached wins on pure key-value simplicity. For counters, feature blobs, and candidate caches, the atomic operations matter, which is why this design uses Redis throughout. A pure Memcached design would require client-side CAS loops for counters, which do not scale to 100K+ writes per second on a single key.
MySQL (Vitess) vs NewSQL (Spanner, CockroachDB)
NewSQL systems offer strong consistency at global scale with transparent sharding. That sounds like a free win over MySQL plus Vitess. At TikTok's scale the cost profile flips the other way, NewSQL licenses and the managed NewSQL per-operation cost at trillions of operations per month are vastly more expensive than Vitess over commodity MySQL. Vitess is operationally heavier (manual resharding, careful schema changes) but it is the chosen trade-off because the cost delta is measured in hundreds of millions of dollars per year. The lesson for the interview: the "better" technology at small scale can be the "worse" technology at TikTok scale once economics dominate.
This same economic flip shows up across the design: Vitess over Spanner, owned CDN over Cloudflare, in-house GPU fleet over managed inference, Kafka plus Cassandra over a managed event platform. At small scale these are the harder, uglier choices, at TikTok scale they are each measured in nine-figure savings per year. A senior answer does not just name the chosen technology, it names the threshold at which the choice flipped, because that threshold is what the interviewer is looking for.
Following Feed Design
The Following tab is the secondary product surface, an explicit graph-based feed that shows only posts from accounts the user follows. Traffic share is estimated at 5-10 percent of sessions, small enough that the feed is computed on read rather than pushed at write time. When the user opens the tab, the Feed Service looks up the follow list (sharded MySQL, 99p under 20ms), queries a per-creator recent-videos cache (Redis list keyed by creator-id, holding the last 50 video-ids with a 1-hour TTL), merges the results by publish time, and returns the top N. No per-user timeline is materialized.
The sharp edge is power users who follow thousands of creators: merging 2000 creator-lists on each open is expensive even from Redis. The mitigation is a two-tier read path. For users following fewer than 200 creators (roughly 95th percentile), the simple merge runs directly. Above that threshold, a precomputed per-user Following-tab cache is maintained with a 30-second TTL, amortizing the merge cost across tab-open frequency. This is a deliberate hybrid, not a pure pull or pure push. The reason to not push universally is the same as FYP: most users rarely open Following, so precomputing timelines that nobody reads is pure waste.
Bandwidth-Adaptive Streaming Ladder
The ladder of rendition bitrates (360p at about 500 kbps, 540p at about 1 Mbps, 720p at about 2 Mbps, 1080p at about 4 Mbps) is itself a trade-off. More rungs give the ABR player finer control but increase storage and transcoding cost. TikTok ships a 4-rung ladder because the marginal quality gain from adding a fifth rung (say 540p60) is smaller than the storage and transcoding cost across a billion-video catalog. This is a pure math problem of encoded storage versus perceived quality, solved empirically.
Do not propose pure push-based feeds (fan-out on write) for TikTok in an interview. It fundamentally cannot express the real-time personalization against just-watched signals that defines the product. Subscription-first fan-out is the wrong tool, say so explicitly.
Kafka vs Per-Event REST
At every ingest and feedback boundary, events could in theory flow over synchronous REST calls between services. In practice, REST would make every downstream service a hard dependency of every producer. If the Notification Service is down, does the Like API return an error? Under REST, yes, the call fails. Under Kafka, no, the producer writes to Kafka and the Notification Service catches up when it recovers. At the scale of 100B+ events per day, temporary downstream outages are constant, and Kafka is what makes them non-user-visible. The cost is at-least-once semantics (forcing idempotent consumers) plus the operational overhead of running a clustered message broker. Both costs are worth it.
Synchronous vs Asynchronous Moderation
Synchronous moderation (block the upload response until moderation completes) is conceptually cleaner: the video is either approved or rejected before the client moves on. But it forces the creator to wait potentially tens of seconds staring at a progress bar, and it couples upload latency to moderation model latency, which means one slow moderation inference degrades the entire upload funnel. Async moderation (accept the upload immediately, gate distribution on moderation verdict) decouples the two and keeps the creator's experience snappy. The cost is slightly more complex state handling (videos in "pending moderation" state need a UI affordance) but the user-experience benefit is substantial.
CDN: Commercial vs Owned
The commercial CDN question (Cloudflare, Akamai, Fastly) versus owned infrastructure (TikTok's own edge PoPs embedded inside ISP data centers) comes down to scale. Below a few petabytes per day of egress, commercial CDNs win on ease of use and breadth of coverage. Above that, the economics invert: the marginal bandwidth cost at owned infrastructure is a small fraction of commercial per-GB pricing. TikTok at 500 PB per day of egress crosses that threshold by two orders of magnitude. The hybrid in practice is owned primary, commercial CDN as overflow or secondary-region coverage where owned PoPs have not yet been built out.
A production TikTok fails in specific, predictable ways. Designing for these before they happen is what separates a safe launch from a public incident.
Viral video hot key
What fails: 100M views hit a single video in one hour. The CDN origin saturates. The like counter in Redis receives 50K INCRs per second on one key, hitting CPU ceiling on that shard. The video metadata row is hit 100M times and the MySQL shard warms up to 100 percent CPU.
Mitigations:
- Tiered caching: the first segment of a trending video is pre-pushed to every edge PoP globally within minutes of detection. This removes origin from the hot path.
- Counter sharding: the like counter for any video above a traffic threshold is split into 16 sub-counters (
video:{id}:likes:{0..15}) that are summed on read. Writes hash-spread across sub-counters, reads sum. - Metadata caching: the video row is cached in Redis with a long TTL because video metadata is essentially immutable once published.
- Signed URL expiry is tuned so a single viral video does not cause a stampede of signed-URL regenerations at the instant its URL expires.
Ranker latency spike
What fails: a new ranker version has a quadratic bug in a feature transformer. p99 inference latency jumps from 30ms to 800ms. Feed requests time out, client sees spinning wheels, session length craters.
Mitigations:
- Circuit breaker at Feed Service: if Ranker p99 crosses 100ms or error rate crosses 1 percent, open the circuit and fall back to cached ranked lists (previous successful ranker output per user) or, as a last resort, popularity-ranked candidates.
- Shadow traffic during rollout: new ranker versions receive 1 percent of traffic in shadow mode (inference runs but results are not served) for 24 hours before any real rollout. Latency regressions are caught here.
- Automated rollback: deployment system watches for p99 regression and rolls back within minutes without paging.
Thundering Herd on Candidate Cache
What fails: many users share candidate cache shards. If a popular shard is evicted or restarted, thousands of users simultaneously miss the cache and stampede to the Candidate Generator, which cascades into ANN index overload.
Mitigations:
- Request coalescing: the Feed Service wraps candidate-cache reads in a single-flight primitive. If 100 simultaneous requests miss the cache for the same user at the same moment, only one goes to Candidate Generator and the other 99 wait for its result.
- Warm standby: candidate cache shards have replicas. On primary failure, a replica takes over with the hot data already loaded.
- Graceful degradation: if Candidate Generator is saturated, fall back to a regional trending list for a few minutes until the cache warms.
Candidate pool poisoning
What fails: a bug in the retrieval service causes an outdated or corrupt video embedding to be inserted into the ANN index. Low-quality or even removed videos reappear in candidate lists.
Mitigations:
- Staged index rollout: new index versions are built in a staging cluster and served to 1 percent of traffic before full cutover. Quality regressions are caught offline.
- Quarantine pipeline: any video with an engagement-quality metric below a floor is automatically removed from the candidate pool, not just the ranker output. This limits the blast radius of a bad video that slips through.
- Embedding validation: ingest writes to the index are validated against a schema (dimension, norm, nan-check) before acceptance.
Moderation backlog
What fails: a large event (world-cup, election) drives an upload spike. Classifier queue grows from seconds to hours. Un-moderated content piles up.
Mitigations:
- Priority queue: videos are queued by predicted view velocity (based on creator history and initial engagement). High-velocity videos are moderated first so the highest-impact content clears the queue fastest.
- Distribution throttling: while a video sits in the moderation queue past its SLA, its distribution is automatically capped (shown to at most N users per minute). Not blocked, just throttled, so moderation lag does not suppress legitimate content entirely.
- Autoscaling: the classifier inference pool autoscales on queue depth. Human review gets a staged overflow path to additional reviewer pools.
Database Shard Hotspot
What fails: a popular creator's activity hits a single sharded MySQL node too hard. Reads for their video metadata, follower list, and engagement summary saturate the shard's CPU.
Mitigations:
- Read replicas: every SQL shard has multiple read replicas with the same schema. Reads are load-balanced across replicas, writes go to the primary. For most query patterns this gives linear scaling on the read side.
- Aggressive caching: creator profile, video metadata, and follower-list summaries are cached in Redis. Cache hit rate for hot creators is above 99 percent.
- Bucket partitioning for celebrities: as described in the database design section, followers_by_user is bucket-partitioned for users over a follower threshold. This eliminates the celebrity hotspot specifically.
Kafka Lag
What fails: consumer group lag on the engagement topic grows from seconds to hours. Online features fall behind, the ranker's feedback loop is stale, training data ingestion lags.
Mitigations:
- Consumer autoscaling: each consumer group scales on lag. A spike in events triggers more consumer instances.
- Lag alerting: if lag exceeds a threshold, on-call is paged. Lag beyond 60 seconds on the engagement topic is a quality incident even if nothing else is failing.
- Decoupling: downstream consumers (notification, feature store, analytics) are independent consumer groups, so lag in one does not affect the others. A bug in the notification consumer does not degrade the feature store.
Transcoding queue overflow
What fails: upload spike exceeds transcoder throughput. Kafka queue grows to hours of backlog. Users wait hours to see their videos live.
Mitigations:
- GPU autoscaling: the transcoder pool scales on queue depth, adding GPU nodes until the queue drains.
- Back-pressure on upload: if the transcode queue exceeds a high watermark, the Upload Service throttles new uploads with a soft rate limit rather than accepting uploads that will wait an hour.
- Priority lane for high-engagement creators: creators whose recent videos hit high view velocity get their new uploads transcoded first. This preserves the critical virality path even under load.
Circuit breakers are only effective if the fallback is pre-computed. A feed service that opens its circuit and then tries to compute a fallback synchronously still blows the latency budget. The cached ranked list or pre-computed popularity list must be ready before the circuit opens, not computed at the moment of failure.
Region Outage
A full regional failure (DNS, power, network partition) is rare but catastrophic. Mitigations are architectural, not reactive. Global Anycast routing fails traffic over to the nearest healthy region within minutes. Cross-region Cassandra replication (at least RF=3 with at least 1 replica in a different region) keeps local feed data available remotely. Kafka MirrorMaker lags by seconds under steady state, so cross-region feedback can resume with minimal loss. The CDN continues serving cached segments even if the origin region is dark. The part that cannot be regionally replicated cheaply is the candidate-pool ANN index (too large, too hot to maintain duplicate indexes) so during a regional outage the degraded region serves from a slightly stale ANN index that catches up when the originating region recovers.
Feature Store Drift
A subtle production failure: the online feature store and the offline training pipeline begin to disagree on feature values due to a schema change that was only applied in one pipeline. The ranker trained on one distribution is now served a different distribution online, and model quality silently regresses. Detection requires continuous feature-distribution monitoring (KL divergence between online and offline feature histograms) plus alerting on drift. Mitigation requires a rigorous feature definition shared between training and serving, usually via a feature library. This failure mode does not cause an outage page, it causes a slow erosion of product metrics over days, which is arguably worse because it is invisible until the damage is done.
Cold Start After a Long Absence
A user returning after a month has stale online features (the event-driven updates expire after a TTL in Redis). The first feed request has a cache miss and must recompute user features from recent history plus defaults. This adds latency to the first batch but is rare enough that the p50 impact is negligible. The mitigation is a warm-up path: when auth detects a returning user, a background job prefetches and refreshes their online features so the first feed request hits a warm cache.
TikTok at its current scale is already hard. The next frontier is even harder.
Multi-modal embeddings
Today the video embedding is learned primarily from engagement and a handful of computed features. The next generation unifies vision, audio, and text into a single embedding via a multimodal transformer. Instead of separate streams for "videos using this sound" and "videos with similar content," one embedding space covers both, retrieving videos that are similar in content, style, or audio mood in one query. This also dramatically improves cold-start for new videos, the embedding is high quality from the first upload because it comes from the content itself, not from accumulated engagement.
Cold-start is where recommendation systems live or die. A new video with zero engagement data is invisible to an engagement-trained ranker, so the platform depends on exploration slots to collect signal, which is expensive and slow. Content-based multi-modal embeddings short-circuit that loop, the retrieval system can surface a good match from the first upload. The payoff is not just better recommendations, it is a flywheel where creators get distribution faster, which increases upload volume, which feeds back into a richer catalog.
On-device ranking
The current ranker scores 500 candidates server-side. At 1.2M QPS this is a massive GPU bill. One experimental direction is to send the top-1000 candidates (plus compact feature vectors) to the device and run the final ranker on-device. Mobile chips can execute a 10M-parameter DNN in tens of milliseconds, and moving ranking to the device eliminates a full server round trip for fine-grained re-ranking after each swipe. The trade-off is model distribution complexity (shipping model updates to 500M devices) and feature-drift risk (the device cannot easily see the freshest server-side features). Early experiments suggest hybrid: coarse ranking server-side, fine re-ranking on-device.
The infrastructure implication is substantial: device-ranked feeds need a compact feature vector per candidate (say 64-256 floats) shipped alongside the video manifest, plus periodic model updates to clients. This adds maybe 10-20 KB per candidate to the feed response, which is still small relative to video bytes, but it does mean the Feature Store's online path has to serve compact representations, not just full feature blobs.
Live streaming integration
Live streaming is a different plane: real-time media ingest, live chat, real-time recommendation-to-live-stream matching. The current architecture supports it via additional ingest services (RTMP, WebRTC) and dedicated edge media servers, but the live discovery problem (who should see this live stream right now, given a few seconds of latency tolerance) is non-trivial. The candidate generator needs a "currently live" index with sub-minute freshness.
The live chat component also has its own scaling properties. A popular live stream can have millions of concurrent viewers commenting in real-time. Standard WebSocket fan-out breaks down at that concurrency, so the chat service uses a hierarchical pub/sub architecture with regional message brokers and a rate-limiting layer that samples comments for display rather than attempting to show every comment to every viewer.
Ads Personalization
Ads are a separate subsystem that overlays the feed. A sponsored video is inserted at a predictable slot (say every 8-10 items in the feed). The ad selection uses a parallel ranker tuned on ad conversion objectives rather than organic engagement, and it integrates with an ad delivery system that handles advertiser bids, pacing, and frequency capping. From the user's perspective, ads appear in the feed like any other video, but the underlying selection logic and backend storage are entirely separate. This is intentional: organic and paid should not contaminate each other's training data, and the economics of the ad business (CPM, attribution, advertiser controls) do not fit cleanly into the organic engagement model.
TikTok Shop and e-commerce
Product links embedded in videos turn the feed into a shopping surface. This introduces a new objective (conversion) into the ranker, which means one more head on the multi-task DNN and a new feedback loop (click -> product page -> purchase -> attribution back to the originating video). Importantly, this creates new regulatory obligations (product claims, consumer protection, payments) that the architecture must support without compromising feed latency.
Regional models and regulations
Different regions have different content norms and laws. Rather than one global model, the future is regional ranker variants that share a common encoder but have region-specific heads and training data. Content compliance (take-down latency, data residency) becomes a first-class requirement, not an afterthought. The storage plane may shard further to meet data-residency rules, and the feature store may have per-region isolation.
Creator Monetization Platform
Creators currently monetize primarily through ad revenue share and creator funds. The natural expansion is direct tipping, paid subscriptions, paid collaboration marketplaces, and programmatic deals with brands. Each of these is a new transactional subsystem with strong-consistency payment paths, tax and regulatory overhead, and complex payout flows. The key architectural insight is that this subsystem must not sit on the serve plane's hot path, payments-related failures would be catastrophic if they touched feed latency, so it lives as a separate plane with its own SLAs and its own event integrations.
Trust and Safety Infrastructure
Beyond moderation of individual videos, a production platform needs infrastructure for coordinated-abuse detection (bots, spam campaigns), misinformation tracking, age-appropriate content delivery, and legal-compliance takedowns. This is an investment in detection pipelines, labeling infrastructure, and red-team tooling. At 500M DAU, even a 0.01 percent bad-actor rate is 50,000 daily adversaries, so this is not a small team's hobby project, it is a first-class engineering discipline with its own architecture.
Edge Compute for Personalization
Today most personalization work happens server-side. The next step is pushing lightweight personalization to the edge PoP, reordering prefetch priorities, adjusting video quality ladders, personalizing first-frame rendering based on device capabilities, all at the edge to shave more latency off the first-frame target. Edge compute is a relatively new pattern and forces the team to decide which parts of the ranker can safely run without full user state, but the latency wins are real at the margin where every tens of milliseconds matters.
Try this problem yourself
Reading a solution is not the same as producing one under interview pressure. Design it on the whiteboard and get scored feedback.
0%
Editorial Solution
System requirements
Functional
Non-Functional
Why This Differs From Instagram
Scope Control for the Interview
What to Not Design For Day One
Read/Write Asymmetry
Two Distinct User Loops
Capacity estimation
Traffic
Daily Active Sessions
Storage
Bandwidth
Why This Matters
Compute Budget for Ranking
Why Not Store Everything in RAM
Working Set and Cache Efficiency
Cost Breakdown
API design
Upload
Feed
Engagement
Social graph
Search
Moderation
Auth
Rate Limits and Abuse Protection
Why These Choices
Pagination Semantics Matter
Why gRPC Internally But REST at the Edge
Versioning
Error Handling
Database design
Video metadata: sharded MySQL / Vitess
Social graph: sharded MySQL with denormalized follow tables
Engagement counters: Redis with async Cassandra flush
Feed candidate cache: Redis
Feature store: hybrid online/offline
Search: Elasticsearch
Analytics: ClickHouse or Druid
Notifications: Cassandra with TTL
Sessions and Auth: Redis
Object storage: S3, with Glacier tiering
Keeping the Hot Path Consistent Enough
Glacier Tiering in Practice
High-level design
The four planes
Why four planes instead of one service mesh diagram
Key shared infrastructure
Service-Level Ownership
Statelessness at the Edge
Why a Model Registry Is a First-Class System
Request flows
Upload flow
FYP read flow
Swipe prefetch
Engagement Write Flow
Follow Flow
Detailed component design
Candidate Generator (retrieval)
Ranker
Feature Store
Feature Categories
Transcoder
Content Moderation
Notification Service
Search Service
Analytics Service
Trade-offs and tech choices
Recommendation-first vs subscription-first feed
Pull-based vs push-based feed generation
HLS vs DASH vs MP4 progressive
Candidate cache invalidation strategy
Monolithic ranker vs multi-task vs cascade
Redis vs Memcached
MySQL (Vitess) vs NewSQL (Spanner, CockroachDB)
Following Feed Design
Bandwidth-Adaptive Streaming Ladder
Kafka vs Per-Event REST
Synchronous vs Asynchronous Moderation
CDN: Commercial vs Owned
Failure scenarios and bottlenecks
Viral video hot key
Ranker latency spike
Thundering Herd on Candidate Cache
Candidate pool poisoning
Moderation backlog
Database Shard Hotspot
Kafka Lag
Transcoding queue overflow
Region Outage
Feature Store Drift
Cold Start After a Long Absence
Future improvements
Multi-modal embeddings
On-device ranking
Live streaming integration
Ads Personalization
TikTok Shop and e-commerce
Regional models and regulations
Creator Monetization Platform
Trust and Safety Infrastructure
Edge Compute for Personalization