Requirements

Functional Requirements:

• Allow users to stream music on demand from a music library, with high-quality audio streaming and playback controls.
• Allow users to create, edit, reorder, and delete personal playlists of songs.
• Enable full-text search across the music catalog by title, artist, album, or genre.
• Support personalized recommendations of songs and playlists based on listening history and preferences.
• Allow users to download songs for offline listening, with DRM protection for cached content.
• Allow users to share playlists or songs, set public/private visibility, follow friends, and see what others are listening to.
• Rate limiting: Per-user rate limits on API calls (search, playlist edits) to prevent abuse. Streaming endpoints rate-limited per account to enforce single-stream policy for free-tier users.

Non-Functional Requirements:

• List the key non-functional requirements (eg low latency, scalability, reliability, etc.)...
• The system should have low latency as user want to listnen the song
• The system should be highly scalable to handle multiple users want to stream different songs from different regions in the world
• The system should have eventual connsistency in terms of storing user preference and storinng listening history
• The system should have search function that is scalabile into huge storage of songs
• The system should be highly availble to have most up times

API Design

Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...

Streaming API

GET /v1/stream/songs/{songId}?quality={bitrate}

Returns a response containing a signed CDN URL for the audio file, song metadata (title, artist, album art URL, duration), and the available bitrate options. The signed URL contains an HMAC signature, expiry timestamp (15 minutes), and the user's nearest CDN region. The client uses this URL to fetch audio chunks directly from the CDN, the backend never serves audio bytes.

Why return a signed URL instead of proxying audio through the API? Because proxying 1.28 Tbps of audio through backend servers would require thousands of servers just for bandwidth. Signed URLs offload delivery to the CDN while preventing unauthorized access (the URL expires and is tied to the user's session).

POST /v1/events/play

json

Fires after 30 seconds of playback (the industry-standard threshold for counting a "stream" for royalty purposes). Fire-and-forget from the client's perspective, the API returns 202 Accepted immediately and publishes to Kafka asynchronously. If the client can't reach the API, it queues the event locally and retries on next connectivity.

Playlist API:

POST /v1/playlists: Create a new playlist (name, description, public/private).

PUT /v1/playlists/{playlistId}/songs: Add or remove songs. Body contains an array of operations: [{action: "add", songId: "...", position: 5}, {action: "remove", songId: "..."}]. Supports batch operations to reduce round trips.

PATCH /v1/playlists/{playlistId}/songs/reorder: Move a song from one position to another. Uses optimistic concurrency: request includes expectedVersion, server rejects with 409 if the playlist was modified since.

Recommendations API

GET /v1/recommendations?seed_songs={ids}&seed_artists={ids}&limit=30

Returns personalized song recommendations. Seeds are optional, if provided, recommendations are biased toward similar content. Without seeds, the system uses the user's full listening profile. Response includes a reason field explaining why each song was recommended ("Because you listened to Artist X", "Popular in your area").

High-Level Design

Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.

Client apps (mobile, web, desktop): Handle audio playback, local caching for offline mode, UI rendering, and adaptive bitrate switching. The client is "smart", it manages its own playback buffer, prefetches upcoming songs, and queues play events for retry if offline. This pushes complexity to the client and keeps backend services stateless.

API Gateway: Single entry point for all client requests. Handles JWT authentication, rate limiting (token bucket per user), request routing to backend services, and TLS termination. Returns cached responses for hot endpoints (trending playlists, popular searches). Implemented with Nginx or AWS ALB + custom auth middleware.

Streaming Service: Receives play requests, validates the user's subscription (can they stream 320kbps?), fetches song metadata from PostgreSQL, generates a signed CDN URL, and returns it. Stateless, scales horizontally. Does NOT serve audio bytes.

Search Service: Thin layer over Elasticsearch. Handles query parsing (tokenization, typo correction), executes multi-field search queries, applies personalization re-ranking (boost songs by artists the user has listened to), and returns paginated results. Caches popular queries in Redis (TTL 5 minutes).

Recommendation Engine: Serves pre-computed recommendations (Discover Weekly, Daily Mix) from a recommendation cache. For real-time requests, combines collaborative filtering embeddings with content-based features to generate on-demand suggestions. Models trained offline on Spark; serving layer is a lightweight API reading from a vector store. Cold-start users (new accounts with no listening history) receive country-specific popular playlists and genre-based suggestions from the onboarding flow (the user picks 3+ artists they like on signup); as listening data accumulates over the first few sessions, the system transitions to personalized recommendations.

Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.

Database design

Schema

PostgreSQL for songs, artists, albums, users, playlists, these are relational data with complex joins (song → album → artist, playlist → songs). Read replicas handle the 200K metadata reads/sec. The total structured data is ~500GB, well within a single PostgreSQL cluster with partitioning.

Elasticsearch for search, the songs table is replicated to an ES index with analyzers for autocomplete (edge n-grams), typo tolerance (fuzzy matching), and multi-field search. ES is not the source of truth; PostgreSQL is. A CDC (Change Data Capture) pipeline keeps ES in sync.

S3 (or equivalent object storage) for audio files, each song stored as {song_id}/{bitrate}.opus (e.g., song_abc/128.opus). Object storage is the only cost-effective option at 3PB. Files are immutable once written.

Kafka for play events, the append-only event stream handles 48K writes/sec. Events are consumed by multiple downstream systems (analytics, recommendations, royalties). Retention: 7 days in Kafka, then archived to Parquet on S3.

Redis for session state, active stream registry (which user is streaming on which device), rate limiting counters, and cached recommendation results. Total Redis footprint: ~50GB (10M active sessions × 500 bytes + caches).

Event Pipeline (Kafka): Ingests play events, playlist edits, search queries, and other user actions. The primary play-events topic is partitioned by song_id for efficient per-song aggregation (royalty calculation, play counts). A secondary topic re-partitioned by user_id supports user-level analytics and recommendation retraining. Multiple consumer groups: analytics aggregator, recommendation retrainer, royalty calculator, fraud detector (detecting play count manipulation).

Data stores: PostgreSQL (metadata), Elasticsearch (search), S3 (audio files), Redis (sessions, caches), Kafka (events). Each chosen for its strength: PG for consistency, ES for search, S3 for scale, Redis for speed, Kafka for throughput.

CDN: Globally distributed edge nodes serving audio chunks. Pre-warms popular content based on regional popularity data. Handles 95%+ of streaming bandwidth, insulating the origin from traffic spikes. Multi-CDN strategy for resilience.

ong Playback Flow

End-to-end song playback: signed URL generation, CDN chunk delivery, and event logging

1. User taps play on a song in the client app.
2. Client sends GET /v1/stream/songs/{songId}?quality=128 to API Gateway.
3. Gateway validates JWT, extracts user claims (plan_type, user_id), routes to Streaming Service.
4. Streaming Service queries PostgreSQL: SELECT audio_key, duration_ms FROM songs WHERE song_id = $1. Checks if requested quality is within user's plan limit.
5. Generates signed CDN URL: https://cdn.example.com/{audio_key}/128.opus?sig={hmac}&exp={timestamp}. Signs with HMAC-SHA256 using a shared secret between backend and CDN.
6. Returns JSON response: {url: "...", duration_ms: 210000, title: "...", artist: "...", albumArt: "..."}.
7. Client begins fetching audio from CDN using HTTP Range requests. First chunk (bytes 0-262143) starts playback immediately. Client maintains a 10-second lookahead buffer.
8. CDN edge serves from cache (95% of popular songs). On cache miss, CDN fetches from S3 origin, caches, and serves.
9. After 30 seconds of playback, client fires POST /v1/events/play with duration, context, and timestamp. Gateway publishes to Kafka topic play-events.

Search Flow

1. User types "bohemian" in the search bar. After 2 characters, client sends autocomplete request: GET /v1/search?q=bo&autocomplete=true.
2. Search Service executes an edge n-gram prefix query on Elasticsearch: matches "Bohemian Rhapsody", "Born to Run", "Bones", etc.
3. Returns top 5 suggestions (title + artist only) within 50ms. Client displays as dropdown.
4. User selects "Bohemian Rhapsody" or continues typing. Full search sends GET /v1/search?q=bohemian+rhapsody&type=songs.
5. Elasticsearch executes multi-match query across title (boost 3x), artist (2x), album (1x). Fuzzy matching handles typos (edit distance 1-2).
6. Results re-ranked by personalization layer: boost songs by artists the user has listened to. Penalize songs the user has recently skipped.
7. Response includes highlighted snippets, album art URLs, and relevance scores. Paginated with cursor-based pagination for efficient scrolling.

Playlist Edit Flow

1. User adds 5 songs from an album to a playlist.
2. Client sends batch request: PUT /v1/playlists/{id}/songs with operations array and expectedVersion: 42.
3. Playlist Service begins transaction: check version = 42 in playlists table. If mismatch, return 409.
4. Insert 5 rows into playlist_songs with positions appended after the last song. Increment song_count by 5. Set version = 43.
5. Commit transaction. Return updated playlist metadata with new version.
6. If the playlist is collaborative, publish playlist_updated event to WebSocket channel. Other collaborators' clients receive the delta and update their local view.