System requirements

Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

• User account management (registration, authentication).
• Music upload and storage.
• Music streaming with high audio quality.
• Playlist creation and management.
• Music discovery ( charts).

Non-Functional:

List non-functional requirements for the system...

• Scalability to handle millions of users.
• Low-latency streaming for optimal user experience.
• High availability and fault tolerance.
• Security for user data and payment information.

Capacity estimation

Estimate the scale of the system you are going to design...

API design

Define what APIs are expected from the system...

GET songs/song_id

sample response

{

song_name: "xxx",

artist: "xx",

length: "xxxx"

}

GET artists/artist_id

sample response

{

artist_name: "xxx",

songs: [] song_id,

}

Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...

We can use relational database for user, song metadata, playlist information. But we also need a blob storage to store the actual audio files.

User table

name

user_id

genre

playlists: [] playlist_id

Playlist table

user_id

playlist_name

playlist_id

songs: [] song_id

Song Metadata table

name

song_id

artist

length

last_updated_time

album_id

chunks: {url: xxxx, status: not_uploaded/uploaded}

High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...

Client -> load balancer -> CDN -> server -> music uploading service

-> music streaming service

both services point to both S3 and relational database

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

When a user streams music

1. client makes a request to stream a song
2. server looks up the song_id in the song table, and fetch the song url
3. server sends back streaming data from the song url
4. client streams the audio file
5. If the song is in CDN, the song can be played from CDN directly
6. When a song is being streamed, the client continues to request the subsequent chunks

When an audio file is uploaded

1. The client would divide the video file into chunks, each with a fingerprint hash. A chunk would be small, ~5-10MB in size.
2. song metadata would have a field called chunks which would be a list of chunk JSONs, each with fingerprint and status field.
3. The client would POST request to the backend to update the song metadata with the list of chunks, each with status NotUploaded.
4. The client would upload each chunk to S3.
5. S3 would fire S3 event notifications that would update the song metadata by marking the chunk (identified by its fingerprint) as Uploaded.
6. If the client stopped uploading, it could resume by fetching the song metadata to see the uploaded chunks and to skip chunks that had been uploaded already.

When a user searches for a song

1. search bar hits ElasticSearch for search suggestions matching the user query
2. user selects a specific suggestion
3. ElasticSearch then make a match on those suggestions and return a list of matching records

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Audio uploading

Audio files can be split into chunks -> all the chunks are dumped into a queue -> consumer pulls jobs into processor to process the chunk (encoding, transcoding..etc) -> a heartbeat and status monitor worker listens to the processor and consumer and update the status of the chunks accordingly. If a processor fails, a new processor will replace it, and repull the jobs from the queue and process it again. If a job needs to retry, processor tries it 3 times on the spot and if it still fails, update the status to be failed and throw it in a dead letter queue.

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?