Loading...
List functional requirements for the system (Ask the chat bot for hints if stuck.)...
1) User Registration Allow users to crate acccounts and log in securely
2) Provide a vast categoires of songs , albums and artists to search for
3) Allow users to crate, edit their playlist
4) Allow music to be played at different speeds and
5) Send Notifictions for new releases from favorite artists
6) Ensure that users have seemless experience across cross devices
List non-functional requirements for the system...
1) System should be highly scalable
2) System should be fault tolorent
3) System should have high throughput
Estimate the scale of the system you are going to design...
Lets assume we have 1 billion total users out of which we have 100 million active users and each user listens to 5 songs in a day so number of qps
5 * 100,000,000/100,000 = 500 qps
Define what APIs are expected from the system...
Admin APi
POST /v1/User Allow user to access spotify
request Body ->
{
"firstName:"",
"lastName":"",
"phoneNo":"",
}
DElete /v1/User - Allow user to delete the user
request body ->
{
userId:"userId"
}
Put /v1/User Api is used to update the user details
{
userId:""
firstName:"",
}
Post /v1/Song -> Allow Artist to add a new Song
{
songId:"",
songName:"",
songGenre:"",
}
Get /v1/Song?songName Allow users to play a particular song
Get /v1/?Genre=genreName allow users to search for a particular genre
Post /v1/addSong -> Allow users to add a particular song to the playlist
{
playListName:"plyaList Name"
song:"songName"
}
Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...
As for spotify we will have more reads than writes as artist can upload atmost 1-2 songs in a day but the same song can be played many times in a day
so we will be using relational database for our case we will have the following table
1 )uerTable -> This table will store the user details for a particular user
2) PlayList -> This table will have the playList details for each user
3) Songs -> This table will store the metadata about the songs
4) Follow Follower Table -> This table will store the follower follow relationship
5) Artist Table -> This table will store the details about the artist
we will also be storing metadata about the songs in elastic search so that it helps with searching songs by geners, albums,artist and etc
elastic search internally uses inverted index which will help us with quick search for a song
You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...
We will also store metadata about the songs in Elasticsearch to facilitate searching by genres, albums, artists, and more. Elasticsearch uses an inverted index, enabling quick song searches. To scale the relational database, we will shard based on the first character of the song title, placing songs from A-K in the first shard, K-W in the second shard, and X-Z in the third shard, allowing us to easily scale the relational database. Each user request is made to the API gateway, which implements authentication and rate limiting using algorithms like token bucket or fixed window. If it detects too many requests, it responds with a status code 429 and a "Too Many Requests" message. Once the request is successfully authenticated, it is forwarded to the API server. For adding a new song, the artist uploads the song, and the API server calls the upload service, which pushes the song metadata to a message queue or event bus. The consuming service reads the metadata, stores it in the database, and also indexes it in Elasticsearch for efficient searching. The song is then uploaded to blob storage such as Amazon S3 or Azure Blob Storage. After the song is successfully uploaded, the alert service queries the follower service to retrieve all followers of the artist, ensuring high scalability by sharding the follow-following database, and notifies each user about the new song upload. When a user requests to play a song, the play service retrieves the URL from the blob storage, fetches the song, and plays it in an MP4 player. Similarly, we will query the playlist service and follower service to add songs or follow artists. If a song reaches a certain number of plays, such as 10 million, it will be stored in cache or at a CDN, reducing the time to serve the song and ensuring low latency and high throughput.
Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...
We will also store metadata about the songs in Elasticsearch to facilitate searching by genres, albums, artists, and more. Elasticsearch uses an inverted index, enabling quick song searches. To scale the relational database, we will shard based on the first character of the song title, placing songs from A-K in the first shard, K-W in the second shard, and X-Z in the third shard, allowing us to easily scale the relational database. Each user request is made to the API gateway, which implements authentication and rate limiting using algorithms like token bucket or fixed window. If it detects too many requests, it responds with a status code 429 and a "Too Many Requests" message. Once the request is successfully authenticated, it is forwarded to the API server. For adding a new song, the artist uploads the song, and the API server calls the upload service, which pushes the song metadata to a message queue or event bus. The consuming service reads the metadata, stores it in the database, and also indexes it in Elasticsearch for efficient searching. The song is then uploaded to blob storage such as Amazon S3 or Azure Blob Storage. After the song is successfully uploaded, the alert service queries the follower service to retrieve all followers of the artist, ensuring high scalability by sharding the follow-following database, and notifies each user about the new song upload. When a user requests to play a song, the play service retrieves the URL from the blob storage, fetches the song, and plays it in an MP4 player. Similarly, we will query the playlist service and follower service to add songs or follow artists. If a song reaches a certain number of plays, such as 10 million, it will be stored in cache or at a CDN, reducing the time to serve the song and ensuring low latency and high throughput.
Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...
We will also store metadata about the songs in Elasticsearch to facilitate searching by genres, albums, artists, and more. Elasticsearch uses an inverted index, enabling quick song searches. To scale the relational database, we will shard based on the first character of the song title, placing songs from A-K in the first shard, K-W in the second shard, and X-Z in the third shard, allowing us to easily scale the relational database. Each user request is made to the API gateway, which implements authentication and rate limiting using algorithms like token bucket or fixed window. If it detects too many requests, it responds with a status code 429 and a "Too Many Requests" message. Once the request is successfully authenticated, it is forwarded to the API server. For adding a new song, the artist uploads the song, and the API server calls the upload service, which pushes the song metadata to a message queue or event bus. The consuming service reads the metadata, stores it in the database, and also indexes it in Elasticsearch for efficient searching. The song is then uploaded to blob storage such as Amazon S3 or Azure Blob Storage. After the song is successfully uploaded, the alert service queries the follower service to retrieve all followers of the artist, ensuring high scalability by sharding the follow-following database, and notifies each user about the new song upload. When a user requests to play a song, the play service retrieves the URL from the blob storage, fetches the song, and plays it in an MP4 player. Similarly, we will query the playlist service and follower service to add songs or follow artists. If a song reaches a certain number of plays, such as 10 million, it will be stored in cache or at a CDN, reducing the time to serve the song and ensuring low latency and high throughput.
Explain any trade offs you have made and why you made certain tech choices...
1) Using elastic search as secondary database will help user in efficient search queries
2) Using cache to store highly played song will enhance user experience
Try to discuss as many failure scenarios/bottlenecks as possible.
1) If a lot of songs have reached a threshold for which they can be stored in cache then we can create a new threshold and invalidate the existing cache and build a new cache solution
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?
1) Use machin learning solutions to provide users suggestions about the songs and genre