Loading...
Initial Songs: 3.28 mb per song * 30 million = 30 tb
New songs a day: 12 gb
50 million Playlist * 20 songs = 1billion * 126 * 20, 256gb
playlist {
links: [link: 126b]
}
10million * user data - 500GB of data
user data {
name: 20b
email 256b
hashPass: 256b
}
Every header will require a JWT token and the song will be encrypted when sent across the network
search(query) {
data: {
songs
musicians
playlist
}
}
play(songId, chunk) {
data[]
}
playlist() {
songs[]
}
addToPlayList(songId, playlistId) {
success
}
rateSong(rating) {
success
}
All of these DBs will use ocnsistent hashing to ensure even load.
User DB: postgres to enable enhanced user queries, share by user id
Song DB: MongoDB, song will be encrypted. This protects the copyright incase the system is compromised. Mongo is a good choice because the data is ambiguous. Shard by song ID
Song Metadata DB: postgres to enable enhanced user queries
This will enable us to search for songs, playlist, and musicians quicker. When a song is upload to the site it gets added to the SongDB and Song MetaData DB. Shard by songID
Playlist DB: postgress, enables relation querring for other users playlists. Will contain the song IDs and a user ID. shard py playlist id
api gateway for routing requests and rate limiting traffic. We can also limit the user to playing one song at a time.
CDN, for popular songs. Songs in the CDN will be kept based on LFU. Different locations will have different CDN content, this will be based on a user's country.
Load balancer, with the least connections strategy to split traffic and reduce load
Song Service
The service will partition the song into 10-second increments. Send the data to the client.
Search Service
A will read the ranked trie from the song search db. Each node will be a word. The service will return the best matches with the most frequency if the search term is more vague.
Playlist Service
Updated the playlist DB with the songs to be added to a specific playlist.
ELK
Elasticsearh, logstash, kibana will monitor the logs from the search to create a cache snapshot for the search service.
Request goest to the api gate way where we decide either update/get a playlist, search for a song, or play a song.
The songs will always try to load from the CDN of they exist there. Otherwise we go to the DB
Search Service
A will read the ranked trie from the song search db. Each node will be a word. The service will return the best matches with the most frequency if the search term is more vague.
API Gateway will limit the number of apis that can be called by a user. The most will be 1 play, and the 15 requests a minute for the rest. We only want the user to be able to listen to one song at a time on one device.
Postgress was used to enable better querying for the song metadata as well as ACID compliance. However it will be more difficult to scale.
The search service cache may have data not relevant to other regions of the world. The most popular region will poison the cache.
Playlist created for you weekly based on songs that you rate highly.
Follow a user and the playlist they make
Better monitoring using open