High-level design

Upload the video ---> API_gateway --> Video_Downloader(s3) --> processor --> S3

* API_gateway-> It helps in authN/ authZ and throtlling of the service

Video Downloader -> It downloads the video to a particular S3 location makes a call to processor to act upon(async fashion, like via aws lambda or something equivalent)

Processor -> The processor will act upon the uploaded video, it converts the video to a more compatible format(whether for speed or storage, depending upon we want low latency or save on storage or some trade-off in between), segregate the video into chunks of 10 sec's each, and update the s3 locations with chunks.

During this, the video_downloader will also update the video table as such, it will have statuses and update those like:

* video_downloader-> before starting, makes status as pending

-> then, uploading

-> then, when processor is acting upon, it makes a call to downloader to update the status to processing

-> then, when processor is done, it makes a call to dowloader to update the status to active

For live-updates on the status, it can open web-socket with the customer's client or we can simply rely on push operation(whenever there is status change), as there aren't much states for the status, push approach is also fine as web-sockets will keep the connection open.

Play the video ---> API_gateway --> Video Feeder --> S3

* The video-feeder will search in the video table to get all the chunks corresponding to the video_id, to fasten up, it can leverage redis to keep the chunk-details for a video_id for subsequent fetches. It takes upon the current timestamp and fetches the video for the particular timstamp from S3 location and then responds back to the customer.

On every t-2th second, the browser, or some monitoring service(client-side service), will make a call the playTheVideo for the next t+1th chunk.

* Search for video-> Api_gateway --> Feed Service(can leverage a cache, redis) --> Ranking Service

|

---> ElastiSearch

we can leverage our Elasticsearch stack to search for a particular keyword provided by the customer

it would be like, for a particular search, the feed-service first of check its cache, if not found, it makes a call to elasticSearch to get the matched records(video_ids)

Then, it leverages the ranking_service to rank these video_ids and provide top: 20 records(based upon some pre-defined criteria), On getting back the top:20 items back from ranking_service, feed service can now responds backs to the customer with the response.. Ranking service can leverage something like heap to get the top 20 items from a list of videos_ids

Since, we want faster latency for read, search operations. we can categorize our backend servers and use L7 LB to segregate read/write/search requests.

Submissions

Displaying Solutions with Scores Ranging from 1 to 10