Design Sora - System Design

Functional Requirements

1. users send prompts -> get video some time later

2. users can choose length, quality

3. users can poll for status of video generation

4. scale: XXM DAU

Non-functional Reqs.

1. workflow progress should be persisted to protect against failures

2. aim to not so high latency

3. availability >> consistency, since users will want to continue using service as much as possible

4. system should be fair for each tenant (not favors some tenants)

5. output safe (needs to run several safety checks before returning to user)

6. should be efficient to minimize costs

Capacity Estimation

Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...

Entities:

1. Users: id (pk), name, email, info

2. Request: id (pk), orig_user (fk), timestamp, prompt, length, quality, status, rw_prompt

API:

REST for users

POST /requests/ -> request_id, status

@user JWT

Request {

description,

length,

quality

}

GET /requests/{id} -> request status, and video info

@users JWT

-> vide_info: {list of BLOB storage URLs to download video chunks}

Database Design

Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...

What we do. The second requirement, let's see. Here, I guess users can choose length and maybe quality of the video. Let's say this. Length... is specific... Users can pull status of the video, generation. Okay. Let me see if there's anything else. Okay, so I think for now there's a question about, I guess, scale here, but do you want to assume a certain scale, or can we just leave this to later? Like what do you mean scaling? Right now you can design based on for to solve the system and also should able to flexible to scaling. Okay, okay. So let me just put it access me and maybe active users for now. In terms of functional requirements, let's see what we can list here. So, okay, so this is a long... I think the first thing that comes to mind is that this is a long process, right? So we have steps on this workflow here. So you definitely don't want to lose the progress you make, right? So let's say the workflow progress should be, I guess, let me put it this way, should be persisted, persistent to avoid, to protect against fakers. Users are already aware that it will take sometimes generate things, but I guess, so we probably want to have aim, I guess, to not so high latency. Three. Let me see here. So availability of the system, consistency. I think the availability sort of starts upon here, but let's also make sure that we're going to be favoring availability over consistency here, since users will want to continue using the service as much as possible. Let's see if there's anything else that I can aim for here. Well, I think... Oh yeah, across a pool of expensive accelerators, well, for per tenant cost, but fairness and... Okay, so fairness, fairness, per tenant fairness, I guess, system should be fair for each tenant, not that it, I guess, favors some tenants. Five is Open safe, Open safe. So needs to check, needs to run several safety checks before returning. Okay. And finally, oh wow, what else here? The budget, right? So should be efficient to minimize cost, minimize costs here, I guess. Okay, so these are what I thought of could be some interesting requirements and functional requirements. I think on top of this, I think we can discuss a little bit the entities that we're going to be dealing with, right? So first thing, I guess goes without saying, but let's say it anyways, would be users, name, email, etc. So maintain. I assume user creation page and all those exist already? Okay. So users move there somewhere where it will be done. I guess request, which also has some user, some IDs, some originating user as a primary key, some, I guess, timestamp. Not sure if we're going to be needing this for now, but description of the request itself. What else did we say? The length and quality of the video. And okay, and I'm going to put it down here, status of the request as well. Okay, so I think for now these are the main entities that I think about. There might be some that end up emerging as we work through the exercise, but I think this makes somewhat sense. Okay. So in terms of APIs, I think I'm going to think about the REST APIs from the user perspective, REST APIs for users. Thinking here. First API is the, oh, sorry, let me, post for requests. And then it returns. So this API here, since this is asynchronous, it will return some info on the request ID. Let's put it this way, request ID and status. This also I'll be a little detailed here, but probably will omit this later, but let's assume as a user, general websocket as well along with this. And what else comes in this request? It comes with the timestamp, perhaps not really needed, but comes with description, length, quality. Finally, then there is the get request for get requests. This is to the request status user, let me see here. Finally, there is a... user can pull for status, which is... And then there is a sort of explicit get video videos here with the ID. But actually, you know what? I will return this as the following. I'll say that this returns the request status and video info. So let me be more explicit here on what the video info here means. Video info is essentially, because I'm already assuming this is going to be a problem, I guess, for the future, and it's the simple. So this is going to be a list of blob storage URLs to download the video chunks. Okay. So this, I think, makes some sense so far. But having said this, people can step into the actual design of the system. Does that sound good to you? Yeah. Okay, so here's what I'm... I don't know why this is... Delete this. Okay, so let's at least have some three guides. So this is the client. I'm going to assume this is the API gateway layer here. I'll also just briefly mention this does authentication, routing, etc., load balancing, etc. Let me make this a bit bigger here.