Design Sora

Design a service that accepts a text prompt and produces a short-form video (5 to 20 seconds, 720p to 1080p). Unlike LLM inference, each request runs for 30 seconds to several minutes on a GPU, so the system is async job shape: clients submit, poll for status, and fetch the result via signed URL when done. The core challenge is running a multi-stage diffusion pipeline (prompt rewrite, keyframe generation, video diffusion, super-resolution, safety classification, encoding) across a pool of expensive accelerators while enforcing per-tenant cost budgets, fairness, and output safety.

The hard parts are step-batching across concurrent jobs on the diffusion model for GPU efficiency, a safety gate that samples output frames and blocks policy violations before they ship, cost-tier routing (fast distilled model for easy prompts, HQ model for hard ones), cancellation with partial refund mid-generation, and graceful degradation under spike traffic where admission control shifts load onto cheaper tiers rather than blowing queues. Asked at OpenAI, Runway, Pika, Luma, Google (Veo), and any company building generative-media products.