Design a Streaming LLM Edit Application System
Last updated: May 21, 2025
Quick Overview
Design a system that applies LLM-generated code edits to files as tokens stream in. Handle partial edits, syntax validation, user interruption, and rollback when the generation goes wrong.
Cursor
May 21, 202513
9
2,402 solved
Design a system that applies LLM-generated code edits to files as tokens stream in. Handle partial edits, syntax validation, user interruption, and rollback when the generation goes wrong.
Cursor applies AI edits in real time as the model generates tokens. This creates unique challenges around partial state, user experience during streaming, and error recovery. The interviewer expects you to think about the UX of streaming edits as much as the technical infrastructure.
What the Interviewer Expects
- Design the streaming pipeline from model output to editor state updates
- Handle partial edits that are syntactically invalid mid-stream
- Implement user interruption and rollback to pre-edit state
- Discuss diff visualization during streaming for user review
- Address error handling when the model produces invalid or harmful edits
Key Topics to Cover
How to Approach This
- Start by clarifying functional and non-functional requirements with the interviewer.
- Estimate the scale: QPS, storage, bandwidth. This drives your design decisions.
- Draw a high-level architecture first, then deep dive into 1-2 critical components.
- Discuss trade-offs explicitly (e.g., consistency vs availability, SQL vs NoSQL).
- Address failure scenarios, monitoring, and how the system handles 10x traffic spikes.
Possible Follow-up Questions
- How would you handle a streaming edit that modifies a file another user is editing?
- How would you implement 'accept partial' where the user keeps some generated lines but not others?
- What happens if the connection drops mid-stream?
Sharpen Your Skills on Codemia
Practice similar problems with our interactive workspace, get AI feedback, and track your progress.
Practice System Design ProblemsSample Answer
Requirements
- Functional Requirements:
- Stream LLM-generated code edits in real-time as tokens are received.
- Apply edits to the editor state with support for partial edits.
- Validate syntax...
Capacity Estimation
- User Load:
Assuming 10,000 concurrent users, with each user generating approximately 5 tokens per second on average. - Data Volume:
10,000 users * 5 tokens/second = 50,000 tokens/s...
Submit Your Answer
