Design a Context-Retrieval System for LLM Prompts
Last updated: May 21, 2025
Quick Overview
Design a system that assembles the optimal context for LLM prompts in a code editor. Given a user query or editing action, retrieve the most relevant code, documentation, and conversation history within a fixed token budget.
Cursor
May 21, 20255
11
2,314 solved
Design a system that assembles the optimal context for LLM prompts in a code editor. Given a user query or editing action, retrieve the most relevant code, documentation, and conversation history within a fixed token budget.
The quality of AI code generation depends heavily on the context provided in the prompt. This question tests your ability to design a retrieval and ranking system that selects the most relevant information from a codebase within strict token limits. The interviewer expects discussion of retrieval strategies, ranking algorithms, and token budget management.
What the Interviewer Expects
- Design a multi-source retrieval pipeline covering code, docs, types, and conversation history
- Implement a ranking algorithm that balances relevance, recency, and proximity
- Handle token budget allocation across different context sources
- Discuss how context quality affects downstream generation quality
- Address performance requirements for real-time context assembly
Key Topics to Cover
How to Approach This
- Start by clarifying functional and non-functional requirements with the interviewer.
- Estimate the scale: QPS, storage, bandwidth. This drives your design decisions.
- Draw a high-level architecture first, then deep dive into 1-2 critical components.
- Discuss trade-offs explicitly (e.g., consistency vs availability, SQL vs NoSQL).
- Address failure scenarios, monitoring, and how the system handles 10x traffic spikes.
Possible Follow-up Questions
- How would you evaluate whether your context retrieval is actually improving generation quality?
- How would you handle a user working on a new file with no project context?
- How would you adapt context retrieval for different LLM providers with different context window sizes?
Sharpen Your Skills on Codemia
Practice similar problems with our interactive workspace, get AI feedback, and track your progress.
Practice System Design ProblemsSample Answer
Requirements
Functional Requirements
- Multi-source Retrieval: The system must retrieve relevant code, documentation, and conversation history from a variety of sources including a code repository, docume...
Capacity Estimation
To estimate the capacity:
- User Base: Assume 10,000 concurrent users.
- Queries Per User: Estimate each user will generate an average of 5 queries per minute.
- *Total Queries Per Minute...
Submit Your Answer
