Requirements

Functional Requirements:

• Distribute graph computation across multiple nodes using frameworks like Spark GraphX or Pregel for parallel algorithm execution.
• Support a library of graph algorithms including: pathfinding (Dijkstra, A*), community detection (Louvain), centrality (betweenness, closeness), clustering, and traversals.
• Ingest graph updates in real time via a streaming pipeline (e.g. Kafka) so analysis reflects current state.
• Support indexed lookups on node/edge properties for fast retrieval at scale using a graph database.
• Scale horizontally to accommodate billions of nodes and edges by adding machines to handle growing data.

Non-Functional Requirements:

List the key non-functional requirements (eg low latency, scalability, reliability, etc.)...

Capacity Estimation

Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...

API Design

Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...

High-Level Design

Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.

Database Design

Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...

Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.