Functional & Non-Functional Requirements

Requirements

The system must provide essential functionalities for users to create, view, edit, and delete advertisements. Each advertisement should include fields such as a title, description, price, and category. Additionally, users should be able to browse through advertisements based on categories and perform searches based on keywords.

To enhance user experience and provide scalability, the system should also allow user authentication and authorization. A rating system for advertisements could improve trustworthiness, and an internal messaging system might facilitate communication between buyers and sellers. Required non-functional requirements include high availability, low latency, and the ability to handle millions of concurrent users.

Capacity Estimation

Estimation

To estimate the resources needed for a system like Craigslist, we can start by evaluating potential traffic. Assuming 1 million daily active users posting, viewing, and managing ads, and estimating that each user generates an average of 10 requests per session translates to about 10 million requests daily. This results in approximately 115 transactions per second (TPS). Based on industry benchmarks, a simple service can handle about 100 TPS on a single instance. Thus, we'll likely need at least 2-3 web service instances behind a load balancer.

For the database, we can estimate users and ads. If each active user has about 5 ads, we need to manage around 5 million records in a NoSQL database. Given this size, a sharded database architecture can help distribute load. We must also consider storage for user profiles and potential images for the ads. A storage estimate stands at around 1GB per 1000 ads, suggesting we may need around 5TB for our photo storage, using cloud-based storage solutions.

API Design

API Design

The following RESTful API endpoints are proposed:

• POST /ads: Create a new advertisement. Requires authentication.
• GET /ads: Retrieve a list of advertisements, with optional query parameters for filtering.
• GET /ads/{id}: Retrieve a specific advertisement by its ID.
• PUT /ads/{id}: Edit an existing advertisement. Requires authentication.
• DELETE /ads/{id}: Delete an advertisement. Requires authentication.

Furthermore, we might implement rate limiting on these endpoints (e.g., 10 requests per minute) to prevent abuse. Furthermore, authentication should leverage OAuth 2.0 to ensure user data security, coupled with JWT tokens for session management.

Database Design

Database Design

The system will employ a NoSQL database, such as MongoDB, for flexible schema management to accommodate the varying advertisement types. Each advertisement document could have fields like title, description, price, category, location, and userID. We'll also store user data in a separate collection including fields like username, email, and password_hash.

A relational database could also complement the system for managing user profiles, ratings, and transaction history. The relationships will be users to ads (one-to-many) and users to ratings (one-to-many), providing a clear mapping of interactions between entities.

High Level Design

High-Level Architecture

The high-level architecture includes a client application (web/mobile), a load balancer, and several microservices functioning behind the scenes:

• Client: Provides the user interface for posting and viewing ads.
• Load Balancer: Distributes incoming requests over multiple service instances for improved availability.
• Ad Service: Handles all advertisement-related requests (create, read, update, delete).
• User Service: Manages user authentication and profiles.
• Search Service: Optimizes querying advertisements based on user input.
• Database: Serves as the data storage platform (both NoSQL and relational).
• Cache: Redis or Memcached for improving response times by storing frequently requested data.

Request Flows

Request Flow

The request flow diagram illustrates how a typical user request would process through the system:

1. The user sends a request from the client application.
2. The load balancer redirects the request to the appropriate microservice.
3. The Ad Service interacts with the Database to fetch or modify data.
4. Responses are processed and sent back to the client via the load balancer.
5. Data caching is employed after fetching to enhance response times for repeated requests.

Detailed Component Design

Key Components

The major components in this system design include:

• Client App: A web/mobile application for user interaction.
• Load Balancer: Essential for distributing incoming HTTP requests efficiently.
• Microservices: Dedicated to various business logic functions (User, Ads, and Search Services).
• Databases: A mix of NoSQL for ads and relational for user-related transactions.
• Caching Layer: Utilizes in-memory data stores to speed up routine requests.
• Monitoring and Logging: Necessary for maintaining system health and performance tracking.

Trade-offs & Tech Choices

Trade-Offs

The major trade-offs in designing a system like Craigslist include:

• NoSQL vs. SQL: Using a NoSQL database provides flexibility and scalability but might complicate querying for complex relationships.
• Microservices vs. Monolith: Microservices enhance modularity and ease of scaling individual components, but can introduce overhead in inter-service communication and complexity.
• Cost vs. Performance: High availability systems may require more resources leading to increased operational costs versus adjusted performance targets.

Failure Scenarios & Bottlenecks

Failure Scenarios

Addressing potential failures is crucial for maintaining system reliability:

• Database Downtime: Implement data replication and failover strategies to maintain availability; caching may temporarily mitigate data access.
• Service Outage: Use health checks and auto-scaling to dynamically create or replace services in a failed state.
• Load Balancer Failure: Ensure redundancy setups so traffic can reroute to other healthy instances without user disruption.

Future Improvements

Future Improvements

Once the core functionalities are established, future enhancements could include:

• Recommendation Algorithm: Leverage user activity and AI-driven analytics to suggest ads based on user interests.
• User Ratings/Feedback: Introducing systems to improve trust via reputation scores and highlighted user feedback.
• Mobile Application: Developing dedicated mobile apps to cater to mobile-device users, ensuring a responsive design.

High Level Architecture Diagram

Database ER Diagram

Request Flow Sequence Diagram