System requirements


Functional:

List functional requirements for the system (Ask the chat bot for hints if stuck.)...

  1. Store Text: The service should allow users to store plain text, formatted text (like paragraphs or quotes), and code snippets.
  2. Syntax Highlighting: Users should have the ability to format their pastes with syntax highlighting for various programming languages when submitting code blocks.
  3. Code Block Formatting: When a user pastes programming code, the service should automatically recognize and format it as a code block for better readability.
  4. Generate Random URL: Upon saving a paste, the system should generate a unique, random URL that users can share to access the paste.
  5. View Paste via URL: Anyone with the generated URL should be able to view the content of the paste, subject to the visibility settings (public or private).
  6. Set Expiration Date: Users can set an expiration date or duration for the paste. After this period, the paste should be marked for automatic deletion.
  7. Automatic Deletion: Once the expiration date is reached, the paste should be automatically deleted from the database, and the URL should become invalid.



Non-Functional:

List non-functional requirements for the system...


  1. Service Level Agreement (SLA): The system should provide users with a selection of paste expiration times, including options for 5, 10, 30, 60, and 90 days.
  2. System Scalability: The service should be designed to handle a burst capacity of at least 10,000 concurrent users. The system should have mechanisms in place to monitor usage and trigger alerts when nearing capacity limits.
  3. Alerting Mechanism: An alerting system should notify the development team when the user count approaches the 10,000 threshold, prompting them to consider increasing horizontal scaling capacity (e.g., to 20,000 or 30,000 users).
  4. User Feedback during Overload: When the system is at capacity or under heavy load, it should provide users with a clear message, such as "Please try again later, the service is currently overloaded."


Capacity estimation

Estimate the scale of the system you are going to design...


the capacity design we can use horizontal scaling so we can keep on adding new servers if the limit exceeds. So, whenever the limit exceeds the server, whenever the user count limits the server limit. So, we can notify it by cutting as severity to the team to add new horizontal servers. And why horizontal servers so that we are able to maintain the existing server without disturbing them. Now, in the event of any catastrophe, if the user is only situated in a single country, then we can put in many states of the country. And if there are multiple countries, we can put in multiple countries. And whoever is and those, wherever the server is kept in that country, the closest it will it should be able to connect to the closest user. So, yeah, and if it is in different countries, if this pastebin is in different countries, we can put in also different countries.

  1. Horizontal Scaling:
    • The system should be designed to support horizontal scaling, allowing additional servers to be added seamlessly to manage increased user load. This ensures high availability and performance without disrupting the existing infrastructure.
  2. User Limit Notification:
    • Implement a monitoring system that tracks user capacity. When the number of concurrent users approaches a defined limit (e.g., 10,000 users), an alert should be generated to notify the engineering team to provision additional servers.
  3. Geographical Distribution:
    • To improve latency and user experience, the service should consider deploying servers in multiple geographic locations. This can include multiple states within a single country or across various countries to ensure that users connect to the nearest server.
  4. Disaster Recovery:
    • In case of server outages, data loss, or other catastrophes, the system should have a disaster recovery plan. This could involve redundancy, such as backup servers in different regions, and data replication across servers to ensure business continuity.
  5. Load Balancing:
    • Use load balancers to distribute incoming traffic across multiple servers, ensuring that no single server becomes a bottleneck.



API design

Define what APIs are expected from the system...

For API design we will be using mainly POST and GET method. POST method is used when we click the save button and we will be checking if there is any duplicate present in the database or not, we will be generating a random URL and putting it on a database if it is not present else we keep on generating until it is not present. If the random url is not present then we will map it to the database as key and the body contains the text and a unique user ID which will be again random. So the main key is the random URL that we are generating. So it should have an expiration date which is set by the user which will be 5, 10, 30 or 60 days and after that the data should automatically get deleted. Now for a GET method, the user has to have the URL and then it should have a look at the database and if the data is already deleted we should be able to give them a no found message else the text should be visible in the UI. If the data is not there, this would be handled in the user interface. Yeah so and about the delete and update method we don't need it because we are not going to allow the user to update anything after it is created and we are not going to allow the user to delete, it will be automatically deleted.


API Endpoints

  1. Create Paste (POST /api/pastes)
    • Description: Creates a new paste with the provided text and generates a unique URL.
    • Request Body:
    • { "userId": "uniqueUserId", "text": "This is the content of the paste.", "expiration": "30" // Could be 5, 10, 30, 60, or 90 days}
    • Process:
      • Check for duplicate URLs and generate a new random URL if necessary.
      • Store the paste in the database with the generated URL as the key.
    • Response:
      • Success (201):
    • { "url": "https://pastebin.com/example123", "message": "Paste successfully created."}
      • Error (409 - Conflict): If unable to generate a unique URL after multiple attempts.
    • { "error": "Unable to generate a unique URL. Please try again."}


  1. Retrieve Paste (GET /api/pastes/{url})
    • Description: Retrieves the content of a paste using its unique URL.
    • Request Parameters:
      • url: The unique URL identifier for the paste (as part of the endpoint).
    • Response:
      • Success (200):
    • { "url": "https://pastebin.com/example123", "text": "This is the content of the paste."}
      • Error (404 - Not Found): If the paste does not exist or has expired.
    • { "error": "Paste not found or has been deleted."}

Notes

  • Update and Delete Operations: As specified, there is no separate API for updates or deletions since the pasted content has an expiration date and will be automatically removed.



Database design

Defining the system data model early on will clarify how data will flow among different components of the system. Also you could draw an ER diagram using the diagramming tool to enhance your design...






High-level design

You should identify enough components that are needed to solve the actual problem from end to end. Also remember to draw a block diagram using the diagramming tool to augment your design. If you are unfamiliar with the tool, you can simply describe your design to the chat bot and ask it to generate a starter diagram for you to modify...






Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...






Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...






Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...






Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.






Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?