Requirements

Functional Requirements:

List the key functional requirements for the system (Ask the AI for hints if stuck)...

1. Create a tag
2. Assign a tag to a content
3. Search items by tags
4. Edit & Delete a tag
5. Suggest a tag for a content
6. Tags Normalization

Non-Functional Requirements:

List the key non-functional requirements (performance, scalability, reliability, etc.)...

1. High read compared to right for tags. System needs to handle millions of users and billions of tag-item association.
2. Eventual Consistance over performance

Capacity Estimation

Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...

1. High association between tags and content . Millions per day.
2. Tag creation - 100s/day.
3. Very high read to write ration for tags. 1/10000
4. High tags associtaion read to write ration. 1/100

API Design

Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...

POST /tags/create

body { tag_name, tag_description}

assign tag to a contnet

POST /tags/{tag_id}/content/{content_id}

GET /contents/tags/{tag_id}

POST /tags/update/{tag_id}

GET /content/suggests/tags

High-Level Design

Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.

Load Balancer, directs the load and implements rate limiter to block the bots.

API Gateway redirects the routes to read and write service to scale them independently. It also implements OAuth/JWT for authentication/authorization.

Read Server - Has all the read api implementaton "GET /contents/tags/{tag_id}" "GET /content/suggests/tags" etc.

For search service, it uses elasticSearch

Write Server - Implements all the write operations "POST /tags/create" "POST /tags/{tag_id}/content/{content_id}" "POST /tags/update/{tag_id}"

Suggestion Service - It uses web crawlers and search for the keywords based on which it provides suggestions for a content.

Redis Cache - Read Server hits redis cache first and the TTL will be 0 everytime there is a write operation, default TTL, will be 10 mins.

Database, we will use DynamoDB for high scale reads.

Database Design

Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...

{

PK: tag#tag_id

SK: metadata

creator:

updator:

createdAt:

}

{

PK: item#itemId

SK: tag_id+itemId

creator:

updator:

createdAt:

tags: [{tag_id1}]

}

{

PK: alias_name

SK: tag_id

aliases :["name1", "name2"]

}

Dynamodb selection to handle high read

Detailed Component Design

Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.

1. Create a tag -> Users after authentication/authorization (API Gateway) can create a Tag (Write Server). Before a tag is created, normalization filters out if there is an already similar tags created (This check relies on third party libraries).
2. Assign a tag to a content -> Users can open a content and they will get an option to assign the tags to that content with the help of suggestion_service response.
3. Search items by tags -> User uses ElasticSearch to search a content based on tags.
4. Edit & Delete a tag -> Authenticated and Authorized user can only edit or delete a tag. On tag deletion, there is a cleanup service which will clean up the existing tags from the db/cache

For idempotent tag assignment, we will create {tag_id+itemId} as Sort Key.

Since the Elastic search is using SQS, if it goes down, the SQS will store the messages for 14 days and once elasticsearch comes back up, the messages will be processed, meanwhile, we will disable the search functionality till it is down.