Loading...
List the key functional requirements for the system (Ask the AI for hints if stuck)...
List the key non-functional requirements (performance, scalability, reliability, etc.)...
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
// Applying a tag to an item
POST /tags/apply
Request:
tagId: tag to apply
itemId: the item to apply tag on
Response: 200 ok - succeeded
400 - bad request - Invalid tagId / itemId
500 - internal errors
// Get all tags
GET /tags
response:
A list of
{
tagId: id of a tag
tagName: string name of a tag to display
}
//Search based on a tag
GET /tags/items/{tagId}
response:
A list of
{
itemId: id of an item
item type: type of item
}
// Create a tag
POST /tags/create
request:
tagName: string name of a tag
response:
tagId: the tagId of the tag created
// Edit a tag
POST /tags/edit/{tagId}
request:
tagName: updated name of a tag
response:
200 ok
400 invalid request
500 internal server error
// Delete a tag, will need authentication
DELETE /tags/{tagId}
// Get suggestions for tagging an item
GET /tags/suggestions
request: itemId
response: A list of
{
tagId: id of a tag
tagName: string name of a tag
}
// Get popular tags
GET /tags/popular
response: A list of
{
tagId: id of a tag
tagName: string name of a tag
}
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
Client: The client where users use to send requests
Tagging Service: the service that handles tagging items with tags
Database: The database that contains tags and its metadata, as well as items that can be tagged.
Read Path: When a client attempts to tag an item, the client first send two requests to tagging service. The first one is Get /tags to get the list of available tags and the second is Get /tags/suggestions to get a list of suggestions of the tags. Both of these are routed to search service. The search service redirect the requests to search index.
When a client try to search based on tags, the request goes through a similar route as above.
Adding tags: Once the client decides what tag to add, it issues a POST /tags/apply to tagging service. The service will need verify from database that the tagId / itemId are valid, and update the database. The tagging service should also add a job in the search index queue. Which should be handled by the search index handler later. The job handler would update search index to include the new change.
Create / edit tags: The client should send requests to tagging service, which then calls tag management service. This service should create a tag / edit a tag by writing to database, and update cache. In addition, it should kick of a reconsiliation job by adding such a job to job queue. It should return the call.
A reconsiliation job handler should read from the job queue and handles reconsiliation process.
The reconsiliation job handler should in turn emits an event about the addition of edition of tag, and add it to search indexing queue. The search index handler should pick it up later and update the search index.
Delete tags: client should call the tagging service, which then calls the tag management service and update database. It should mark the deleted tag as deleted (instead of just deleting it from the database). A deleted tag should not be returned when an item is fetched.
Tag management system should emits an event to the search indexing queue, indicating that a tag has been deleted. This should then handled by handler and update index asynchronously.
The actual digital items are stored in an Object Store (like Amazon S3 or Google Cloud Storage), while the Primary Database only stores the item's metadata (e.g., the S3 URI, title) alongside its tag associations.
To handle spikes from trending tags or viral items, we will track tag popularity using Redis atomic counters (INCR), batch our POST /tags/apply writes to the database, and debounce/rate-limit updates to the Search Index.
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
Keeping the Search Index Consistent
To guarantee eventual consistency between the primary database and the search index without falling victim to the dual-write problem, we utilize Change Data Capture (CDC). By tailing the primary database's Write-Ahead Log using a tool like Debezium, every tag application or soft-deletion automatically publishes a reliable event to our Kafka broker. An Indexer service consumes these events and updates Elasticsearch, explicitly using the database record's updated_at timestamp for optimistic concurrency control—this ensures that even if Kafka delivers messages out of order or duplicates them, the index updates idempotently. Finally, a low-priority background reconciliation job runs nightly to compare database records against the index, automatically healing any subtle data drift caused by dead-letter queue failures or transient bugs.
The Tag Merge Flow
Merging two heavily used tags synchronously would cause massive database lock contention and widespread write failures, so we implement a two-phase asynchronous merge. In the fast phase, we instantly update the deprecated tag's database record with a merged_into_tag_id pointer; our Redis cache and Search Service immediately use this to alias and redirect user queries, making the merge appear instantaneous on the read path. In the slow phase, a background worker is queued to paginate through the underlying item-tag relationship tables, updating the old tag IDs to the new one in small, rate-limited batches. As these batch updates occur, they automatically trigger our CDC pipeline to seamlessly update the Elasticsearch documents in the background without overwhelming the system.
We should have a unique contraint of database on (item_id, tag_id), something like INSERT ... ON CONFLICT DO NOTHING. This would avoid user sending duplicate requests for tagging due to retrying and introducing duplicate records in database.
We can use ElasticSearch's completion suggestor for autocomplete suggestions.
To efficiently serve multi-tag filters, the Search Service executes Elasticsearch boolean queries, which perform rapid, low-latency intersections of inverted index posting lists rather than expensive database joins.