Design a Time-Windowed GPU Credit Ledger

Question

Design a system for managing GPU credits that includes granting credits with validity windows, consuming credits at specific times, and querying the balance at any given time.

Codemia · Accepted Answer

The GPU credit ledger system should support the following functionalities:
1. **Grant Credit**: Allow users to grant a certain number of credits with a specified start and end time.
2. **Consume Credit**: Enable users to consume credits at a specific point in time, ensuring that only valid credits are deducted.
3. **Get Balance**: Provide a way to query the available credits at any given time, factoring in the validity of the credits.

The system can be designed using a combination of a database and in-memory data structures. The main components include:
- **Credit Store**: A database table to store granted credits with fields for user ID, amount, start time, and end time.
- **In-Memory Cache**: An in-memory structure to quickly access active credits, allowing for faster balance queries.
- **API Layer**: RESTful APIs for granting, consuming, and querying credits, ensuring that operations are atomic and thread-safe.

To manage time-windowed credits, we can implement the following logic:
- When granting credits, insert the record into the credit store with its validity window.
- For consuming credits, check the current time against the start and end time of the credits in the store. If valid, deduct the amount from the user's balance.
- For querying balance, iterate through the credits in the store, summing up the amounts that are currently valid based on the provided time.

One trade-off to consider is between consistency and availability. Using an in-memory cache improves performance but may introduce stale data if not synchronized properly with the database. To scale, we can partition the credit store by user ID, allowing for horizontal scaling as the number of users grows. Additionally, implementing background jobs to clean up expired credits can help maintain performance.

In summary, the GPU credit ledger system requires careful consideration of time-sensitive operations and data management. By utilizing a combination of database storage and in-memory caching, along with a clear API structure, we can create an efficient and scalable system that meets user needs.

Design a Time-Windowed GPU Credit Ledger

OpenAI

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Sharpen Your Skills on Codemia

Sample Answer

Requirements

High-Level Design

Submit Your Answer

OpenAI Software Engineer Interview Guide

Related Questions

Design a scalable Search System

Design a Video Streaming for Rippling

Design a Load Balancing Service

Design a Image Processing Service

Build a high-throughput Image Processing Pipeline