Efficient way to compute number of hits to a server within the last minute, in real time
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
To count hits from the last minute in real time, you need a sliding-window data structure. The goal is to answer "how many hits happened in the last 60 seconds?" quickly while new hits keep arriving continuously.
The simplest correct approach: a queue of timestamps
A basic solution stores hit timestamps in a queue. Every time a hit arrives, append its timestamp. Every time you query, remove timestamps older than 60 seconds from the front.
This works well when the hit volume is moderate and you genuinely need per-hit precision.
Why this works
The queue always contains only timestamps in the active 60-second window. That means len(queue) is the answer. Insertions are cheap, and each timestamp is removed only once, so the amortized cost is excellent.
The key idea is that the window moves forward in time and old entries never come back. That is what makes the sliding-window approach efficient.
When a queue is not enough
If the traffic is extremely high, storing one entry per hit can become too expensive. In that case, bucket the counts by second instead of by request.
A common design is a fixed-size ring buffer of 60 slots, where each slot stores:
- the second it represents
- the number of hits seen in that second
When a new hit arrives, map it to timestamp % 60. If the stored second is stale, overwrite it with the new second and reset its count. Then increment the count.
This reduces memory drastically because you store at most 60 buckets instead of one record per hit.
Trade-offs between exactness and efficiency
The timestamp queue is exact and conceptually simple. The ring buffer is far more memory-efficient at high throughput, but it assumes second-level time buckets are acceptable.
That is often a good trade. Most operational dashboards and rate-limit decisions do not need millisecond-perfect window boundaries. They need fast and stable counts.
Distributed systems add another problem
If traffic is spread across multiple servers, each server can maintain its own local counter, but the global answer requires aggregation. At that point the local sliding-window algorithm is only part of the solution. You also need a system design for merging counts across instances, such as Redis, stream processing, or shard-aware metrics.
So the right algorithm depends on the scale boundary you care about: one process, one machine, or the whole fleet.
Common Pitfalls
- Recomputing the count by scanning the entire event history on every query.
- Storing one timestamp per hit even when traffic volume makes bucket aggregation more appropriate.
- Forgetting to evict stale entries, which makes counts drift upward forever.
- Confusing a single-node in-memory solution with a multi-server global analytics design.
- Overengineering millisecond precision when second-level buckets are operationally enough.
Summary
- The right abstraction is a sliding window over the last 60 seconds.
- A queue of timestamps is the simplest exact single-process solution.
- A fixed 60-slot ring buffer is often better for very high hit rates.
- Distributed counting requires aggregation beyond the local data structure.
- Choose the design based on throughput, precision needs, and deployment scale.
Related reading
- Elastic Beanstalk Change ELB Type
- Elastic Beanstalk Ruby/Rails need to install git so bundle install works.. but is not
- Elastic Beanstalk without Elastic Load Balancer
- Elastic IP on application deployed using Elastic Beanstalk
- Efficient way to filter out elements from stdvector
- Efficient way to find degrees of separation between two nodes in a graph
- Efficient Way to Find Pair Orderings?
- Efficient way to insert a number into a sorted array of numbers?

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.