Select row with most recent date per user
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
In data processing and analysis, it's often necessary to extract the most recent record for each user from a dataset. This task is common in scenarios such as retrieving the latest activity logs, the most recent purchase records, or the latest update timestamps. Extracting such records requires understanding both SQL and data strategy to ensure efficiency and accuracy. This article provides a comprehensive guide on how to select the row with the most recent date per user from a dataset, featuring examples and technical explanations.
Understanding the Dataset
To tackle this problem, let's consider a simple dataset representing user activities:
| User_ID | Activity | Activity_Date |
| 1 | Login | 2023-09-15 |
| 2 | Purchase | 2023-09-18 |
| 1 | Logout | 2023-09-16 |
| 3 | Login | 2023-09-15 |
| 2 | Review | 2023-09-17 |
| 3 | Logout | 2023-09-19 |
| 1 | Purchase | 2023-09-19 |
The goal is to retrieve the most recent `Activity` for each `User_ID`.
SQL Solution
To achieve the desired result using SQL, we can use various methods such as window functions or common table expressions (CTEs).
Using Window Functions
Window functions can be a robust solution as they allow us to rank records and then filter based on rank:
- The `ROW_NUMBER()` function is used to rank records within each group (`PARTITION BY User_ID`) based on `Activity_Date` (ordered by `DESC` to list recent ones first).
- The outer query filters the results to fetch only rows with `rank = 1`, representing the most recent activity for each user.
- The CTE `Recent_Activities` computes the most recent date of activity for each user.
- The result is joined back to the `activities` table to retrieve corresponding `Activity`.
Related reading
- Select rows in pandas MultiIndex DataFrame
- Selecting a row of pandas series/dataframe by integer index
- Selecting multiple columns in a Pandas dataframe
- Selecting with complex criteria from pandas.DataFrame
- Select rows from a table that are not in another
- SELECT WHERE NOT EXISTS
- Selecting/excluding sets of columns in pandas
- Sending metrics from kafka to grafana

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.