MySQL
SQL
database queries
group by
last record retrieval

Retrieving the last record in each group - MySQL

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

Retrieving the last record in each group is a common requirement when working with SQL databases. In MySQL, this can be challenging due to its syntax and functionalities. However, there are several methods to efficiently perform this task. This article will explore different approaches to retrieve the last record in each group, explain their mechanisms, and provide examples for better understanding.

Key Approaches

1. Using GROUP BY and MAX()

When the "last" record is determined by a value that represents the latest entry (such as a timestamp), you can leverage the MAX() function. The plan here is to group by the desired column(s) while selecting the maximum value from the other column.

Syntax

sql
SELECT group_column, MAX(latest_column) as last_value
FROM table
GROUP BY group_column;

Example

Consider a transactions table with columns customer_id, transaction_date, and amount. To find the latest transaction date for each customer, you would write:

sql
SELECT customer_id, MAX(transaction_date) as last_transaction_date
FROM transactions
GROUP BY customer_id;

While this gives the latest transaction date per customer, it does not include other details from the transaction. For that, further steps are required.

2. Using Subqueries

A common way to fetch additional information (besides the group key and aggregate) is to use subqueries. We can store the result of the first query in a subquery and then join it back with the original table.

Syntax

sql
1SELECT t1.*
2FROM table t1
3JOIN (
4  SELECT group_column, MAX(latest_column) as last_value
5  FROM table
6  GROUP BY group_column
7) t2 ON t1.group_column = t2.group_column AND t1.latest_column = t2.last_value;

Example

Refining the previous example to include the transaction amount:

sql
1SELECT t1.*
2FROM transactions t1
3JOIN (
4  SELECT customer_id, MAX(transaction_date) as last_transaction_date
5  FROM transactions
6  GROUP BY customer_id
7) t2 ON t1.customer_id = t2.customer_id AND t1.transaction_date = t2.last_transaction_date;

3. Using Window Functions

With the advent of MySQL 8.0, we can use window functions such as ROW_NUMBER() to solve this task more elegantly.

Syntax

sql
1SELECT group_column, other_columns
2FROM (
3  SELECT group_column, other_columns,
4         ROW_NUMBER() OVER (PARTITION BY group_column ORDER BY ordering_column DESC) as row_num
5  FROM table
6) sub
7WHERE sub.row_num = 1;

Example

Again, applying this to our transactions example:

sql
1SELECT customer_id, transaction_date, amount
2FROM (
3  SELECT customer_id, transaction_date, amount,
4         ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY transaction_date DESC) as row_num
5  FROM transactions
6) sub
7WHERE sub.row_num = 1;

Performance Considerations

  • Indexes: Ensure that the columns used in the ORDER BY clauses and joins are indexed for performance optimization.
  • Execution Plan: Use EXPLAIN to analyze and optimize any SQL query performance issues.

Additional Tips

  • DISTINCT ON: In some SQL dialects like PostgreSQL, the DISTINCT ON keyword simplifies this task, but it isn't available in MySQL, reinforcing the importance of understanding these workarounds.
  • Handling Ties: If you want to handle ties (e.g., multiple records having the same "last" timestamp), consider using RANK() or DENSE_RANK() instead of ROW_NUMBER().

Summary Table

ApproachProsCons
GROUP BY and MAX()Simple and intuitiveLimited to retrieving max value, not surrounding data
SubqueriesCan retrieve complete recordsMore complex, possible performance issues with large data sets
Window FunctionsClean and flexibleRequires MySQL 8.0 or above, learning curve

Conclusion

MySQL offers several techniques to retrieve the last record in each group, each suitable for different scenarios and capabilities. As a developer or DBA, understanding the nuances and efficiencies of each method will greatly enhance your SQL querying skills and database performance management. Always consider the specific needs and limitations of your context when choosing the most appropriate method.


Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design