MySQL
master-master replication
unique indexes
database replication
MySQL tutorial

How to handle unique indexes with MySQL master master replication

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

Master-master replication in MySQL, also known as active-active replication, is a setup where two or more servers operate as masters and replicate to each other. This setup provides high availability and allows for read/write access on all nodes. However, managing unique indexes in such a setup introduces challenges that need careful planning and configuration to ensure data consistency and avoid conflicts.

Challenge: Unique Indexes in Master-Master Replication

A unique index in MySQL ensures that all values in a column are distinct within a table. In a master-master replication scenario, if unique indexes are not handled properly, data inconsistency and conflicts can arise. Consider a situation where the same unique key is inserted simultaneously on both nodes. Without proper handling, this would lead to replication errors and possibly data conflicts.

Potential Problems:

  • Duplicate Key Errors: Both servers may generate the same unique key, causing conflicts.
  • Replication Conflicts: Data integrity may be compromised if unique constraints are violated.
  • Inconsistent Data: One node may succeed in a unique key insertion, while the other fails, leading to data discrepancies.

Strategies for Handling Unique Indexes

To handle unique indexes in a master-master replication setup effectively, consider the following strategies:

1. Auto-Increment with Offset and Increment

MySQL allows tuning of auto-increment settings using the auto_increment_increment and auto_increment_offset system variables. This can be used to ensure that auto-generated unique values are distinct across nodes.

  • auto_increment_increment: Sets the step value between successive auto-increment values.
  • auto_increment_offset: Sets the starting point of the sequence.

For example, if you have two nodes, you can configure them as follows:

  • Node A:
sql
  SET GLOBAL auto_increment_increment = 2;
  SET GLOBAL auto_increment_offset = 1;
  • Node B:
sql
  SET GLOBAL auto_increment_increment = 2;
  SET GLOBAL auto_increment_offset = 2;

By configuring this way, Node A generates odd numbers (1, 3, 5, ...) while Node B generates even numbers (2, 4, 6, ...).

2. Use UUIDs for Unique Fields

Another approach is using UUIDs (Universally Unique Identifiers) as primary keys or unique columns. UUIDs are inherently unique and can be generated using the UUID() function in MySQL:

sql
INSERT INTO my_table (id, name) VALUES (UUID(), 'Sample Name');

3. Conflict Avoidance with Application Logic

Implementing additional application logic can prevent conflicts by checking for existing keys before inserting or updating records. This can include:

  • Collision Detection: Check for the presence of keys before inserting.
  • Retry Logic: Implement logic to retry transactions if a conflict is detected.

4. Row-based Replication

Instead of statement-based replication, use row-based replication, which can reduce conflicts with unique indexes because it replicates exact row changes.

To enable row-based replication, modify the MySQL configuration file (usually my.cnf):

ini
[mysqld]
binlog-format = ROW

5. Conflict Resolution Strategies

In scenarios where conflicts arise, establish a resolution strategy, such as:

  • Last Update Wins: The most recent update takes precedence.
  • Merge Logic: Custom logic to merge conflicting changes.

Summary Table

StrategyDescriptionProsCons
Auto-Increment with OffsetAdjusts auto-increment settings to prevent key collisionsEasy to implementLimited to numerical keys
UUIDs for Unique FieldsUses UUIDs as unique keys to guarantee uniquenessEnsures unique keysLarger index size
Application LogicIncorporates checks and retry logic to avoid and resolve conflictsFlexibleIncreases application complexity
Row-based ReplicationUses row-based replication to minimize replication conflictsReduces conflict chancesMay increase replication size
Conflict Resolution StrategiesEstablishes rules for resolving conflicts when they occurAllows custom conflict managementInvolves post-conflict handling

Additional Considerations

  • Bi-directional Replication: Ensure bi-directional replication is set up correctly by verifying replicate-same-server-id=0 and that server ids are unique.
  • Network Latency: Consider the network latency between nodes, as high latency can increase the likelihood of conflicts.

Conclusion

Handling unique indexes in a master-master replication setup requires careful configuration and planning. By utilizing techniques such as adjusting auto-increment settings, using UUIDs, or implementing application logic for conflict avoidance, you can maintain data consistency and reliability. Always test your configurations in a staging environment to ensure that they work correctly before deploying them in a production setting.


Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.