AWS RDS
MySQL Replication
Data Migration
Database Sync
Cross-Instance Replication

Replicate subset of tables from AWS RDS mysql to another RDS/external mysql instance

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Replication in MySQL is a powerful feature that allows you to create copies of databases or specific tables and synchronize them between multiple instances. This is particularly useful for disaster recovery, geographical distribution of data, load balancing, and more. AWS RDS (Relational Database Service) provides managed MySQL instances, which simplify the deployment and maintenance of MySQL databases.

In this article, we'll walk through the process of replicating a subset of tables from an AWS RDS MySQL instance to another RDS MySQL instance or an external MySQL server.

Requirements

  1. AWS RDS Instance: Source RDS instance with MySQL that contains the tables to be replicated.
  2. Destination MySQL Instance: Another RDS instance or an external MySQL server where tables will be replicated.
  3. Network Configuration: Ensure that the destination instance can connect to the source instance. This might require changes in security groups and network ACLs.
  4. Replication User: Create a user with replication privileges on the source database.

Creating the Replication User

First, log in to your source RDS instance and create a user with replication privileges.

sql
CREATE USER 'replication_user'@'%' IDENTIFIED BY 'strong_password';
GRANT REPLICATION SLAVE ON *.* TO 'replication_user'@'%';
FLUSH PRIVILEGES;

Configuring the Source RDS Instance

To enable replication, you need binlogging enabled on your source RDS instance. Verify that binary logging is enabled and configure the parameters appropriately:

  1. Check Binlog Configuration:
    • Log into your RDS console.
    • Navigate to the parameter groups for your RDS instance.
    • Ensure that binlog_format is set to MIXED or ROW.
  2. Enable Binary Logging: Ensure rds.enhanced_monitoring_interval is set for real-time insights as needed.

Filtering Tables for Replication

To replicate only a subset of tables, you can use replication filters on the destination server. These can be set up by altering the MySQL configuration file (my.cnf or my.ini) on the destination instance.

Example configuration:

ini
[mysqld]
replicate-do-table=database_name.table1
replicate-do-table=database_name.table2

Full Configuration for Replication

On Source RDS:

  • Find the Binlog File and Position:
sql
  SHOW MASTER STATUS;

It will output File and Position, which you'll use to configure the slave.

On Destination MySQL Instance:

  1. Stop the Slave:
sql
   STOP SLAVE;
  1. Change Master:
sql
1   CHANGE MASTER TO 
2   MASTER_HOST='source_instance_endpoint',
3   MASTER_USER='replication_user',
4   MASTER_PASSWORD='strong_password',
5   MASTER_LOG_FILE='binlog-file-from-master',
6   MASTER_LOG_POS=log-position-from-master;
  1. Start the Slave:
sql
   START SLAVE;
  1. Verify Slave Status:
sql
   SHOW SLAVE STATUS\G;

This command checks for "Slave_IO_Running: Yes" and "Slave_SQL_Running: Yes", which confirm replication is operational.

Monitoring and Error Handling

Regularly monitor the replication performance and address errors promptly. Use:

sql
SHOW SLAVE STATUS\G;

This will provide you with detailed status, including any errors that may have occurred. Check the Last_Error field for specifics on what went wrong.

Enhancements and Best Practices

  1. Secure Replication Traffic: Use SSL to encrypt the traffic between the source and destination instances.
  2. Regular Backups: Always have regular backup protocols in place for disaster recovery.
  3. Monitoring and Alerts: Use AWS CloudWatch to monitor the RDS performance and set up alerts for critical metrics.

Summary Table

StepDescriptionCommands/Details
Create Replication UserCreate a user with appropriate privileges on the sourceCREATE USER 'replication_user'@'%' IDENTIFIED BY 'strong_password'; GRANT REPLICATION SLAVE ON *.* TO 'replication_user'@'%';
Enable BinloggingEnsure binary logging is enabled on the sourceCheck binlog_format and rds.enhanced_monitoring_interval
Configure DestinationSet up replication filters and master configurationCHANGE MASTER TO ... Modify [mysqld] section
Start/Verify ReplicationExecute commands to start and verify the replicationSTART SLAVE; SHOW SLAVE STATUS\G;

Conclusion

Replicating a subset of tables from an AWS RDS MySQL to another MySQL instance involves preparation of both the source and the destination instances. By following the steps outlined, and maintaining good practices such as secure connections and regular monitoring, you can ensure efficient and reliable replication. Always remember to regularly back up your data and maintain vigilance with replication health checks.


Course illustration
Course illustration

All Rights Reserved.