PostgreSQL
Rails
Data-Fabric
Replication
Ruby on Rails

Postgresql replication in rails with data-fabric gem

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

PostgreSQL is a powerful, open-source object-relational database system with a strong reputation for reliability, feature robustness, and performance. In the context of Ruby on Rails applications, PostgreSQL can be utilized effectively alongside various gems and libraries to enhance the scalability and responsiveness of your web application. One such library is data-fabric, which focuses on database sharding and replication.

Overview of Data-Fabric Gem

The data-fabric gem is designed to allow Rails applications to connect to multiple databases seamlessly. It integrates with ActiveRecord, Rails' default ORM (Object-Relational Mapping) library, to provide support for multitenancy, replication, and sharding.

Some key features of data-fabric include:

  • Replication Support: Read from replicas and write to the primary database.
  • Multitenancy: Enable routing to different databases based on tenant identifier.
  • Flexible Configuration: Utilize YAML configuration files to define database roles.

Configuring PostgreSQL Replication

Replication in PostgreSQL involves copying data from a primary server to one or more replica servers. This allows the replica servers to handle read queries, which can help distribute the load and improve performance.

Basic Configuration

Ensure you have at least two PostgreSQL servers running. The primary server is where all write operations take place, while read queries can be directed to replica servers. Here’s a simple setup:

  1. On the Primary Server (postgresql.conf):
plaintext
   wal_level = replica
   max_wal_senders = 3
  1. On the Replica Server (postgresql.conf):
plaintext
   hot_standby = on
  1. Set up replication roles and permissions.

Streaming Replication Setup

  1. Create a Replication User:
sql
   CREATE ROLE replicator WITH REPLICATION LOGIN ENCRYPTED PASSWORD 'replicator_password';
  1. On the Primary Server (pg_hba.conf): Add:
plaintext
   host replication replicator 0.0.0.0/0 md5
  1. Start Replication on Replica Server:
bash
   pg_basebackup -h PRIMARY_SERVER_IP -D /var/lib/postgresql/12/main -U replicator -vP --wal-method=fetch

Integrating Data-Fabric in a Rails Application

Here's how you can manage PostgreSQL replication in a Rails application using the data-fabric gem:

Gem Installation

Add data-fabric to your Gemfile and run bundle install:

ruby
gem 'data-fabric'

Database Configuration

Create or modify config/database.yml to define different roles for your database connections:

yaml
1default: &default
2  adapter: postgresql
3  encoding: unicode
4  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
5
6development:
7  primary:
8    <<: *default
9    database: myapp_development
10    username: app_user
11    password: <%= ENV['APP_DATABASE_PASSWORD'] %>
12  replicas:
13    primary_replica:
14      <<: *default
15      database: myapp_development
16      host: replica_host_1
17      username: app_user
18      password: <%= ENV['APP_DATABASE_PASSWORD'] %>
19      
20production:
21  primary:
22    <<: *default
23    database: myapp_production
24    username: app_user
25    password: <%= ENV['APP_DATABASE_PASSWORD'] %>
26  replicas:
27    primary_replica_one:
28      <<: *default
29      database: myapp_production
30      host: replica_host_1
31      username: app_user
32      password: <%= ENV['APP_DATABASE_PASSWORD'] %>
33    primary_replica_two:
34      <<: *default
35      database: myapp_production
36      host: replica_host_2
37      username: app_user
38      password: <%= ENV['APP_DATABASE_PASSWORD'] %>

Using ActiveRecord with Data-Fabric

To ensure that the application reads from the replicas and writes to the primary, you can use the following pattern in your Rails application:

ruby
1class ApplicationController < ActionController::Base
2  around_action :swap_replica_role_if_needed
3
4  private
5
6  def swap_replica_role_if_needed(&block)
7    if request.get?
8      DataFabric.activate(:replicas, &block)
9    else
10      DataFabric.activate(:primary, &block)
11    end
12  rescue => e
13    Rails.logger.error("Replication swap failed: #{e.message}")
14    raise
15  end
16end

Error Handling and Monitoring

Monitoring and error handling are crucial for effective replication setup management. Here are some guidelines:

  • Logging: Ensure all replica activity is logged.
  • Monitoring: Set up tools like pg_stat_replication for monitoring replication lag.
  • Fallback: Plan emergency fallback to the primary database during replica failure.

Conclusion

PostgreSQL replication in Rails using data-fabric can significantly improve the performance and resilience of a Rails application. By offloading read operations to replicas and reserving write operations for the primary server, applications can handle a greater number of simultaneous requests without a performance hit.

Below is a table summarizing the key points:

Feature/AspectDescription
Replication SupportRead from replicas write to primary
MultitenancyUse different DBs for different tenants
Config FlexibilityConfigure using YAML
Setup ComplexityModerate, technical background needed
Key Commandspg_basebackup, SQL Replication roles
Error HandlingLog errors Use monitoring tools

By employing these strategies and tools, you can greatly enhance the scalability and reliability of your application.


Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.