MongoDB
Sharded Cluster
Database Tools
Cluster Management
Data Sharding

Tool to create mongodb sharded cluster

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

MongoDB, a popular NoSQL database, is known for its high performance, high availability, and easy scalability. In larger applications, where data grows to a substantial size, setting up a MongoDB sharded cluster becomes essential. This type of cluster helps in distributing the data across multiple machines and managing the database's load effectively.

Understanding MongoDB Sharding

Sharding is a method for distributing data across multiple servers, where each shard holds a portion of the data and acts as an independent database. The overall cluster's operations are managed by query routers and configuration servers.

Components of a MongoDB Sharded Cluster

  1. Shards: Each shard contains a subset of the sharded data. Each shard can be deployed as a replica set for fault tolerance.
  2. Config Servers: These store metadata and configuration information about the cluster. Typically, a production cluster has three config servers.
  3. Query Routers (mongos instances): These act as the interface for client applications. They direct operations to the appropriate shards and return results to the clients.

Steps to Create a MongoDB Sharded Cluster

1. Set Up Config Servers

First, initiate the configuration servers. You typically need to start 3 config servers for redundancy.

bash
mkdir -p /data/configdb
mongod --configsvr --replSet configReplSet --dbpath /data/configdb --port 27019

Next, initiate the replica set for these configuration servers.

bash
1mongo --port 27019
2rs.initiate({
3  _id: "configReplSet",
4  configsvr: true,
5  members: [ { _id: 0, host: "configsvr1:27019" },
6             { _id: 1, host: "configsvr2:27019" },
7             { _id: 2, host: "configsvr3:27019" } ]
8})

2. Start the Shard Servers

Each shard in the cluster is usually a replica set. Start mongod instances for each member of the replica set.

bash
mkdir -p /data/shard1
mongod --shardsvr --replSet shard1 --dbpath /data/shard1 --port 27018

You will also need to initiate each shard as a replica set.

bash
1mongo --port 27018
2rs.initiate({
3  _id: "shard1",
4  members: [{ _id: 0, host: "shard1a:27018" },
5            { _id: 1, host: "shard1b:27018" }]
6})

3. Start the mongos Instances

The mongos instances route queries to the correct shards and aggregate the results.

bash
mongos --configdb configReplSet/configsvr1:27019,configsvr2:27019,configsvr3:27019 --port 27017

4. Add Shards to the Cluster

Once the config servers and shards are operational, connect to one of the mongos instances and add the shards.

bash
mongo --port 27017
sh.addShard("shard1/shard1a:27018,shard1b:27018")

Key Considerations

  • Shard Key Selection: It is crucial to choose an appropriate shard key to ensure the even distribution of data and minimize hotspots.
  • Redundancy and Fault Tolerance: Each shard can be a replica set to ensure data redundancy and high availability.
  • Monitoring and Maintenance: Regular monitoring is essential for checking the health and performance of a sharded cluster.

Summary Table of Key Commands

ActionCommand
Start Config Servermongod --configsvr --dbpath /data/configdb --port 27019
Initiate Config Server RSrs.initiate({...})
Start Shard Servermongod --shardsvr --dbpath /data/shard1 --port 27018
Initiate Shard RSrs.initiate({...})
Start mongos Instancemongos --configdb configReplSet/... --port 27017
Add Shardsh.addShard("shard1/...")

Deploying a MongoDB sharded cluster involves multiple components and steps, but it substantially increases the database’s ability to handle large-scale operations efficiently. After setup, ongoing maintenance and optimization of the cluster will further enhance its performance.


Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.