Replica Sets and Sharding Intro

Introduction

Production MongoDB runs as a replica set—multiple servers that replicate data for high availability and optional read scaling. At very large scale, sharded clusters split data across shards by a shard key. This chapter explains primary/secondary roles, oplog, read/write concern, local dev replica set setup, and sharding concepts without full production ops depth.

Prerequisites

Schema Validation and Transactions — transactions require replica set
Insert and Import Export — backup with mongodump
Basic networking and ports (27017)

Why Not a Single Standalone Server?

Standalone	Replica set
Single point of failure	Automatic failover to secondary
No transactions (modern)	Multi-doc transactions
Manual backup only	Oplog for replication + PITR on Atlas
Dev-only shortcut	Production default

Replica Set Architecture

text

        ┌─────────────┐
        │   Primary   │  ← writes default here
        └──────┬──────┘
               │ replication (oplog)
     ┌─────────┴─────────┐
     ▼                   ▼
┌──────────┐       ┌──────────┐
│Secondary │       │Secondary │  ← can serve reads (optional)
└──────────┘       └──────────┘

Optional arbiter — votes in election only, holds no data (avoid in production if possible; prefer 3 data-bearing nodes).

Roles

Role	Accepts writes	Default reads	Notes
Primary	Yes	Yes	One per set
Secondary	No	Yes (if configured)	Can become primary
Arbiter	No	No	Election tie-breaker only

Oplog (Operations Log)

Secondary members tail the oplog—a capped collection of recent writes on the primary—to stay in sync. Replication lag is the delay between primary write and secondary apply.

Check status:

javascript

rs.status()
rs.printReplicationInfo()
rs.printSecondaryReplicationInfo()

High lag: network, disk, heavy writes on primary, or secondary doing long tasks.

Read Concern and Write Concern

Write concern — when write is acknowledged:

javascript

db.posts.insertOne(
  { title: "Durable" },
  { writeConcern: { w: "majority", j: true } }
)

`w`	Meaning
`1`	Primary only
`"majority"`	Majority of voting nodes (durable)

Read concern — consistency of read:

javascript

db.posts.find().readConcern("majority")

Read preference (driver / connection string):

Value	Use
primary (default)	Strong consistency
secondary	Offload reads; may be stale
nearest	Lowest latency node

Analytics on secondaries; money transfers on primary.

Dev Replica Set With Docker Compose

docker-compose.replicaset.yml:

yaml

Initialize (run once):

javascript

rs.initiate({
  _id: "rs0",
  members: [
    { _id: 0, host: "localhost:27017" },
    { _id: 1, host: "localhost:27018" },
    { _id: 2, host: "localhost:27019" }
  ]
})

Connection string:

text

mongodb://localhost:27017,localhost:27018,localhost:27019/hello_demo?replicaSet=rs0

Single-container dev shortcut:

bash

docker run -d --name mongo-rs -p 27017:27017 mongo:7 mongod --replSet rs0 --bind_ip_all
docker exec mongo-rs mongosh --eval "rs.initiate()"

Failover (Concept)

Primary dies → remaining members elect new primary (seconds to minutes). App drivers retry with same replica set URI—use official drivers with retryable writes.

Plan maintenance: step down primary gracefully with rs.stepDown().

Sharding Overview

When one replica set is not enough storage or write throughput, shard data horizontally.

text

                    ┌─────────┐
         App ──────►│ mongos  │  (query router)
                    └────┬────┘
           ┌─────────────┼─────────────┐
           ▼             ▼             ▼
      ┌─────────┐  ┌─────────┐  ┌─────────┐
      │ Shard A │  │ Shard B │  │ Shard C │
      │ rs0     │  │ rs1     │  │ rs2     │
      └─────────┘  └─────────┘  └─────────┘
           Config servers store metadata (chunk ranges)

Shard key

Field(s) that determine document placement—immutable after sharding.

Good shard keys:

High cardinality (many distinct values)
Even distribution (avoid monotonic _id only hot shard unless hashed)
Appears in common queries (targeted queries hit one shard)

Bad: low cardinality status: "active" — everything on one chunk.

When to shard

Signal	Action
Dataset fits one server, queries fast	Replica set only
Disk > single machine, writes saturate one primary	Consider sharding
Most apps never need it	Scale vertically + indexes first

Atlas can enable sharding on M30+ tiers; M0 dev stays single replica set — Atlas.

Sharded cluster limitations (awareness)

Multi-shard transactions supported but costlier
$lookup across shards may be slower
Unique index must include shard key (unless special cases)

Backup and Replica Sets

mongodump from secondary reduces primary load. Atlas offers continuous backup and point-in-time restore.

Post-Chapter Checklist

You can explain primary vs secondary
You know why transactions need replica set
You read rs.status() once in dev
You understand shard key purpose at high level

FAQ

Three nodes minimum?

Production: 3 data-bearing members (or 2 + arbiter for non-critical dev). Elections need majority.

Read from secondary always stale?

Usually milliseconds behind; not for read-your-writes unless readConcern: "majority" and careful routing.

Sharding vs separate databases?

Separate DBs on same cluster share resources; sharding splits one collection across hardware.

Compare MySQL replication?

MySQL primary/replica similar role split; MongoDB election automates failover — MySQL replication concept.

Kubernetes MongoDB?

Use MongoDB Kubernetes Operator or Atlas—manual shard ops are expert territory.