MongoDB ReplicaSet in K8S -- can't connect via port forward
Master System Design with Codemia
Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.
Introduction
When you port-forward to a MongoDB replica set member in Kubernetes, the initial connection succeeds but the client immediately disconnects or hangs. This happens because MongoDB replica sets return their internal hostnames (like mongo-0.mongo-headless.default.svc.cluster.local) in the isMaster response, and your local client cannot resolve these names. The fix is to either add the internal hostnames to your local /etc/hosts pointing to 127.0.0.1, connect with directConnection=true to skip replica set discovery, or use a mongos router instead.
Why It Fails
When a MongoDB client connects to a replica set member, it runs isMaster (or hello) to discover the full topology:
Your local machine cannot resolve mongo-0.mongo-headless.default.svc.cluster.local, so the driver drops the connection or times out trying to reach the other members.
Fix 1: Use directConnection=true
The simplest fix — tell the driver to treat this as a standalone connection:
In application code:
This bypasses replica set discovery entirely. The client talks only to the forwarded pod.
Fix 2: Add Hosts to /etc/hosts
If you need replica set features (like read preferences), map the internal DNS names to localhost:
Add to /etc/hosts:
Then connect with the replica set URI:
This approach is fragile but allows full replica set functionality locally.
Fix 3: Use a Temporary Pod
Run a client pod inside the cluster where DNS resolution works:
Typical StatefulSet Configuration
Initialize the Replica Set
Common Pitfalls
- Connecting without
directConnection=truevia port-forward: MongoDB drivers perform replica set discovery by default. The internal hostnames returned are unresolvable from outside the cluster, causing the driver to hang or disconnect after the initial handshake. - Port-forwarding to the Service instead of the Pod:
kubectl port-forward svc/mongo-headless 27017:27017routes to a random pod. For predictable connections, port-forward to the specific pod:kubectl port-forward pod/mongo-0 27017:27017. - Using
bind_ip: localhostin MongoDB config: By default, MongoDB only listens on localhost. In Kubernetes,--bind_ip_allis required so the pod accepts connections from other pods and from the port-forward tunnel. - Forgetting that writes only go to the primary: If you port-forward to a secondary member and try to write, MongoDB rejects the operation. Use
directConnection=truewith the primary pod, or users.status()to identify the current primary first. - Not initializing the replica set after deploying the StatefulSet: Deploying the StatefulSet starts three independent
mongodprocesses. They do not form a replica set until you runrs.initiate()on one member. Without initialization, each pod acts as a standalone instance.
Summary
- Port-forwarding to a MongoDB replica set fails because the client receives internal DNS names it cannot resolve
- Use
directConnection=truein the connection string to bypass replica set discovery (simplest fix) - Map internal hostnames in
/etc/hostsif you need full replica set features locally - Use a temporary pod inside the cluster for debugging with full DNS resolution
- Always use
--bind_ip_allwhen running MongoDB in Kubernetes containers

