MongoDB StatefulSet in Kubernetes

MongoDB StatefulSet in Kubernetes

Deeptiman Pattnaik's photo
Deeptiman Pattnaik
·Sep 30, 2020·

6 min read

Subscribe to my newsletter and never miss my upcoming articles

In this article, We will see how we can deploy a MongoDB replica set into the Kubernetes as a StatefulSet. There is a brief understanding of StatefulSet is required before deploying MongoDB into the Kubernetes.

What is StatefulSet?

In stateful application, the N-replicas of master nodes manages several worker nodes under a cluster. So, if any master node goes down the other ordinal instances will be active to execute the workflow. The master node instances must be identified as a unique ordinal number known as StatefulSet.

StatefulSet design applies to all kinds of stateful applications such as MongoDB, Redis Cache, Kafka, and any other database management system that runs under a cluster.

Design of StatefulSet

The StatefulSet will create N-replica Pods define under .spec.replicas. The Pod identity starts from 0 to N-1 ordinal set.

Example-

N = 3 (replicas) Name of StatefulSet = mongod Pod Identity = [ mongod-0, mongod-1, mongod-2 ]

Network Identity

  • The Pods that are defined under a StatefulSet will have a hostname construct with *$(statefulset name)-$(ordinal). So, hostnames will be [mongod-0, mongod-1, mongod-2*].

  • The domain name is known as connection string for database connectivity will be identified as

* $(service name).$(namespace).svc.cluster.local*

The service will run as a Headless Service to manage the domain of a Pod. In general understanding of Headless Service, there is no need for LoadBalancer or a *kube-proxy to interact directly with Pods but using a Service IP, so the Cluster IP is set to *none.

  • The StatefulSet domain for each Pod will look like
*$(statefulset name)-$(ordinal).$(service name).$(namespace).svc.cluster.local*

Example- mongod-0.mongodb-service.default.svc.cluster.local mongod-1.mongodb-service.default.svc.cluster.local mongod-2.mongodb-service.default.svc.cluster.local Each StatefulSet domain will work as a DNS subdomain depending on the configuration of DNS for a Cluster.

Deployment and Scaling of StatefulSet

  • Pods are deployed in {0..N-1} order for a StatefulSet of N-replicas.

  • The termination of Pods is performed in reverse {N-1..0}.

  • The execution of a Pod depends on other ordinal index Pods to be running and active.

  • The termination of a Pod also required other ordinal index Pods has to be terminated before.

Deploying MongoDB as Kubernetes StatefulSet

The deployment will follow as creating a Headless Service and StatefulSet with N=3 replicas under a cluster. We will also see how to set up the MongoDB administrator for a container that runs under a cluster.

Create Headless Service

YAML: [mongodb-service.yaml](github.com/Deeptiman/go-cache-kubernetes/bl..)

apiVersion: v1
kind: Service
metadata:
  name: mongodb-service
  labels:
    name: mongo
spec:
  ports:
  - port: 27017
    targetPort: 27017
  clusterIP: None
  selector:
    role: mongo
$ kubectl apply -f mongodb-service.yaml

Deploy MongoDB StatefulSet

YAML: [mongodb-statefulset.yaml](github.com/Deeptiman/go-cache-kubernetes/bl..)

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mongod
spec:
  serviceName: mongodb-service
  replicas: 3
  selector:
    matchLabels:
      role: mongo
  template:
    metadata:
      labels:
        role: mongo
        environment: test
        replicaset: MainRepSet
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: mongod-container
          image: mongo
          command:
            - "mongod"
            - "--bind_ip"
            - "0.0.0.0"
            - "--replSet"
            - "MainRepSet"
          resources:
            requests:
              cpu: 0.2
              memory: 200Mi
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: mongodb-persistent-storage-claim
              mountPath: /data/db
  volumeClaimTemplates:
  - metadata:
      name: mongodb-persistent-storage-claim
      annotations:
        volume.beta.kubernetes.io/storage-class: "standard"
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi
$ kubectl apply -f mongodb-statefulset.yaml

The StatefulSet requires at least 1GB of persistent storage that can be achieved through *StorageClass, [PersistentVoulmeClaim](kubernetes.io/docs/concepts/storage/persist.. while deploying the StatefulSet.

Setup the ReplicaSet and Administrator

We can create an Administrator for any single Pod among N-replicas.

$ kubectl exec -it mongod-0 -c mongod-container-app bash
root@mongod-0:/# hostname -f
root@mongod-0:/# mongod-0.mongodb-service.default.svc.cluster.local

Type the following command to generate the replica set.

root@mongod-0:/# mongo
> rs.initiate({ _id: "MainRepSet", version: 1, 
members: [ 
 { _id: 0, host: "mongod-0.mongodb-service.default.svc.cluster.local:27017" }, 
 { _id: 1, host: "mongod-1.mongodb-service.default.svc.cluster.local:27017" }, 
 { _id: 2, host: "mongod-2.mongodb-service.default.svc.cluster.local:27017" } ]});
{
  "ok" : 1,
  "$clusterTime" : {
    "clusterTime" : Timestamp(1601481520, 1),
    "signature" : {
      "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId" : NumberLong(0)
    }
  },
  "operationTime" : Timestamp(1601481520, 1)
}
MainRepSet:SECONDARY>  rs.status();
{
  "set" : "MainRepSet",
  "date" : ISODate("2020-09-30T15:59:04.013Z"),
  "myState" : 1,
  "term" : NumberLong(1),
  "syncSourceHost" : "",
  "syncSourceId" : -1,
  "heartbeatIntervalMillis" : NumberLong(2000),
  "majorityVoteCount" : 2,
  "writeMajorityCount" : 2,
  "votingMembersCount" : 3,
  "writableVotingMembersCount" : 3,
  "optimes" : {
    "lastCommittedOpTime" : {
      "ts" : Timestamp(1601481536, 1),
      "t" : NumberLong(1)
    },
    "lastCommittedWallTime" : ISODate("2020-09-30T15:58:56.243Z"),
    "readConcernMajorityOpTime" : {
      "ts" : Timestamp(1601481536, 1),
      "t" : NumberLong(1)
    },
    "readConcernMajorityWallTime" : ISODate("2020-09-30T15:58:56.243Z"),
    "appliedOpTime" : {
      "ts" : Timestamp(1601481536, 1),
      "t" : NumberLong(1)
    },
    "durableOpTime" : {
      "ts" : Timestamp(1601481536, 1),
      "t" : NumberLong(1)
    },
    "lastAppliedWallTime" : ISODate("2020-09-30T15:58:56.243Z"),
    "lastDurableWallTime" : ISODate("2020-09-30T15:58:56.243Z")
  },
  "lastStableRecoveryTimestamp" : Timestamp(1601481533, 1),
  "electionCandidateMetrics" : {
    "lastElectionReason" : "electionTimeout",
    "lastElectionDate" : ISODate("2020-09-30T15:58:52.739Z"),
    "electionTerm" : NumberLong(1),
    "lastCommittedOpTimeAtElection" : {
      "ts" : Timestamp(0, 0),
      "t" : NumberLong(-1)
    },
    "lastSeenOpTimeAtElection" : {
      "ts" : Timestamp(1601481520, 1),
      "t" : NumberLong(-1)
    },
    "numVotesNeeded" : 2,
    "priorityAtElection" : 1,
    "electionTimeoutMillis" : NumberLong(10000),
    "numCatchUpOps" : NumberLong(0),
    "newTermStartDate" : ISODate("2020-09-30T15:58:53.164Z"),
    "wMajorityWriteAvailabilityDate" : ISODate("2020-09-30T15:58:54.898Z")
  },
  "members" : [
    {
      "_id" : 0,
      "name" : "mongod-0.mongodb-service.default.svc.cluster.local:27017",
      "health" : 1,
      "state" : 1,
      "stateStr" : "PRIMARY",
      "uptime" : 283,
      "optime" : {
        "ts" : Timestamp(1601481536, 1),
        "t" : NumberLong(1)
      },
      "optimeDate" : ISODate("2020-09-30T15:58:56Z"),
      "syncSourceHost" : "",
      "syncSourceId" : -1,
      "infoMessage" : "",
      "electionTime" : Timestamp(1601481532, 1),
      "electionDate" : ISODate("2020-09-30T15:58:52Z"),
      "configVersion" : 1,
      "configTerm" : 1,
      "self" : true,
      "lastHeartbeatMessage" : ""
    },
    {
      "_id" : 1,
      "name" : "mongod-1.mongodb-service.default.svc.cluster.local:27017",
      "health" : 1,
      "state" : 2,
      "stateStr" : "SECONDARY",
      "uptime" : 22,
      "optime" : {
        "ts" : Timestamp(1601481536, 1),
        "t" : NumberLong(1)
      },
      "optimeDurable" : {
        "ts" : Timestamp(1601481536, 1),
        "t" : NumberLong(1)
      },
      "optimeDate" : ISODate("2020-09-30T15:58:56Z"),
      "optimeDurableDate" : ISODate("2020-09-30T15:58:56Z"),
      "lastHeartbeat" : ISODate("2020-09-30T15:59:02.847Z"),
      "lastHeartbeatRecv" : ISODate("2020-09-30T15:59:02.461Z"),
      "pingMs" : NumberLong(0),
      "lastHeartbeatMessage" : "",
      "syncSourceHost" : "mongod-0.mongodb-service.default.svc.cluster.local:27017",
      "syncSourceId" : 0,
      "infoMessage" : "",
      "configVersion" : 1,
      "configTerm" : 1
    },
    {
      "_id" : 2,
      "name" : "mongod-2.mongodb-service.default.svc.cluster.local:27017",
      "health" : 1,
      "state" : 2,
      "stateStr" : "SECONDARY",
      "uptime" : 22,
      "optime" : {
        "ts" : Timestamp(1601481536, 1),
        "t" : NumberLong(1)
      },
      "optimeDurable" : {
        "ts" : Timestamp(1601481536, 1),
        "t" : NumberLong(1)
      },
      "optimeDate" : ISODate("2020-09-30T15:58:56Z"),
      "optimeDurableDate" : ISODate("2020-09-30T15:58:56Z"),
      "lastHeartbeat" : ISODate("2020-09-30T15:59:02.848Z"),
      "lastHeartbeatRecv" : ISODate("2020-09-30T15:59:02.472Z"),
      "pingMs" : NumberLong(0),
      "lastHeartbeatMessage" : "",
      "syncSourceHost" : "mongod-0.mongodb-service.default.svc.cluster.local:27017",
      "syncSourceId" : 0,
      "infoMessage" : "",
      "configVersion" : 1,
      "configTerm" : 1
    }
  ],
  "ok" : 1,
  "$clusterTime" : {
    "clusterTime" : Timestamp(1601481536, 1),
    "signature" : {
      "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId" : NumberLong(0)
    }
  },
  "operationTime" : Timestamp(1601481536, 1)
}
MainRepSet:PRIMARY>

Create the Administrator for the MongoDB

MainRepSet:PRIMARY> db.getSiblingDB("admin").createUser({
...       user : "demoadmin",
...       pwd  : "demopwd123",
...       roles: [ { role: "root", db: "admin" } ]
...  });
Successfully added user: {
  "user" : "demoadmin",
  "roles" : [
    {
      "role" : "root",
      "db" : "admin"
    }
  ]
}

Verify MongoDB StatefulSet Setup from minikube dashboard

Type the following command to navigate to the Kubernetes dashboard in the browser.

$ minikube dashboard

Workload StatusWorkload Status

mongod StatefulSetmongod StatefulSet

MongoDB StatefulSet MetadataMongoDB StatefulSet Metadata

MongoDB StatefulSet PodsMongoDB StatefulSet Pods

MongoDB Headless ServiceMongoDB Headless Service

MongoDB Persistent Volume Claims 1GB storage capacity for each PodMongoDB Persistent Volume Claims 1GB storage capacity for each Pod

MongoDB StatefulSet EventsMongoDB StatefulSet Events

So, this covers the overview of understanding Kubernetes StatefulSet and deploying a stateful application like MongoDB as StatefulSet in the minikube environment.

Please also check the project on Kubernetes at my Github.

I hope you find this article useful :)

Thanks

 
Share this