Configuring a minimal MongoDB Replica Set with authentication

7 minutes read
databases • MongoDB
databases • mongodb • high availability

Forewords

This was tested using 3 different nodes with debian 10 GNU/Linux

A replica set in MongoDB is a group of mongod processes that maintain the same data set. Replica sets provide redundancy and high availability, and are the basis for all production deployments.

The primary node receives all write operations and records all changes to its data sets in its operation log, i.e. oplog. The secondaries replicate the primary’s oplog and apply the operations to their data sets such that the secondaries’ data sets reflect the primary’s data set.

The first requirement is to have mongodb running, this is, to have a mongod process running at a certain host. We require that the mongod can be accessed from the outside. This poses a risk, hence we need that our mongo database uses authentication.

Now, in debian, a rapid way to install the official mongodb server is to run the sequence

sudo apt update
sudo apt install -y gnupg
wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add -
echo "deb http://repo.mongodb.org/apt/debian buster/mongodb-org/4.4 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.4.list
sudo apt update
sudo apt-get install -y mongodb-org
systemctl start mongod.service

This should expose mongo at the port 27017 of the host. In what follows I’ll consider a fresh install, which looks more likely to be the use case

Preparation

It is advisable to go and create an admin user for this instance. run mongo and type

# In the context of the mongo console
db.createUser({ user: 'admin', pwd: passwordPrompt(), roles: [{ role:'root', db: 'admin'}] })

Configuration file /etc/mongod.conf

Edit the configuration file to look like

net:
  port: 27017
  #bindIp: 127.0.0.1
  bindIp: 0.0.0.0
...
#security:
  #keyFile: /etc/mongod_rs.key
  #authorization: enabled
#operationProfiling:

replication:
  replSetName: "rs0"

This achieves,

a) Our instance is reachable through 0.0.0.0, hence open to the world b) Pre-fills the auth mechanisms that we will enable once the replicaset is configure (the keyFile will be created soon) c) sets the replicaset to rs0. It could be anything more creative

Creating the key for inter-replica authentication

The key authentication relies on the fact that all replicas have the same key. The key must have certain size and be a base64 string. It can be created with

openssl rand -base64 756 > /etc/mongod_rs.key

And I insist, it must be the same key for all the nodes on the replica set, so create it once and place it under /etc/mongod_rs.key of all nodes.

The /etc/mongod_rs.key contents should look like,

0jWT0/u5vQd89NESR7Dvunyi3aQGLPlu3y6kxyMTSVNYJC3HWSM96t8K7OUrSsud
Oa0nenkWuYYip+RGl/kPeR6oT4YXhEWsOi8BJOV9x0zp99Irx39b18k9t3vYrvZh
M1ChKfPfgEB/Gf5abrv+F85ZbL5W1ived4POknMPTyzv/I6LDxzzsLLYEY4U2TQT
aWZ/80WqjHVVY0tyOsjwWEcuYDI7YlDCzvzw3nTqbegBqU1W4pscQ01yzpK9gHyf
43HJtAn05mmOtbzdx6fmV3bR0swV/BtdnZn6F5PmlOLRb4KLxUc+NGc4HDnWn/eJ
wwH1Hsc9oTtMxKI4JcTHzt3fnY3OuWw8jlrgjzepeTbxAZyVLCwpGsOtPtQjtkh9
MPE9VDkG4WFXJ07gCGJfklEvt6W8nDpH+CSFn5nDvgF4FP8ffWDFKV8KKbG/T7PU
KH7c4pqIzg4kS9OrVhDcrRH3HpwqUcuAlVW2Zspum6mdgHfG6OBMbUcJYmckbTgi
nllh+Nzyhxmgarfl1n2oCUaEuuvhPyGvqyjmuRvIWjjmqMt+QanG+mHMYjamYY3K
bBIkO4M1Ec6vNpk7a5T6OQurctyCC6VQYXRMFkvfLic+6sNDVXa9B9ymt3V+kzYj
4MbHyAhINLGKgPLRcVvhHJD2t3p/4O9SoNlbo2Wh6C+v51sX6qULI/M2OYSL1snL
4beDWqNH6aq/6m6vlGcarHipaTRd+C7XEMw43aP0m4QJ4fNH7k14r4MH4edoxErd
WaTq87Up54wEIdVRKpS2rpz1NjnBheDIDysczaMSat+xhKyX3dzzAuYr2vCHFJlh
gHkaRxYtTde0YPFCLe6Hz/vzTeEzKcEZGvHM1ZO+YAxtsJToywWRhe4TqninM9n/
R3bR5Jm3ppO0X11SD9HsqsvJy3zlYTnn91/yOOR9kby8JKkKTZlfPw9W0oUyxPB+
Y5BUoxAlgJ6xR4JLz8l6sfKwBGVY0sJqVKVQZmw3BKEtOI+b

Again this file must be the same in all hosts serving a mongodb replica.

Be sure to change the permissions to

chown mongodb:root /etc/mongod_rs.key
chmod 400 /etc/mongod_rs.key

And to enable autostart and restart the mongo replica for the changes to take effect

systemctl enable mongod.service
systemctl restart mongod.service

Configuring hosts

We need to modify each host’s /etc/hosts file in order to add a reference to the other hosts Also, mongo forbids mixing hosts, either 3 instances run in the same host, or the three of them are running at separate hosts The host file should have something like

# MongoDB nodes
xxx.xxx.xxx.xxx mongodb-node
xxx.xxx.xxx.xxx mongodb-node-02
xxx.xxx.xxx.xxx mongodb-node-03

where xxx.xxx.xxx.xxx represents the IP of the corresponding host

Initiating the replication set

Pick one of the nodes running mongo and open the mongod service. Issue a rs.initiate() command putting the hostnames for each of the hosts with its port. It is advisable to have auth disabled at this point so the replicaset initiates without problems

rs.initiate(
  {
    _id : "rs0",
    members: [
      { _id : 0, host : "mongodb-node:27017" },
      { _id : 1, host : "mongodb-node-02:27017" },
      { _id : 2, host : "mongodb-node-03:27017" }
    ]
  }
)

Now, perform rs.status() and look for the primary. If errors due to authentication are raised as in,

rs0:PRIMARY> rs.status()
{
	"operationTime" : Timestamp(1603902158, 1),
	"ok" : 0,
	"errmsg" : "command replSetGetStatus requires authentication",
	"code" : 13,
	"codeName" : "Unauthorized",
	"$clusterTime" : {
		"clusterTime" : Timestamp(1603902158, 1),
		"signature" : {
			"hash" : BinData(0,"B/2vzM5Hr/dc/87LzyqMtaYCQsA="),
			"keyId" : NumberLong("6888691302955745283")
		}
	}
}

We need to authenticate first, using

# Use the user we've created before, when setting up the mongo instance
db.auth("admin", passwordPrompt())

and issue a rs.status() again. It should show something like,

rs0:PRIMARY> rs.status()
{
	"set" : "rs0",
	"date" : ISODate("2020-10-28T16:23:30.698Z"),
	"myState" : 1,
	"term" : NumberLong(6),
	"syncSourceHost" : "",
	"syncSourceId" : -1,
	"heartbeatIntervalMillis" : NumberLong(2000),
	"majorityVoteCount" : 2,
	"writeMajorityCount" : 2,
	"votingMembersCount" : 3,
	"writableVotingMembersCount" : 3,
	"optimes" : {
		"lastCommittedOpTime" : {
			"ts" : Timestamp(1603902208, 1),
			"t" : NumberLong(6)
		},
		"lastCommittedWallTime" : ISODate("2020-10-28T16:23:28.974Z"),
		"readConcernMajorityOpTime" : {
			"ts" : Timestamp(1603902208, 1),
			"t" : NumberLong(6)
		},
		"readConcernMajorityWallTime" : ISODate("2020-10-28T16:23:28.974Z"),
		"appliedOpTime" : {
			"ts" : Timestamp(1603902208, 1),
			"t" : NumberLong(6)
		},
		"durableOpTime" : {
			"ts" : Timestamp(1603902208, 1),
			"t" : NumberLong(6)
		},
		"lastAppliedWallTime" : ISODate("2020-10-28T16:23:28.974Z"),
		"lastDurableWallTime" : ISODate("2020-10-28T16:23:28.974Z")
	},
	"lastStableRecoveryTimestamp" : Timestamp(1603902198, 1),
	"electionCandidateMetrics" : {
		"lastElectionReason" : "stepUpRequestSkipDryRun",
		"lastElectionDate" : ISODate("2020-10-28T15:26:28.867Z"),
		"electionTerm" : NumberLong(6),
		"lastCommittedOpTimeAtElection" : {
			"ts" : Timestamp(1603898780, 1),
			"t" : NumberLong(5)
		},
		"lastSeenOpTimeAtElection" : {
			"ts" : Timestamp(1603898780, 1),
			"t" : NumberLong(5)
		},
		"numVotesNeeded" : 2,
		"priorityAtElection" : 1,
		"electionTimeoutMillis" : NumberLong(10000),
		"priorPrimaryMemberId" : 0,
		"numCatchUpOps" : NumberLong(0),
		"newTermStartDate" : ISODate("2020-10-28T15:26:28.874Z"),
		"wMajorityWriteAvailabilityDate" : ISODate("2020-10-28T15:26:28.918Z")
	},
	"members" : [
		{
			"_id" : 0,
			"name" : "mongodb-node:27017",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 3417,
			"optime" : {
				"ts" : Timestamp(1603902208, 1),
				"t" : NumberLong(6)
			},
			"optimeDurable" : {
				"ts" : Timestamp(1603902208, 1),
				"t" : NumberLong(6)
			},
			"optimeDate" : ISODate("2020-10-28T16:23:28Z"),
			"optimeDurableDate" : ISODate("2020-10-28T16:23:28Z"),
			"lastHeartbeat" : ISODate("2020-10-28T16:23:30.421Z"),
			"lastHeartbeatRecv" : ISODate("2020-10-28T16:23:30.006Z"),
			"pingMs" : NumberLong(0),
			"lastHeartbeatMessage" : "",
			"syncSourceHost" : "mongodb-node-03:27017",
			"syncSourceId" : 2,
			"infoMessage" : "",
			"configVersion" : 1,
			"configTerm" : 6
		},
		{
			"_id" : 1,
			"name" : "mongodb-node-02:27017",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
			"uptime" : 3433,
			"optime" : {
				"ts" : Timestamp(1603902208, 1),
				"t" : NumberLong(6)
			},
			"optimeDate" : ISODate("2020-10-28T16:23:28Z"),
			"syncSourceHost" : "",
			"syncSourceId" : -1,
			"infoMessage" : "",
			"electionTime" : Timestamp(1603898788, 1),
			"electionDate" : ISODate("2020-10-28T15:26:28Z"),
			"configVersion" : 1,
			"configTerm" : 6,
			"self" : true,
			"lastHeartbeatMessage" : ""
		},
		{
			"_id" : 2,
			"name" : "mongodb-node-03:27017",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 3431,
			"optime" : {
				"ts" : Timestamp(1603902208, 1),
				"t" : NumberLong(6)
			},
			"optimeDurable" : {
				"ts" : Timestamp(1603902208, 1),
				"t" : NumberLong(6)
			},
			"optimeDate" : ISODate("2020-10-28T16:23:28Z"),
			"optimeDurableDate" : ISODate("2020-10-28T16:23:28Z"),
			"lastHeartbeat" : ISODate("2020-10-28T16:23:30.583Z"),
			"lastHeartbeatRecv" : ISODate("2020-10-28T16:23:29.823Z"),
			"pingMs" : NumberLong(0),
			"lastHeartbeatMessage" : "",
			"syncSourceHost" : "mongodb-node-02:27017",
			"syncSourceId" : 1,
			"infoMessage" : "",
			"configVersion" : 1,
			"configTerm" : 6
		}
	],
	"ok" : 1,
	"$clusterTime" : {
		"clusterTime" : Timestamp(1603902208, 1),
		"signature" : {
			"hash" : BinData(0,"A5FW5fxYTcD2+6fnXpO7TWEAKZo="),
			"keyId" : NumberLong("9888691302955745288")
		}
	},
	"operationTime" : Timestamp(1603902208, 1)
}

The members property shows who is the primary and the secondaries.

Connect to your primary node (I’ve already done that as you see on the promts above) and create an user to be an admin on the replica set,

admin = db.getSiblingDB("admin")
admin.createUser(
  {
    user: "rs-admin",
    pwd: passwordPrompt(), // or cleartext password
    roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
  }
)

The whole process looks like this

rs0:PRIMARY> admin = db.getSiblingDB("admin")
admin
rs0:PRIMARY> admin.createUser(
...   {
...     user: "rs-admin",
...     pwd: passwordPrompt(),
...     roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
...   }
... )
Enter password: 
Successfully added user: {
	"user" : "rs-admin",
	"roles" : [
		{
			"role" : "userAdminAnyDatabase",
			"db" : "admin"
		}
	]
}

Now, authenticate as this new created user

db.getSiblingDB("admin").auth("rs-admin", passwordPrompt())

And create a cluster administrator user,

db.getSiblingDB("admin").createUser(
  {
    "user" : "rs-cluster-admin",
    "pwd" : passwordPrompt(), 
    roles: [ { "role" : "clusterAdmin", "db" : "admin" } ]
  }
)

Now, you can create users to authenticate against the replica-set, create database, or do whatever mongodb use case you have

db.getSiblingDB("mydb").createUser(
  {
    "user" : "mydb-owner",
    "pwd" : passwordPrompt(),     // or cleartext password
    roles: [ { "role" : "dbOwner", "db" : "mydb" } ]
  }
)

References

This document presents a combination of practical experience accompanied of lots of reading of the official documentation on Deploying Replica Sets.

comments powered by Disqus