分佈式文檔存儲數據庫之MongoDB副本集
- 2020 年 11 月 10 日
- 筆記
- mongodb, MongoDB副本集, NoSQL
前文我們聊到了mongodb的索引的相關作用和介紹以及索引的管理，回顧請參考//www.cnblogs.com/qiuhom-1874/p/13950287.html；今天我們來聊下mongodb的副本集；
1、什麼是副本集？主要作用是什麼？它是怎麼工作的呢？
所謂副本集是指服務於同一數據集的多個mongodb實例；這意味着在mongodb中數據集副本的數量可以是多個，每個副本的數據都是一樣的；副本的存在主要作用是對mongodb數據庫中的數據做冗餘備份和提高數據服務的可用性；在mongodb中對數據做冗餘的方式有兩種，一種是主從架構（master/slave），這種架構和mysql中的主從架構沒有什麼不同，但是在mongodb中，主從架構的方式幾乎沒有人用，處於廢棄的狀態。。另外一種是副本集(replica set)，副本集本質上也是主從架構的一種，它和我們剛才說的主從架構，有一個顯著的區別，副本集支持故障自動轉移，不需要人工手動干預；所謂的故障自動轉移是指當一個副本集中主節點因各種原因宕機或者從節點在一定時間內沒有收到主節點發送到心跳信息，此時就會觸發從節點重新選舉主節點的操作；在多個從節點中選舉出來一個主節點以後，其他從節點能夠自動指向新選舉的主節點同步數據，從而完成這次故障轉移；
2、mongodb副本集架構
提示：對於mongodb中的副本集來說，為了保證數據的一致性，一個副本集中只能有一個主節點，並且只有主節點才擁有讀寫權限，其他從節點都只有讀權限；在mongodb的副本集中，主節點我們叫做primary，從節點叫做secondary；
3、副本集是通過什麼來複制數據的呢？
我知道mysql的主從複製是通過主節點記錄寫操作到binlog中，從節點通過同步主節點的binlog，然後把同步的binlog在本地進行重放，從而實現數據同步；在mongodb中也是類似的操作，不同的是在mongodb中主節點將寫操作記錄到oplog(操作日誌)，各從節點通過複製主節點的oplog，在本地重放實現數據同步；oplog的作用和mysql中的binlog的作用很類似，都是用來記錄寫操作事件，都是用來同步數據；mongodb中的oplog是一個定容集合，所謂定容是指它的大小是固定的，它不會像binlog隨日誌的增多而逐漸變大；通常oplog我們不人為指定其大小，默認就是佔用所在文件系統空閑磁盤的5%；除了大小是不變的，它還有一個冪等的特點；所謂冪等就是不管在monogdb中執行多少次oplog中的操作，其最終的結果都是一樣的；如果寫操作日誌把oplog寫滿了，它會從頭覆蓋之前的oplog，依次循環寫oplog；
4、副本集是怎麼完成故障轉移的？
在前邊我們聊zookeeper時，聊到過分佈式場景中的選舉過程；mongodb的主節點故障以後，也是通過這種機制來完成選舉；在mongodb的副本集中，主節點每隔兩秒向各從節點發送心跳信息，各從節點通過檢測主節點的心跳信息來判定主節點是否存活；如果在一定的時間範圍內，從節點沒有收到主節點發送到心跳信息，此時從節點會認為主節點掛了，從而觸發主節點選舉的操作；這個選舉和zookeeper里的選舉很類似，通常情況都是大於集群總票的一半一方可以代表集群正常工作；所以通常情況mongodb副本集都是基數個節點組成；當然mongodb中也可以是偶數節點（正常提供服務的節點），如果是偶數節點，通常會藉助一個仲裁節點來完成選舉；仲裁節點擁有選票，但不被選舉成為主節點，也不擁有副本數據，同時它必須要能夠檢測主節點心跳；簡單總結就是mongodb副本集是通過從節點檢測主節點心跳來判斷主節點是否存活，如果在一定時間範圍內，沒有檢測到主節點的心跳信息，此時就會觸發主節點選舉操作，如果集群節點數量為偶數個，通常會藉助仲裁節點來完成選舉；從而實現完成故障自動轉移；
5、mongodb副本集中特殊類型節點分類
0優先級節點：這種節點的特點是優先級為0，可參與選舉，擁有副本數據，但不被選舉成為主節點，可讀不可寫；這種節點我們也叫冷備節點；通常用於異地容災使用；
被隱藏的從節點：這種節點的特點是，可參與選舉，擁有副本數據，但不被選舉成為主節點，對客戶端不可讀寫也不可見；通常用於同步同一副本集中的其他節點的不同工作流的場景中；
延遲複製的從節點：這種節點的特點是，副本數據落後主節點一個時間窗口，也就說這種節點上的副本數據總是過期的；除此它可參與選舉，不可被選舉為主節點；主要用於特殊功用；比如在主節點上執行了一個刪除操作，我們可以在延遲複製的從節點上把數據找回；
arbiter節點：這種就是我們說的仲裁節點，它可參與選舉，不擁有副本數據，不被選舉成為主節點，不可讀寫；主要作用是輔助判定主節點是否存活，輔助從節點完成選舉，實現故障轉移；
6、創建mongodb的副本集合
環境準備
| 主機名 | ip地址 |
| node01 | 192.168.0.41 |
| node02 | 192.168.0.42 |
| node03 | 192.168.0.43 |
在三個節點上分別做好時間同步，關閉selinux,關閉防火牆，主機名解析，有必要還可以做ssh互信。準備好基礎環境以後，在三台server上配置mongodb的yum源
[root@node01 ~]# cat /etc/yum.repos.d/mongodb.repo [mongodb-org] name = MongoDB Repository baseurl = //mirrors.aliyun.com/mongodb/yum/redhat/7/mongodb-org/4.4/x86_64/ gpgcheck = 1 enabled = 1 gpgkey = //www.mongodb.org/static/pgp/server-4.4.asc [root@node01 ~]# scp /etc/yum.repos.d/mongodb.repo node02:/etc/yum.repos.d/ mongodb.repo 100% 206 80.4KB/s 00:00 [root@node01 ~]# scp /etc/yum.repos.d/mongodb.repo node03:/etc/yum.repos.d/ mongodb.repo 100% 206 88.9KB/s 00:00 [root@node01 ~]#
安裝mongodb-org
yum install -y mongodb-org
準備數據目錄和存放日誌的目錄，並將其屬主和屬組更改為mongod用戶
[root@node01 ~]# mkdir -pv /mongodb/{data,log}
mkdir: created directory 『/mongodb』
mkdir: created directory 『/mongodb/data』
mkdir: created directory 『/mongodb/log』
[root@node01 ~]# chown -R mongod.mongod /mongodb/
[root@node01 ~]# ll -d /mongodb/
drwxr-xr-x 4 mongod mongod 29 Nov 10 19:36 /mongodb/
[root@node01 ~]#
提示：以上需在三個節點都要做一遍；
配置mongodb
提示：主要配置replication，其中oplogSizeMB用來指定oplog的大小，默認不指定，其大小就是所在文件系統空閑磁盤的%5；replSetName用來指定副本集的名稱，這個名稱非常重要，主要用來標識不同副本集；enableMajorityReadConcern用來指定是否開啟mongodb周期性的做快照，並記錄oplog的時間戳；
完整配置
[root@node01 ~]# cat /etc/mongod.conf systemLog: destination: file logAppend: true path: /mongodb/log/mongod.log storage: dbPath: /mongodb/data/ journal: enabled: true processManagement: fork: true pidFilePath: /var/run/mongodb/mongod.pid timeZoneInfo: /usr/share/zoneinfo net: port: 27017 bindIp: 0.0.0.0 replication: oplogSizeMB: 2048 replSetName: test_replset enableMajorityReadConcern: false #security: #operationProfiling: #sharding: ## Enterprise-Only Options #auditLog: #snmp: [root@node01 ~]#
View Code
複製配置文件到node02,node03
[root@node01 ~]# scp /etc/mongod.conf node02:/etc/ mongod.conf 100% 494 315.1KB/s 00:00 [root@node01 ~]# scp /etc/mongod.conf node03:/etc/ mongod.conf 100% 494 351.6KB/s 00:00 [root@node01 ~]#
啟動服務
提示：請確保對應節點上的mongodb所監聽的端口正常即可；
連接三個節點任意一個節點查看副本集狀態
> rs.help()
rs.status() { replSetGetStatus : 1 } checks repl set status
rs.initiate() { replSetInitiate : null } initiates set with default settings
rs.initiate(cfg) { replSetInitiate : cfg } initiates set with configuration cfg
rs.conf() get the current configuration object from local.system.replset
rs.reconfig(cfg) updates the configuration of a running replica set with cfg (disconnects)
rs.add(hostportstr) add a new member to the set with default attributes (disconnects)
rs.add(membercfgobj) add a new member to the set with extra attributes (disconnects)
rs.addArb(hostportstr) add a new member which is arbiterOnly:true (disconnects)
rs.stepDown([stepdownSecs, catchUpSecs]) step down as primary (disconnects)
rs.syncFrom(hostportstr) make a secondary sync from the given member
rs.freeze(secs) make a node ineligible to become primary for the time specified
rs.remove(hostportstr) remove a host from the replica set (disconnects)
rs.secondaryOk() allow queries on secondary nodes
rs.printReplicationInfo() check oplog size and time range
rs.printSecondaryReplicationInfo() check replica set members and replication lag
db.isMaster() check who is primary
reconfiguration helpers disconnect from the database so the shell will display
an error, even if the command succeeds.
> rs.status()
{
"operationTime" : Timestamp(0, 0),
"ok" : 0,
"errmsg" : "no replset config has been received",
"code" : 94,
"codeName" : "NotYetInitialized",
"$clusterTime" : {
"clusterTime" : Timestamp(0, 0),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
>
提示：所有關於副本集的操作命令都可以使用rs.help()方法去查看幫助；其中rs.status()是用來查看副本集狀態；這裡顯示沒有副本集配置，其原因是沒有初始化副本集；
初始化副本集
> rs.initiate()
{
"info2" : "no configuration specified. Using a default configuration for the set",
"me" : "node01.test.org:27017",
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605010821, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605010821, 1)
}
test_replset:SECONDARY> rs.status()
{
"set" : "test_replset",
"date" : ISODate("2020-11-10T12:20:32.079Z"),
"myState" : 1,
"term" : NumberLong(1),
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"majorityVoteCount" : 1,
"writeMajorityCount" : 1,
"votingMembersCount" : 1,
"writableVotingMembersCount" : 1,
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1605010821, 8),
"t" : NumberLong(1)
},
"lastCommittedWallTime" : ISODate("2020-11-10T12:20:21.720Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1605010821, 8),
"t" : NumberLong(1)
},
"readConcernMajorityWallTime" : ISODate("2020-11-10T12:20:21.720Z"),
"appliedOpTime" : {
"ts" : Timestamp(1605010821, 8),
"t" : NumberLong(1)
},
"durableOpTime" : {
"ts" : Timestamp(1605010821, 8),
"t" : NumberLong(1)
},
"lastAppliedWallTime" : ISODate("2020-11-10T12:20:21.720Z"),
"lastDurableWallTime" : ISODate("2020-11-10T12:20:21.720Z")
},
"electionCandidateMetrics" : {
"lastElectionReason" : "electionTimeout",
"lastElectionDate" : ISODate("2020-11-10T12:20:21.632Z"),
"electionTerm" : NumberLong(1),
"lastCommittedOpTimeAtElection" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"lastSeenOpTimeAtElection" : {
"ts" : Timestamp(1605010821, 1),
"t" : NumberLong(-1)
},
"numVotesNeeded" : 1,
"priorityAtElection" : 1,
"electionTimeoutMillis" : NumberLong(10000),
"newTermStartDate" : ISODate("2020-11-10T12:20:21.694Z"),
"wMajorityWriteAvailabilityDate" : ISODate("2020-11-10T12:20:21.719Z")
},
"members" : [
{
"_id" : 0,
"name" : "node01.test.org:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 674,
"optime" : {
"ts" : Timestamp(1605010821, 8),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2020-11-10T12:20:21Z"),
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "Could not find member to sync from",
"electionTime" : Timestamp(1605010821, 2),
"electionDate" : ISODate("2020-11-10T12:20:21Z"),
"configVersion" : 1,
"configTerm" : 1,
"self" : true,
"lastHeartbeatMessage" : ""
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605010821, 8),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605010821, 8)
}
test_replset:PRIMARY>
提示：初始化副本集以後，再次查看副本集狀態，它告訴我們有一個成員，其主機名為node01.test.org:27017,其health狀態為1，stateStr為PRIMARY等等一堆信息；
查看副本集配置
test_replset:PRIMARY> rs.conf()
{
"_id" : "test_replset",
"version" : 1,
"term" : 1,
"protocolVersion" : NumberLong(1),
"writeConcernMajorityJournalDefault" : true,
"members" : [
{
"_id" : 0,
"host" : "node01.test.org:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : -1,
"catchUpTakeoverDelayMillis" : 30000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("5faa85853d5a2d8fdf8af85e")
}
}
test_replset:PRIMARY>
提示：可以看到心跳時間間隔為2s，超時為10s,選舉超時時長為10等等信息；
添加node02，node03節點到副本集成員
test_replset:PRIMARY> rs.add("node02")
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605011243, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605011243, 1)
}
test_replset:PRIMARY> rs.add("node03")
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605011250, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605011250, 1)
}
test_replset:PRIMARY> rs.status()
{
"set" : "test_replset",
"date" : ISODate("2020-11-10T12:27:45.849Z"),
"myState" : 1,
"term" : NumberLong(1),
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"majorityVoteCount" : 2,
"writeMajorityCount" : 2,
"votingMembersCount" : 3,
"writableVotingMembersCount" : 3,
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"lastCommittedWallTime" : ISODate("2020-11-10T12:27:41.873Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"readConcernMajorityWallTime" : ISODate("2020-11-10T12:27:41.873Z"),
"appliedOpTime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"durableOpTime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"lastAppliedWallTime" : ISODate("2020-11-10T12:27:41.873Z"),
"lastDurableWallTime" : ISODate("2020-11-10T12:27:41.873Z")
},
"electionCandidateMetrics" : {
"lastElectionReason" : "electionTimeout",
"lastElectionDate" : ISODate("2020-11-10T12:20:21.632Z"),
"electionTerm" : NumberLong(1),
"lastCommittedOpTimeAtElection" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"lastSeenOpTimeAtElection" : {
"ts" : Timestamp(1605010821, 1),
"t" : NumberLong(-1)
},
"numVotesNeeded" : 1,
"priorityAtElection" : 1,
"electionTimeoutMillis" : NumberLong(10000),
"newTermStartDate" : ISODate("2020-11-10T12:20:21.694Z"),
"wMajorityWriteAvailabilityDate" : ISODate("2020-11-10T12:20:21.719Z")
},
"members" : [
{
"_id" : 0,
"name" : "node01.test.org:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 1107,
"optime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2020-11-10T12:27:41Z"),
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"electionTime" : Timestamp(1605010821, 2),
"electionDate" : ISODate("2020-11-10T12:20:21Z"),
"configVersion" : 3,
"configTerm" : 1,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 1,
"name" : "node02:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 21,
"optime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2020-11-10T12:27:41Z"),
"optimeDurableDate" : ISODate("2020-11-10T12:27:41Z"),
"lastHeartbeat" : ISODate("2020-11-10T12:27:44.967Z"),
"lastHeartbeatRecv" : ISODate("2020-11-10T12:27:43.983Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncSourceHost" : "node01.test.org:27017",
"syncSourceId" : 0,
"infoMessage" : "",
"configVersion" : 3,
"configTerm" : 1
},
{
"_id" : 2,
"name" : "node03:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 14,
"optime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2020-11-10T12:27:41Z"),
"optimeDurableDate" : ISODate("2020-11-10T12:27:41Z"),
"lastHeartbeat" : ISODate("2020-11-10T12:27:44.967Z"),
"lastHeartbeatRecv" : ISODate("2020-11-10T12:27:44.571Z"),
"pingMs" : NumberLong(1),
"lastHeartbeatMessage" : "",
"syncSourceHost" : "node02:27017",
"syncSourceId" : 1,
"infoMessage" : "",
"configVersion" : 3,
"configTerm" : 1
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605011261, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605011261, 1)
}
test_replset:PRIMARY>
提示：添加節點到副本成員使用rs.add（），括號裏面寫主機名，或者ip地址，如果對應節點監聽端口不是27017，還需要寫明端口；除此我們也可以直接用一個子文檔的形式，手動指定節點的各屬性信息來添加節點到副本集成員；
到此副本集的配置就完成了，三個節點都添加到副本集；
驗證：在主庫上插入數據，看看從庫是否都可以正常同步其數據呢？
在主節點插入數據
test_replset:PRIMARY> use students
switched to db students
test_replset:PRIMARY> db.student_info.insert({name:"tom",age:18,gender:"M"})
WriteResult({ "nInserted" : 1 })
test_replset:PRIMARY> db.student_info.find()
{ "_id" : ObjectId("5faa89297077300f4fc31d1c"), "name" : "tom", "age" : 18, "gender" : "M" }
test_replset:PRIMARY>
在從節點查看數據
test_replset:SECONDARY> show dbs
uncaught exception: Error: listDatabases failed:{
"topologyVersion" : {
"processId" : ObjectId("5faa83025f9218c5d19ea91b"),
"counter" : NumberLong(4)
},
"operationTime" : Timestamp(1605011801, 1),
"ok" : 0,
"errmsg" : "not master and slaveOk=false",
"code" : 13435,
"codeName" : "NotMasterNoSlaveOk",
"$clusterTime" : {
"clusterTime" : Timestamp(1605011801, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs/<@src/mongo/shell/mongo.js:147:19
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:99:12
shellHelper.show@src/mongo/shell/utils.js:937:13
shellHelper@src/mongo/shell/utils.js:819:15
@(shellhelp2):1:1
test_replset:SECONDARY>
提示：這裡在從節點上沒法查看數據，原因是默認情況副本集配置好以後，要在從節點運行rs.secondaryOk()來告訴從節點已經配置好了，否則它會不允許我們讀，有點類似從節點為就緒的感覺；
在從節點上運行rs.secondaryOk(),然後再次查看數據
test_replset:SECONDARY> rs.secondaryOk()
test_replset:SECONDARY> show dbs
admin 0.000GB
config 0.000GB
local 0.000GB
students 0.000GB
test_replset:SECONDARY> use students
switched to db students
test_replset:SECONDARY> show collections
student_info
test_replset:SECONDARY> db.student_info.find().pretty()
{
"_id" : ObjectId("5faa89297077300f4fc31d1c"),
"name" : "tom",
"age" : 18,
"gender" : "M"
}
test_replset:SECONDARY>
提示：可以看到在secondary節點上能夠查詢到對應數據庫中的對應collection中插入的數據；
驗證：在從節點上插入數據，看看是否能插入成功？
test_replset:SECONDARY> db.student_info.insert({name:"jerry",age:19,gender:"M"})
WriteCommandError({
"topologyVersion" : {
"processId" : ObjectId("5faa831bde03c4ae108f0be4"),
"counter" : NumberLong(3)
},
"operationTime" : Timestamp(1605012904, 1),
"ok" : 0,
"errmsg" : "not master",
"code" : 10107,
"codeName" : "NotMaster",
"$clusterTime" : {
"clusterTime" : Timestamp(1605012904, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
})
test_replset:SECONDARY>
提示：我們在從節點上插入數據，它提示我們不是主節點，不允許插入數據；這說明副本集的從節點是不允許寫；
驗證：將主節點上的mongodb服務停掉，然後在其他兩個節點上查看副本集狀態，看看兩個節點是否重新選舉了master?
[root@node01 ~]# systemctl stop mongod.service
[root@node01 ~]# ss -tnl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
[root@node01 ~]# mongo --host node02
MongoDB shell version v4.4.1
connecting to: mongodb://node02:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("90d60f76-fc95-44a8-bbf7-d927d5947331") }
MongoDB server version: 4.4.1
---
The server generated these startup warnings when booting:
2020-11-10T20:09:39.555+08:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
2020-11-10T20:09:39.555+08:00: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never'
2020-11-10T20:09:39.555+08:00: /sys/kernel/mm/transparent_hugepage/defrag is 'always'. We suggest setting it to 'never'
---
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).
The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.
To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---
test_replset:PRIMARY> rs.status()
{
"set" : "test_replset",
"date" : ISODate("2020-11-10T12:59:25.775Z"),
"myState" : 1,
"term" : NumberLong(2),
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"majorityVoteCount" : 2,
"writeMajorityCount" : 2,
"votingMembersCount" : 3,
"writableVotingMembersCount" : 3,
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"lastCommittedWallTime" : ISODate("2020-11-10T12:59:24.417Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"readConcernMajorityWallTime" : ISODate("2020-11-10T12:59:24.417Z"),
"appliedOpTime" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"durableOpTime" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"lastAppliedWallTime" : ISODate("2020-11-10T12:59:24.417Z"),
"lastDurableWallTime" : ISODate("2020-11-10T12:59:24.417Z")
},
"electionCandidateMetrics" : {
"lastElectionReason" : "stepUpRequestSkipDryRun",
"lastElectionDate" : ISODate("2020-11-10T12:58:34.398Z"),
"electionTerm" : NumberLong(2),
"lastCommittedOpTimeAtElection" : {
"ts" : Timestamp(1605013112, 1),
"t" : NumberLong(1)
},
"lastSeenOpTimeAtElection" : {
"ts" : Timestamp(1605013112, 1),
"t" : NumberLong(1)
},
"numVotesNeeded" : 2,
"priorityAtElection" : 1,
"electionTimeoutMillis" : NumberLong(10000),
"priorPrimaryMemberId" : 0,
"numCatchUpOps" : NumberLong(0),
"newTermStartDate" : ISODate("2020-11-10T12:58:34.410Z"),
"wMajorityWriteAvailabilityDate" : ISODate("2020-11-10T12:58:34.506Z")
},
"members" : [
{
"_id" : 0,
"name" : "node01.test.org:27017",
"health" : 0,
"state" : 8,
"stateStr" : "(not reachable/healthy)",
"uptime" : 0,
"optime" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDurable" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
"optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
"lastHeartbeat" : ISODate("2020-11-10T12:59:24.558Z"),
"lastHeartbeatRecv" : ISODate("2020-11-10T12:58:35.045Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "Error connecting to node01.test.org:27017 (192.168.0.41:27017) :: caused by :: Connection refused",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"configVersion" : 3,
"configTerm" : 1
},
{
"_id" : 1,
"name" : "node02:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 2987,
"optime" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2020-11-10T12:59:24Z"),
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"electionTime" : Timestamp(1605013114, 1),
"electionDate" : ISODate("2020-11-10T12:58:34Z"),
"configVersion" : 3,
"configTerm" : 2,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 2,
"name" : "node03:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 1914,
"optime" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2020-11-10T12:59:24Z"),
"optimeDurableDate" : ISODate("2020-11-10T12:59:24Z"),
"lastHeartbeat" : ISODate("2020-11-10T12:59:24.459Z"),
"lastHeartbeatRecv" : ISODate("2020-11-10T12:59:25.506Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncSourceHost" : "node02:27017",
"syncSourceId" : 1,
"infoMessage" : "",
"configVersion" : 3,
"configTerm" : 2
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605013164, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605013164, 1)
}
test_replset:PRIMARY>
提示：可以看到當node01上的mongodb服務停掉以後，對應的主節點也就不能夠正常的向從節點發送心跳，所以剩下的兩個從節點都認為主節點掛了，此時他倆就開始重新選舉主節點；我們再次連接到node02上的mongodb查看副本集狀態時，此時node02上的mongodb就成為了主節點；同時我們也能看到node01的健康狀態為0，表示不健康；
查看從節點的複製信息
test_replset:PRIMARY> rs.printSecondaryReplicationInfo()
source: node01.test.org:27017
syncedTo: Thu Jan 01 1970 08:00:00 GMT+0800 (CST)
1605013534 secs (445837.09 hrs) behind the primary
source: node03:27017
syncedTo: Tue Nov 10 2020 21:05:34 GMT+0800 (CST)
0 secs (0 hrs) behind the primary
test_replset:PRIMARY>
提示：這裡可以看到node01落後主節點很長時間，node03不落後主節點；
查看同步信息時間窗口，以及oplog大小
test_replset:PRIMARY> rs.printReplicationInfo() configured oplog size: 2048MB log length start to end: 2291secs (0.64hrs) oplog first event time: Tue Nov 10 2020 20:27:23 GMT+0800 (CST) oplog last event time: Tue Nov 10 2020 21:05:34 GMT+0800 (CST) now: Tue Nov 10 2020 21:05:35 GMT+0800 (CST) test_replset:PRIMARY>
提示：可以看到oplog的小為2048MB，日誌開始到結束的時間窗口為2291秒，意思是說當前啟動副本集到查詢複製信息是的時長為2291秒；第一個oplog時間時間和最後oplog事件時間以及現在的時間都可查看得到；
移除節點（從副本集中，將node01節點移除）
test_replset:PRIMARY> rs.remove("node01.test.org:27017")
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605014063, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605014063, 1)
}
test_replset:PRIMARY> rs.status()
{
"set" : "test_replset",
"date" : ISODate("2020-11-10T13:14:35.362Z"),
"myState" : 1,
"term" : NumberLong(2),
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"majorityVoteCount" : 2,
"writeMajorityCount" : 2,
"votingMembersCount" : 2,
"writableVotingMembersCount" : 2,
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1605014074, 1),
"t" : NumberLong(2)
},
"lastCommittedWallTime" : ISODate("2020-11-10T13:14:34.514Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1605014074, 1),
"t" : NumberLong(2)
},
"readConcernMajorityWallTime" : ISODate("2020-11-10T13:14:34.514Z"),
"appliedOpTime" : {
"ts" : Timestamp(1605014074, 1),
"t" : NumberLong(2)
},
"durableOpTime" : {
"ts" : Timestamp(1605014074, 1),
"t" : NumberLong(2)
},
"lastAppliedWallTime" : ISODate("2020-11-10T13:14:34.514Z"),
"lastDurableWallTime" : ISODate("2020-11-10T13:14:34.514Z")
},
"electionCandidateMetrics" : {
"lastElectionReason" : "stepUpRequestSkipDryRun",
"lastElectionDate" : ISODate("2020-11-10T12:58:34.398Z"),
"electionTerm" : NumberLong(2),
"lastCommittedOpTimeAtElection" : {
"ts" : Timestamp(1605013112, 1),
"t" : NumberLong(1)
},
"lastSeenOpTimeAtElection" : {
"ts" : Timestamp(1605013112, 1),
"t" : NumberLong(1)
},
"numVotesNeeded" : 2,
"priorityAtElection" : 1,
"electionTimeoutMillis" : NumberLong(10000),
"priorPrimaryMemberId" : 0,
"numCatchUpOps" : NumberLong(0),
"newTermStartDate" : ISODate("2020-11-10T12:58:34.410Z"),
"wMajorityWriteAvailabilityDate" : ISODate("2020-11-10T12:58:34.506Z")
},
"members" : [
{
"_id" : 1,
"name" : "node02:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 3897,
"optime" : {
"ts" : Timestamp(1605014074, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2020-11-10T13:14:34Z"),
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"electionTime" : Timestamp(1605013114, 1),
"electionDate" : ISODate("2020-11-10T12:58:34Z"),
"configVersion" : 4,
"configTerm" : 2,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 2,
"name" : "node03:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 2824,
"optime" : {
"ts" : Timestamp(1605014063, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1605014063, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2020-11-10T13:14:23Z"),
"optimeDurableDate" : ISODate("2020-11-10T13:14:23Z"),
"lastHeartbeat" : ISODate("2020-11-10T13:14:33.958Z"),
"lastHeartbeatRecv" : ISODate("2020-11-10T13:14:33.970Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncSourceHost" : "node02:27017",
"syncSourceId" : 1,
"infoMessage" : "",
"configVersion" : 4,
"configTerm" : 2
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605014074, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605014074, 1)
}
test_replset:PRIMARY>
提示：移除節點需要寫明對應節點的主機名地址和端口，或者是ip地址端口；這個字串必須是同加入到副本集中的名字相同才可移除，否則它會提示我們給定主機字符串找不到；
設置node03的優先級為10
test_replset:PRIMARY> cfg = rs.conf()
{
"_id" : "test_replset",
"version" : 4,
"term" : 2,
"protocolVersion" : NumberLong(1),
"writeConcernMajorityJournalDefault" : true,
"members" : [
{
"_id" : 1,
"host" : "node02:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 2,
"host" : "node03:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : -1,
"catchUpTakeoverDelayMillis" : 30000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("5faa85853d5a2d8fdf8af85e")
}
}
test_replset:PRIMARY> cfg.members[1].priority = 10
10
test_replset:PRIMARY> rs.reconfig(cfg)
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605015945, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605015945, 1)
}
test_replset:PRIMARY> rs.config()
{
"_id" : "test_replset",
"version" : 5,
"term" : 3,
"protocolVersion" : NumberLong(1),
"writeConcernMajorityJournalDefault" : true,
"members" : [
{
"_id" : 1,
"host" : "node02:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 2,
"host" : "node03:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 10,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : -1,
"catchUpTakeoverDelayMillis" : 30000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("5faa85853d5a2d8fdf8af85e")
}
}
test_replset:SECONDARY>
提示：設置某個成員的優先級，首先要把當前配置用一個變量保存，然後通過修改保存的配置變量來修改優先級以後，再使用rs.reconfig()來讀取修改後的配置即可生效；這裡需要注意我在修改配置文件時指定成員，是指定對應成員在members列表的下標，而非id；所以修改前要先確定對應成員的下標是多少；我們修改了node03的優先級以後，心細的你一定會發現node02此時就變成了secondary;這意味着，只要在副本集中有更高優先級的成員加入，會觸發一次選舉，當然選舉成為主節點，影響最大的就是各節點的優先級，優先級越高，當選主節點的幾率就越大；默認情況每個成員的優先級都為1，優先級設定的取值範圍是0-1000；
將node01上的mongodb服務啟動起來，然後在主節點上將其添加為仲裁節點
啟動node01上的mongodb服務
[root@node01 ~]# systemctl start mongod.service [root@node01 ~]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 *:22 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 128 *:27017 *:* LISTEN 0 128 :::22 :::* LISTEN 0 100 ::1:25 :::* [root@node01 ~]#
在主節點上將node01添加為仲裁節點
test_replset:PRIMARY> rs.addArb("node01")
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605016848, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605016848, 1)
}
test_replset:PRIMARY>
查看配置信息，看看node01上的arbiterOnly是否變成了true?
test_replset:PRIMARY> rs.config()
{
"_id" : "test_replset",
"version" : 8,
"term" : 3,
"protocolVersion" : NumberLong(1),
"writeConcernMajorityJournalDefault" : true,
"members" : [
{
"_id" : 1,
"host" : "node02:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 2,
"host" : "node03:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 10,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 3,
"host" : "node01:27017",
"arbiterOnly" : true,
"buildIndexes" : true,
"hidden" : false,
"priority" : 0,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : -1,
"catchUpTakeoverDelayMillis" : 30000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("5faa85853d5a2d8fdf8af85e")
}
}
test_replset:PRIMARY>
提示：可以看到node01節點上的arbiterOnly屬性已經啟用，說明這就是一個仲裁節點；我們說過仲裁節點上沒有保存副本，我們到node01上看看是否保存的有副本集數據呢？
test_replset:ARBITER> show databases
uncaught exception: Error: listDatabases failed:{
"topologyVersion" : {
"processId" : ObjectId("5faa9d87e504fa222c4a584d"),
"counter" : NumberLong(1)
},
"ok" : 0,
"errmsg" : "not master and slaveOk=false",
"code" : 13435,
"codeName" : "NotMasterNoSlaveOk"
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs/<@src/mongo/shell/mongo.js:147:19
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:99:12
shellHelper.show@src/mongo/shell/utils.js:937:13
shellHelper@src/mongo/shell/utils.js:819:15
@(shellhelp2):1:1
test_replset:ARBITER> rs.secondaryOk()
test_replset:ARBITER> show databases
uncaught exception: Error: listDatabases failed:{
"topologyVersion" : {
"processId" : ObjectId("5faa9d87e504fa222c4a584d"),
"counter" : NumberLong(1)
},
"ok" : 0,
"errmsg" : "node is not in primary or recovering state",
"code" : 13436,
"codeName" : "NotMasterOrSecondary"
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs/<@src/mongo/shell/mongo.js:147:19
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:99:12
shellHelper.show@src/mongo/shell/utils.js:937:13
shellHelper@src/mongo/shell/utils.js:819:15
@(shellhelp2):1:1
test_replset:ARBITER>
提示：我們在仲裁節點上查看數據庫列表，它提示我們不是主節點和從節點，不允許讀；既然都不允許讀，寫肯定是沒有辦法進行；
ok,到此mongodb的副本集的配置、測試就到此為止；
