zookeeper系列（一）环境安装与部署配置 有更新！
- 37套精品Java架构师高并发高性能高可用分布式集群电商缓存性能调优设计项目实战视教程 置顶！ 有更新！
- 1. 简要说明
- 2. 主机名称与IP映射
- 3. 下载解压
- 4. 相关配置
- 5. 设置myid
- 6. 启动集群
- 运行日志
- 运行状态
- 客户端连接
37套精品Java架构师高并发高性能高可用分布式集群电商缓存性能调优设计项目实战视教程 置顶！ 有更新！
1. 简要说明
ZooKeeper Distributed模式，ZooKeeper分布式模式安装（ZooKeeper集群）比较容易，这里说明一下基本要点。
首先要明确的是，ZooKeeper集群是一个独立的分布式协调服务集群，“独立”的含义就是说，如果想使用ZooKeeper实现分布式应用的协调与管理，简化协调与管理，任何分布式应用都可以使用，这就要归功于Zookeeper的数据模型（Data Model）和层次命名空间（Hierarchical Namespace）结构，详细可以参考
http://zookeeper.apache.org/doc/trunk/zookeeperOver.html。
在设计你的分布式应用协调服务时，首要的就是考虑如何组织层次命名空间。下面说明分布式模式的安装配置。
2. 主机名称与IP映射
ZooKeeper集群中具有两个关键的角色：Leader和Follower。集群中所有的结点作为一个整体对分布式应用提供服务，集群中每个结点之间都互相连接，所以，在配置的ZooKeeper集群的时候，每一个结点的host到IP地址的映射都要配置上集群中其它结点的映射信息。
因为之前hadoop环境已经有配置过，不再说明，如下：
|
[root@slave1 ~]# hostname slave1 [root@slave1 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.211.132 master 192.168.211.133 slave1 192.168.211.134 slave2 192.168.211.135 slave3 |
3. 下载解压
http://apache.fayea.com/zookeeper/current/
解压到指定目录
|
[root@slave1 boom]# mkdir /usr/program/zookeeper [root@slave1 boom]# tar -zxf zookeeper-3.4.10.tar.gz -C /usr/program/zookeeper [root@slave1 boom]# cd /usr/program/zookeeper [root@slave1 zookeeper]# ll 总用量 4 drwxr-xr-x. 10 1001 1001 4096 3月 23 19:28 zookeeper-3.4.10 [root@slave1 zookeeper]# mv zookeeper-3.4.10/* ./ |
4. 相关配置
|
tickTime=2000 dataDir=/usr/data/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1=slave1:2888:3888 server.2=slave2:2888:3888 server.3=slave3:2888:3888 |
上面配置内容可以参考下面文档：
http://zookeeper.apache.org/doc/trunk/zookeeperStarted.html#sc_RunningReplicatedZooKeeper
|
[root@slave1 zookeeper]# cd conf [root@slave1 conf]# cp zoo_sample.cfg zoo.cfg [root@slave1 conf]# vi zoo.cfg # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/usr/data/zookeeper # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1 server.1=slave1:2888:3888 server.2=slave2:2888:3888 server.3=slave3:2888:3888 |
上面已经在一台机器slave1上配置完成ZooKeeper，现在可以将该配置好的安装文件远程拷贝到集群中的各个结点对应的目录下。
|
[root@slave1 conf]# scp zoo.cfg slave1:/usr/program/zookeeper/conf/ zoo.cfg 100% 1000 1.0KB/s 00:00 [root@slave1 conf]# scp zoo.cfg slave2:/usr/program/zookeeper/conf/ zoo.cfg 100% 1000 1.0KB/s 00:00 [root@slave1 conf]# scp zoo.cfg slave3:/usr/program/zookeeper/conf/ |
5. 设置myid
在我们配置的dataDir指定的目录下面，创建一个myid文件，里面内容为一个数字，用来标识当前主机，conf/zoo.cfg文件中配置的server.X中X为什么数字，则myid文件中就输入这个数字。每台节点都配置自己的myid
slave1
|
[root@slave1 conf]# cd /usr/data/zookeeper [root@slave1 zookeeper]# echo 1 >> myid [root@slave1 zookeeper]# cat myid 1 |
slave2
|
[root@slave2 data]# cd zookeeper [root@slave2 zookeeper]# echo 2 >> myid |
slave3
|
[root@slave3 data]# cd zookeeper [root@slave3 zookeeper]# echo 3 >> myid |
6. 启动集群
在ZooKeeper集群的每个结点上，执行启动ZooKeeper服务的脚本，如下所示：
|
[root@slave1 zookeeper]# cd /usr/program/zookeeper/bin [root@slave1 bin]# ll 总用量 36 -rwxr-xr-x. 1 1001 1001 232 3月 23 18:14 README.txt -rwxr-xr-x. 1 1001 1001 1937 3月 23 18:14 zkCleanup.sh -rwxr-xr-x. 1 1001 1001 1056 3月 23 18:14 zkCli.cmd -rwxr-xr-x. 1 1001 1001 1534 3月 23 18:14 zkCli.sh -rwxr-xr-x. 1 1001 1001 1628 3月 23 18:14 zkEnv.cmd -rwxr-xr-x. 1 1001 1001 2696 3月 23 18:14 zkEnv.sh -rwxr-xr-x. 1 1001 1001 1089 3月 23 18:14 zkServer.cmd -rwxr-xr-x. 1 1001 1001 6773 3月 23 18:14 zkServer.sh [root@slave1 bin]# ./zkServer.sh start ZooKeeper JMX enabled by default Using config: /usr/program/zookeeper/bin/../conf/zoo.cfg Starting zookeeper ... STARTED |
运行日志
查看各个节点的运行日志
|
[root@slave3 bin]# ll 总用量 48 -rwxr-xr-x. 1 root root 232 5月 9 13:44 README.txt -rwxr-xr-x. 1 root root 1937 5月 9 13:44 zkCleanup.sh -rwxr-xr-x. 1 root root 1056 5月 9 13:44 zkCli.cmd -rwxr-xr-x. 1 root root 1534 5月 9 13:44 zkCli.sh -rwxr-xr-x. 1 root root 1628 5月 9 13:44 zkEnv.cmd -rwxr-xr-x. 1 root root 2696 5月 9 13:44 zkEnv.sh -rwxr-xr-x. 1 root root 1089 5月 9 13:44 zkServer.cmd -rwxr-xr-x. 1 root root 6773 5月 9 13:44 zkServer.sh -rw-r--r--. 1 root root 11539 5月 9 13:48 zookeeper.out [root@slave3 bin]# tail -500f zookeeper.out 2017-05-09 13:48:42,706 [myid:] - INFO [main:QuorumPeerConfig@134] - Reading configuration from: /usr/program/zookeeper/bin/../conf/zoo.cfg 2017-05-09 13:48:42,733 [myid:] - INFO [main:QuorumPeer$QuorumServer@167] - Resolved hostname: slave1 to address: slave1/192.168.211.133 2017-05-09 13:48:42,734 [myid:] - INFO [main:QuorumPeer$QuorumServer@167] - Resolved hostname: slave3 to address: slave3/192.168.211.135 2017-05-09 13:48:42,734 [myid:] - INFO [main:QuorumPeer$QuorumServer@167] - Resolved hostname: slave2 to address: slave2/192.168.211.134 2017-05-09 13:48:42,735 [myid:] - INFO [main:QuorumPeerConfig@396] - Defaulting to majority quorums 2017-05-09 13:48:42,739 [myid:3] - INFO [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-05-09 13:48:42,739 [myid:3] - INFO [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2017-05-09 13:48:42,739 [myid:3] - INFO [main:DatadirCleanupManager@101] - Purge task is not scheduled. 2017-05-09 13:48:42,765 [myid:3] - INFO [main:QuorumPeerMain@127] - Starting quorum peer 2017-05-09 13:48:42,777 [myid:3] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181 2017-05-09 13:48:42,804 [myid:3] - INFO [main:QuorumPeer@1134] - minSessionTimeout set to -1 2017-05-09 13:48:42,804 [myid:3] - INFO [main:QuorumPeer@1145] - maxSessionTimeout set to -1 2017-05-09 13:48:42,805 [myid:3] - INFO [main:QuorumPeer@1419] - QuorumPeer communication is not secured! 2017-05-09 13:48:42,805 [myid:3] - INFO [main:QuorumPeer@1448] - quorum.cnxn.threads.size set to 20 2017-05-09 13:48:42,809 [myid:3] - INFO [main:QuorumPeer@624] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-05-09 13:48:42,813 [myid:3] - INFO [main:QuorumPeer@639] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation 2017-05-09 13:48:42,828 [myid:3] - INFO [ListenerThread:QuorumCnxManager$Listener@739] - My election bind port: slave3/192.168.211.135:3888 2017-05-09 13:48:42,839 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:QuorumPeer@865] - LOOKING 2017-05-09 13:48:42,839 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@818] - New election. My id = 3, proposed zxid=0x0 2017-05-09 13:48:42,854 [myid:3] - WARN [WorkerSender[myid=3]:QuorumCnxManager@588] - Cannot open channel to 1 at election address slave1/192.168.211.133:3888 java.net.ConnectException: 拒绝连接 (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:562) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433) at java.lang.Thread.run(Thread.java:748) 2017-05-09 13:48:42,859 [myid:3] - INFO [WorkerSender[myid=3]:QuorumPeer$QuorumServer@167] - Resolved hostname: slave1 to address: slave1/192.168.211.133 2017-05-09 13:48:42,860 [myid:3] - WARN [WorkerSender[myid=3]:QuorumCnxManager@588] - Cannot open channel to 2 at election address slave2/192.168.211.134:3888 java.net.ConnectException: 拒绝连接 (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:562) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:538) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433) at java.lang.Thread.run(Thread.java:748) 2017-05-09 13:48:42,861 [myid:3] - INFO [WorkerSender[myid=3]:QuorumPeer$QuorumServer@167] - Resolved hostname: slave2 to address: slave2/192.168.211.134 2017-05-09 13:48:42,862 [myid:3] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2017-05-09 13:48:42,893 [myid:3] - INFO [slave3/192.168.211.135:3888:QuorumCnxManager$Listener@746] - Received connection request /192.168.211.133:38031 2017-05-09 13:48:42,898 [myid:3] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2017-05-09 13:48:42,900 [myid:3] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2017-05-09 13:48:42,906 [myid:3] - INFO [slave3/192.168.211.135:3888:QuorumCnxManager$Listener@746] - Received connection request /192.168.211.134:41879 2017-05-09 13:48:42,916 [myid:3] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2017-05-09 13:48:42,918 [myid:3] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state) 2017-05-09 13:48:43,119 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:QuorumPeer@947] - LEADING 2017-05-09 13:48:43,123 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Leader@62] - TCP NoDelay set to: true 2017-05-09 13:48:43,135 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT 2017-05-09 13:48:43,135 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:host.name=slave3 2017-05-09 13:48:43,136 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.version=1.8.0_131 2017-05-09 13:48:43,136 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.vendor=Oracle Corporation 2017-05-09 13:48:43,136 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.home=/usr/program/jdk/jre 2017-05-09 13:48:43,138 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.class.path=/usr/program/zookeeper/bin/../build/classes:/usr/program/zookeeper/bin/../build/lib/*.jar:/usr/program/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/program/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/usr/program/zookeeper/bin/../lib/netty-3.10.5.Final.jar:/usr/program/zookeeper/bin/../lib/log4j-1.2.16.jar:/usr/program/zookeeper/bin/../lib/jline-0.9.94.jar:/usr/program/zookeeper/bin/../zookeeper-3.4.10.jar:/usr/program/zookeeper/bin/../src/java/lib/*.jar:/usr/program/zookeeper/bin/../conf: 2017-05-09 13:48:43,139 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2017-05-09 13:48:43,139 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.io.tmpdir=/tmp 2017-05-09 13:48:43,139 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.compiler=<NA> 2017-05-09 13:48:43,139 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:os.name=Linux 2017-05-09 13:48:43,139 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:os.arch=amd64 2017-05-09 13:48:43,140 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:os.version=2.6.32-431.el6.x86_64 2017-05-09 13:48:43,140 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:user.name=root 2017-05-09 13:48:43,140 [myid:3] - INFO [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:user.home=/root |
由于ZooKeeper集群启动的时候，每个结点都试图去连接集群中的其它结点，先启动的肯定连不上后面还没启动的，所以上面日志前面部分的异常是可以忽略的。通过后面部分可以看到，集群在选出一个Leader后，最后稳定了。
运行状态
查看各个节点的运行状态
|
[root@slave1 bin]# ./zkServer.sh status ZooKeeper JMX enabled by default Using config: /usr/program/zookeeper/bin/../conf/zoo.cfg Mode: follower |
|
[root@slave2 bin]# ./zkServer.sh status ZooKeeper JMX enabled by default Using config: /usr/program/zookeeper/bin/../conf/zoo.cfg Mode: follower |
|
[root@slave3 bin]# ./zkServer.sh status ZooKeeper JMX enabled by default Using config: /usr/program/zookeeper/bin/../conf/zoo.cfg Mode: leader |
通过上面状态查询结果可见，slave3是集群的Leader，其余的两个结点是Follower。
客户端连接
可以通过客户端脚本，连接到ZooKeeper集群上。对于客户端来说，ZooKeeper是一个整体（ensemble），连接到ZooKeeper集群实际上感觉在独享整个集群的服务，所以，你可以在任何一个结点上建立到服务集群的连接。
例如：上面的leader是slave3，但是我们的客户端也可以连到slave2上。
|
[root@slave1 bin]# ./zkCli.sh -server slave2:2181 Connecting to slave2:2181 。。。 WatchedEvent state:SyncConnected type:None path:null [zk: slave2:2181(CONNECTED) 0] ls / [zookeeper] |
当前根路径为/zookeeper。
评论
发表评论
|
|
|