Set Up a Zookeeper Cluster
Zookeeper is a coordination service that allows for easy management of common services like naming, configuration management, synchronization and group services.
Zookeeper is bundled with Incorta. You must run at least one Zookeeper instance for each Incorta cluster of one or more servers to coordinate Zookeeper's various services. As part of a High Availability (HA) Incorta cluster where you add redundancy for each component to avoid having a single point of failure, you can set up a Zookeeper cluster to have fault tolerance for Zookeeper. You must run an odd number of Zookeeper processes on separate servers to create a Zookeeper cluster (called a quorum). For example, a three-node Zookeeper cluster can have one process or server fail and the other two servers can maintain service.
This page applies to the version of Zookeeper that is bundled with Incorta. If you use a standalone version of Zookeeper, instructions on this page may be different.
Set Up Zookeeper on a Cluster
Example servers for the Zookeeper cluster:
Server ID | Node IP | Client Port | P2P Port | Leader Election Port |
---|---|---|---|---|
1 | 1.1.1.1 | 2181 | 2888 | 3888 |
2 | 2.2.2.2 | 2181 | 2888 | 3888 |
3 | 3.3.3.3 | 2181 | 2888 | 3888 |
To set up a three-node Zookeeper cluster, perform the following steps:
- Edit each Zookeeper configuration file
(<install home>/IncortaNode/zookeeper/conf/zoo.cfg
) and add the following:
server.1=zoo-1:2888:3888server.2=zoo-2:2888:3888server.3=zoo-3:2888:3888
The required format is server.<ID>.<HOST_OR_IP>:
<P2P_PORT>:<LEADER_PORT>
. In the example, zoo-1
, zoo-2
, and zoo-3
are machine names. Use an IP address or host name instead of zoo.<ID>
.
2. Create a file in each Zookeeper data directory that contains the ID of that instance:
- On zoo-1:
$ echo 1 > myid
- On zoo-2:
$ echo 2 > myid
- On zoo-3:
$ echo 3 > myid
The zoo.cfg
file specifies the data directory by default, but if you have never started the process, you must create the directory before adding the myid
file.
- In the Cluster Managemet Console (CMC), open the Details tab for your cluster.
- In edit mode, add the Zookeeper instances in a comma-delimited list:
1.1.1.1:2181,2.2.2.2:2181,3.3.3.3:2181
. - Start Zookeeper on each node by running the
start-zookeeper.sh
command from the containing folder. Starting zookeeper creates the zookeeper.out log file in the folder from which you ran the command.
You must configure each machine hosting a Zookeeper node to allow inbound/outbound connections between all Zookeeper nodes for the client, P2P, and Leader election ports (ports 2181, 2888 and 3888). These ports must be dedicated to Zookeeper and no other processes.