|
rpm-build |
3ee90c |
:compat-mode: legacy
|
|
rpm-build |
3ee90c |
= Replicate Storage Using DRBD =
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Even if you're serving up static websites, having to manually synchronize
|
|
rpm-build |
3ee90c |
the contents of that website to all the machines in the cluster is not
|
|
rpm-build |
3ee90c |
ideal. For dynamic websites, such as a wiki, it's not even an option. Not
|
|
rpm-build |
3ee90c |
everyone care afford network-attached storage, but somehow the data needs
|
|
rpm-build |
3ee90c |
to be kept in sync.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Enter DRBD, which can be thought of as network-based RAID-1.
|
|
rpm-build |
3ee90c |
footnote:[See http://www.drbd.org/ for details.]
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
== Install the DRBD Packages ==
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
DRBD itself is included in the upstream kernel,footnote:[Since version 2.6.33]
|
|
rpm-build |
3ee90c |
but we do need some utilities to use it effectively.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
CentOS does not ship these utilities, so we need to enable a third-party
|
|
rpm-build |
3ee90c |
repository to get them. Supported packages for many OSes are available from
|
|
rpm-build |
3ee90c |
DRBD's maker http://www.linbit.com/[LINBIT], but here we'll use the free
|
|
rpm-build |
3ee90c |
http://elrepo.org/[ELRepo] repository.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
On both nodes, import the ELRepo package signing key, and enable the
|
|
rpm-build |
3ee90c |
repository:
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
# rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
|
|
rpm-build |
3ee90c |
# rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
|
|
rpm-build |
3ee90c |
Retrieving http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
|
|
rpm-build |
3ee90c |
Preparing... ################################# [100%]
|
|
rpm-build |
3ee90c |
Updating / installing...
|
|
rpm-build |
3ee90c |
1:elrepo-release-7.0-3.el7.elrepo ################################# [100%]
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Now, we can install the DRBD kernel module and utilities:
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
# yum install -y kmod-drbd84 drbd84-utils
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
DRBD will not be able to run under the default SELinux security policies.
|
|
rpm-build |
3ee90c |
If you are familiar with SELinux, you can modify the policies in a more
|
|
rpm-build |
3ee90c |
fine-grained manner, but here we will simply exempt DRBD processes from SELinux
|
|
rpm-build |
3ee90c |
control:
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
# semanage permissive -a drbd_t
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
We will configure DRBD to use port 7789, so allow that port from each host to
|
|
rpm-build |
3ee90c |
the other:
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# firewall-cmd --permanent --add-rich-rule='rule family="ipv4" \
|
|
rpm-build |
3ee90c |
source address="192.168.122.102" port port="7789" protocol="tcp" accept'
|
|
rpm-build |
3ee90c |
success
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# firewall-cmd --reload
|
|
rpm-build |
3ee90c |
success
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-2 ~]# firewall-cmd --permanent --add-rich-rule='rule family="ipv4" \
|
|
rpm-build |
3ee90c |
source address="192.168.122.101" port port="7789" protocol="tcp" accept'
|
|
rpm-build |
3ee90c |
success
|
|
rpm-build |
3ee90c |
[root@pcmk-2 ~]# firewall-cmd --reload
|
|
rpm-build |
3ee90c |
success
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
[NOTE]
|
|
rpm-build |
3ee90c |
======
|
|
rpm-build |
3ee90c |
In this example, we have only two nodes, and all network traffic is on the same LAN.
|
|
rpm-build |
3ee90c |
In production, it is recommended to use a dedicated, isolated network for cluster-related traffic,
|
|
rpm-build |
3ee90c |
so the firewall configuration would likely be different; one approach would be to
|
|
rpm-build |
3ee90c |
add the dedicated network interfaces to the trusted zone.
|
|
rpm-build |
3ee90c |
======
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
== Allocate a Disk Volume for DRBD ==
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
DRBD will need its own block device on each node. This can be
|
|
rpm-build |
3ee90c |
a physical disk partition or logical volume, of whatever size
|
|
rpm-build |
3ee90c |
you need for your data. For this document, we will use a 512MiB logical volume,
|
|
rpm-build |
3ee90c |
which is more than sufficient for a single HTML file and (later) GFS2 metadata.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# vgdisplay | grep -e Name -e Free
|
|
rpm-build |
3ee90c |
VG Name centos_pcmk-1
|
|
rpm-build |
3ee90c |
Free PE / Size 255 / 1020.00 MiB
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# lvcreate --name drbd-demo --size 512M centos_pcmk-1
|
|
rpm-build |
3ee90c |
Logical volume "drbd-demo" created.
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# lvs
|
|
rpm-build |
3ee90c |
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
|
|
rpm-build |
3ee90c |
drbd-demo centos_pcmk-1 -wi-a----- 512.00m
|
|
rpm-build |
3ee90c |
root centos_pcmk-1 -wi-ao---- 3.00g
|
|
rpm-build |
3ee90c |
swap centos_pcmk-1 -wi-ao---- 1.00g
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Repeat for the second node, making sure to use the same size:
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# ssh pcmk-2 -- lvcreate --name drbd-demo --size 512M centos_pcmk-2
|
|
rpm-build |
3ee90c |
Logical volume "drbd-demo" created.
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
== Configure DRBD ==
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
There is no series of commands for building a DRBD configuration, so simply
|
|
rpm-build |
3ee90c |
run this on both nodes to use this sample configuration:
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
# cat <<END >/etc/drbd.d/wwwdata.res
|
|
rpm-build |
3ee90c |
resource wwwdata {
|
|
rpm-build |
3ee90c |
protocol C;
|
|
rpm-build |
3ee90c |
meta-disk internal;
|
|
rpm-build |
3ee90c |
device /dev/drbd1;
|
|
rpm-build |
3ee90c |
syncer {
|
|
rpm-build |
3ee90c |
verify-alg sha1;
|
|
rpm-build |
3ee90c |
}
|
|
rpm-build |
3ee90c |
net {
|
|
rpm-build |
3ee90c |
allow-two-primaries;
|
|
rpm-build |
3ee90c |
}
|
|
rpm-build |
3ee90c |
on pcmk-1 {
|
|
rpm-build |
3ee90c |
disk /dev/centos_pcmk-1/drbd-demo;
|
|
rpm-build |
3ee90c |
address 192.168.122.101:7789;
|
|
rpm-build |
3ee90c |
}
|
|
rpm-build |
3ee90c |
on pcmk-2 {
|
|
rpm-build |
3ee90c |
disk /dev/centos_pcmk-2/drbd-demo;
|
|
rpm-build |
3ee90c |
address 192.168.122.102:7789;
|
|
rpm-build |
3ee90c |
}
|
|
rpm-build |
3ee90c |
}
|
|
rpm-build |
3ee90c |
END
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
[IMPORTANT]
|
|
rpm-build |
3ee90c |
=========
|
|
rpm-build |
3ee90c |
Edit the file to use the hostnames, IP addresses and logical volume paths
|
|
rpm-build |
3ee90c |
of your nodes if they differ from the ones used in this guide.
|
|
rpm-build |
3ee90c |
=========
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
[NOTE]
|
|
rpm-build |
3ee90c |
=======
|
|
rpm-build |
3ee90c |
Detailed information on the directives used in this configuration (and
|
|
rpm-build |
3ee90c |
other alternatives) is available in the
|
|
rpm-build |
3ee90c |
https://docs.linbit.com/docs/users-guide-8.4/#ch-configure[DRBD User's Guide].
|
|
rpm-build |
3ee90c |
The *allow-two-primaries* option would not normally be used in
|
|
rpm-build |
3ee90c |
an active/passive cluster. We are adding it here for the convenience
|
|
rpm-build |
3ee90c |
of changing to an active/active cluster later.
|
|
rpm-build |
3ee90c |
=======
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
== Initialize DRBD ==
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
With the configuration in place, we can now get DRBD running.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
These commands create the local metadata for the DRBD resource,
|
|
rpm-build |
3ee90c |
ensure the DRBD kernel module is loaded, and bring up the DRBD resource.
|
|
rpm-build |
3ee90c |
Run them on one node:
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# drbdadm create-md wwwdata
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
--== Thank you for participating in the global usage survey ==--
|
|
rpm-build |
3ee90c |
The server's response is:
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
you are the 2147th user to install this version
|
|
rpm-build |
3ee90c |
initializing activity log
|
|
rpm-build |
3ee90c |
initializing bitmap (16 KB) to all zero
|
|
rpm-build |
3ee90c |
Writing meta data...
|
|
rpm-build |
3ee90c |
New drbd meta data block successfully created.
|
|
rpm-build |
3ee90c |
success
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# modprobe drbd
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# drbdadm up wwwdata
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
--== Thank you for participating in the global usage survey ==--
|
|
rpm-build |
3ee90c |
The server's response is:
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
We can confirm DRBD's status on this node:
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# cat /proc/drbd
|
|
rpm-build |
3ee90c |
version: 8.4.11-1 (api:1/proto:86-101)
|
|
rpm-build |
3ee90c |
GIT-hash: 66145a308421e9c124ec391a7848ac20203bb03c build by mockbuild@, 2018-04-26 12:10:42
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
1: cs:WFConnection ro:Secondary/Unknown ds:Inconsistent/DUnknown C r----s
|
|
rpm-build |
3ee90c |
ns:0 nr:0 dw:0 dr:0 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:524236
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Because we have not yet initialized the data, this node's data
|
|
rpm-build |
3ee90c |
is marked as *Inconsistent*. Because we have not yet initialized
|
|
rpm-build |
3ee90c |
the second node, the local state is *WFConnection* (waiting for connection),
|
|
rpm-build |
3ee90c |
and the partner node's status is marked as *Unknown*.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Now, repeat the above commands on the second node, starting with creating
|
|
rpm-build |
3ee90c |
wwwdata.res. After giving it time to connect, when we check the status, it
|
|
rpm-build |
3ee90c |
shows:
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-2 ~]# cat /proc/drbd
|
|
rpm-build |
3ee90c |
version: 8.4.11-1 (api:1/proto:86-101)
|
|
rpm-build |
3ee90c |
GIT-hash: 66145a308421e9c124ec391a7848ac20203bb03c build by mockbuild@, 2018-04-26 12:10:42
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
1: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r-----
|
|
rpm-build |
3ee90c |
ns:0 nr:0 dw:0 dr:0 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:524236
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
You can see the state has changed to *Connected*, meaning the two DRBD nodes
|
|
rpm-build |
3ee90c |
are communicating properly, and both nodes are in *Secondary* role
|
|
rpm-build |
3ee90c |
with *Inconsistent* data.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
To make the data consistent, we need to tell DRBD which node should be
|
|
rpm-build |
3ee90c |
considered to have the correct data. In this case, since we are creating
|
|
rpm-build |
3ee90c |
a new resource, both have garbage, so we'll just pick pcmk-1
|
|
rpm-build |
3ee90c |
and run this command on it:
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# drbdadm primary --force wwwdata
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
[NOTE]
|
|
rpm-build |
3ee90c |
======
|
|
rpm-build |
3ee90c |
If you are using a different version of DRBD, the required syntax may be different.
|
|
rpm-build |
3ee90c |
See the documentation for your version for how to perform these commands.
|
|
rpm-build |
3ee90c |
======
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
If we check the status immediately, we'll see something like this:
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# cat /proc/drbd
|
|
rpm-build |
3ee90c |
version: 8.4.11-1 (api:1/proto:86-101)
|
|
rpm-build |
3ee90c |
GIT-hash: 66145a308421e9c124ec391a7848ac20203bb03c build by mockbuild@, 2018-04-26 12:10:42
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
1: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
|
|
rpm-build |
3ee90c |
ns:43184 nr:0 dw:0 dr:45312 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:481052
|
|
rpm-build |
3ee90c |
[>...................] sync'ed: 8.6% (481052/524236)K
|
|
rpm-build |
3ee90c |
finish: 0:01:51 speed: 4,316 (4,316) K/sec
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
We can see that this node has the *Primary* role, the partner node has
|
|
rpm-build |
3ee90c |
the *Secondary* role, this node's data is now considered *UpToDate*,
|
|
rpm-build |
3ee90c |
the partner node's data is still *Inconsistent*, and a progress bar
|
|
rpm-build |
3ee90c |
shows how far along the partner node is in synchronizing the data.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
After a while, the sync should finish, and you'll see something like:
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# cat /proc/drbd
|
|
rpm-build |
3ee90c |
version: 8.4.11-1 (api:1/proto:86-101)
|
|
rpm-build |
3ee90c |
GIT-hash: 66145a308421e9c124ec391a7848ac20203bb03c build by mockbuild@, 2018-04-26 12:10:42
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
1: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
|
|
rpm-build |
3ee90c |
ns:524236 nr:0 dw:0 dr:526364 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Both sets of data are now *UpToDate*, and we can proceed to creating
|
|
rpm-build |
3ee90c |
and populating a filesystem for our WebSite resource's documents.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
== Populate the DRBD Disk ==
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
On the node with the primary role (pcmk-1 in this example),
|
|
rpm-build |
3ee90c |
create a filesystem on the DRBD device:
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# mkfs.xfs /dev/drbd1
|
|
rpm-build |
3ee90c |
meta-data=/dev/drbd1 isize=512 agcount=4, agsize=32765 blks
|
|
rpm-build |
3ee90c |
= sectsz=512 attr=2, projid32bit=1
|
|
rpm-build |
3ee90c |
= crc=1 finobt=0, sparse=0
|
|
rpm-build |
3ee90c |
data = bsize=4096 blocks=131059, imaxpct=25
|
|
rpm-build |
3ee90c |
= sunit=0 swidth=0 blks
|
|
rpm-build |
3ee90c |
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
|
|
rpm-build |
3ee90c |
log =internal log bsize=4096 blocks=855, version=2
|
|
rpm-build |
3ee90c |
= sectsz=512 sunit=0 blks, lazy-count=1
|
|
rpm-build |
3ee90c |
realtime =none extsz=4096 blocks=0, rtextents=0
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
[NOTE]
|
|
rpm-build |
3ee90c |
====
|
|
rpm-build |
3ee90c |
In this example, we create an xfs filesystem with no special options.
|
|
rpm-build |
3ee90c |
In a production environment, you should choose a filesystem type and
|
|
rpm-build |
3ee90c |
options that are suitable for your application.
|
|
rpm-build |
3ee90c |
====
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Mount the newly created filesystem, populate it with our web document,
|
|
rpm-build |
3ee90c |
give it the same SELinux policy as the web document root,
|
|
rpm-build |
3ee90c |
then unmount it (the cluster will handle mounting and unmounting it later):
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# mount /dev/drbd1 /mnt
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# cat <<-END >/mnt/index.html
|
|
rpm-build |
3ee90c |
<html>
|
|
rpm-build |
3ee90c |
<body>My Test Site - DRBD</body>
|
|
rpm-build |
3ee90c |
</html>
|
|
rpm-build |
3ee90c |
END
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# chcon -R --reference=/var/www/html /mnt
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# umount /dev/drbd1
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
== Configure the Cluster for the DRBD device ==
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
One handy feature `pcs` has is the ability to queue up several changes
|
|
rpm-build |
3ee90c |
into a file and commit those changes all at once. To do this, start by
|
|
rpm-build |
3ee90c |
populating the file with the current raw XML config from the CIB.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs cluster cib drbd_cfg
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Using pcs's `-f` option, make changes to the configuration saved
|
|
rpm-build |
3ee90c |
in the +drbd_cfg+ file. These changes will not be seen by the cluster until
|
|
rpm-build |
3ee90c |
the +drbd_cfg+ file is pushed into the live cluster's CIB later.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Here, we create a cluster resource for the DRBD device, and an additional _clone_
|
|
rpm-build |
3ee90c |
resource to allow the resource to run on both nodes at the same time.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs -f drbd_cfg resource create WebData ocf:linbit:drbd \
|
|
rpm-build |
3ee90c |
drbd_resource=wwwdata op monitor interval=60s
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs -f drbd_cfg resource master WebDataClone WebData \
|
|
rpm-build |
3ee90c |
master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 \
|
|
rpm-build |
3ee90c |
notify=true
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs -f drbd_cfg resource show
|
|
rpm-build |
3ee90c |
ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-1
|
|
rpm-build |
3ee90c |
WebSite (ocf::heartbeat:apache): Started pcmk-1
|
|
rpm-build |
3ee90c |
Master/Slave Set: WebDataClone [WebData]
|
|
rpm-build |
3ee90c |
Stopped: [ pcmk-1 pcmk-2 ]
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
.Note
|
|
rpm-build |
3ee90c |
[NOTE]
|
|
rpm-build |
3ee90c |
====
|
|
rpm-build |
3ee90c |
In Fedora 29 and CentOS 8.0, 'master' resources have been renamed to
|
|
rpm-build |
3ee90c |
'promotable clone' resources and the `pcs` command has been changed
|
|
rpm-build |
3ee90c |
accordingly:
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs -f drbd_cfg resource promotable WebData \
|
|
rpm-build |
3ee90c |
promoted-max=1 promoted-node-max=1 clone-max=2 clone-node-max=1 \
|
|
rpm-build |
3ee90c |
notify=true
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
The new command does not allow to set a custom name for the resulting
|
|
rpm-build |
3ee90c |
promotable resource. `Pcs` automatically creates a name for the resource in
|
|
rpm-build |
3ee90c |
the form of *pass:[<replaceable>resource_name</replaceable>]-clone*, that is
|
|
rpm-build |
3ee90c |
*WebData-clone* in this case.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
To avoid confusion whether the +pcs resource show+ command displays resources'
|
|
rpm-build |
3ee90c |
status or configuration, the command has been deprecated in Fedora 29 and
|
|
rpm-build |
3ee90c |
CentOS 8.0. Two new commands have been introduced for displaying resources'
|
|
rpm-build |
3ee90c |
status and configuration: `pcs resource status` and `pcs resource config`,
|
|
rpm-build |
3ee90c |
respectively.
|
|
rpm-build |
3ee90c |
====
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
After you are satisfied with all the changes, you can commit
|
|
rpm-build |
3ee90c |
them all at once by pushing the drbd_cfg file into the live CIB.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs cluster cib-push drbd_cfg --config
|
|
rpm-build |
3ee90c |
CIB updated
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Let's see what the cluster did with the new configuration:
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs status
|
|
rpm-build |
3ee90c |
Cluster name: mycluster
|
|
rpm-build |
3ee90c |
Stack: corosync
|
|
rpm-build |
3ee90c |
Current DC: pcmk-2 (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum
|
|
rpm-build |
3ee90c |
Last updated: Mon Sep 10 17:58:07 2018
|
|
rpm-build |
3ee90c |
Last change: Mon Sep 10 17:57:53 2018 by root via cibadmin on pcmk-1
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
2 nodes configured
|
|
rpm-build |
3ee90c |
4 resources configured
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Online: [ pcmk-1 pcmk-2 ]
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Full list of resources:
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-1
|
|
rpm-build |
3ee90c |
WebSite (ocf::heartbeat:apache): Started pcmk-1
|
|
rpm-build |
3ee90c |
Master/Slave Set: WebDataClone [WebData]
|
|
rpm-build |
3ee90c |
Masters: [ pcmk-1 ]
|
|
rpm-build |
3ee90c |
Slaves: [ pcmk-2 ]
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Daemon Status:
|
|
rpm-build |
3ee90c |
corosync: active/disabled
|
|
rpm-build |
3ee90c |
pacemaker: active/disabled
|
|
rpm-build |
3ee90c |
pcsd: active/enabled
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
We can see that *WebDataClone* (our DRBD device) is running as master (DRBD's
|
|
rpm-build |
3ee90c |
primary role) on *pcmk-1* and slave (DRBD's secondary role) on *pcmk-2*.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
[IMPORTANT]
|
|
rpm-build |
3ee90c |
====
|
|
rpm-build |
3ee90c |
The resource agent should load the DRBD module when needed if it's not already
|
|
rpm-build |
3ee90c |
loaded. If that does not happen, configure your operating system to load the
|
|
rpm-build |
3ee90c |
module at boot time. For &DISTRO; &DISTRO_VERSION;, you would run this on both
|
|
rpm-build |
3ee90c |
nodes:
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
# echo drbd >/etc/modules-load.d/drbd.conf
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
====
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
== Configure the Cluster for the Filesystem ==
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Now that we have a working DRBD device, we need to mount its filesystem.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
In addition to defining the filesystem, we also need to
|
|
rpm-build |
3ee90c |
tell the cluster where it can be located (only on the DRBD Primary)
|
|
rpm-build |
3ee90c |
and when it is allowed to start (after the Primary was promoted).
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
We are going to take a shortcut when creating the resource this time.
|
|
rpm-build |
3ee90c |
Instead of explicitly saying we want the *ocf:heartbeat:Filesystem* script, we
|
|
rpm-build |
3ee90c |
are only going to ask for *Filesystem*. We can do this because we know there is only
|
|
rpm-build |
3ee90c |
one resource script named *Filesystem* available to pacemaker, and that pcs is smart
|
|
rpm-build |
3ee90c |
enough to fill in the *ocf:heartbeat:* portion for us correctly in the configuration.
|
|
rpm-build |
3ee90c |
If there were multiple *Filesystem* scripts from different OCF providers, we would need
|
|
rpm-build |
3ee90c |
to specify the exact one we wanted.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Once again, we will queue our changes to a file and then push the
|
|
rpm-build |
3ee90c |
new configuration to the cluster as the final step.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs cluster cib fs_cfg
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs -f fs_cfg resource create WebFS Filesystem \
|
|
rpm-build |
3ee90c |
device="/dev/drbd1" directory="/var/www/html" fstype="xfs"
|
|
rpm-build |
3ee90c |
Assumed agent name 'ocf:heartbeat:Filesystem' (deduced from 'Filesystem')
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs -f fs_cfg constraint colocation add \
|
|
rpm-build |
3ee90c |
WebFS with WebDataClone INFINITY with-rsc-role=Master
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs -f fs_cfg constraint order \
|
|
rpm-build |
3ee90c |
promote WebDataClone then start WebFS
|
|
rpm-build |
3ee90c |
Adding WebDataClone WebFS (kind: Mandatory) (Options: first-action=promote then-action=start)
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
We also need to tell the cluster that Apache needs to run on the same
|
|
rpm-build |
3ee90c |
machine as the filesystem and that it must be active before Apache can
|
|
rpm-build |
3ee90c |
start.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs -f fs_cfg constraint colocation add WebSite with WebFS INFINITY
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs -f fs_cfg constraint order WebFS then WebSite
|
|
rpm-build |
3ee90c |
Adding WebFS WebSite (kind: Mandatory) (Options: first-action=start then-action=start)
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Review the updated configuration.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs -f fs_cfg constraint
|
|
rpm-build |
3ee90c |
Location Constraints:
|
|
rpm-build |
3ee90c |
Resource: WebSite
|
|
rpm-build |
3ee90c |
Enabled on: pcmk-1 (score:50)
|
|
rpm-build |
3ee90c |
Ordering Constraints:
|
|
rpm-build |
3ee90c |
start ClusterIP then start WebSite (kind:Mandatory)
|
|
rpm-build |
3ee90c |
promote WebDataClone then start WebFS (kind:Mandatory)
|
|
rpm-build |
3ee90c |
start WebFS then start WebSite (kind:Mandatory)
|
|
rpm-build |
3ee90c |
Colocation Constraints:
|
|
rpm-build |
3ee90c |
WebSite with ClusterIP (score:INFINITY)
|
|
rpm-build |
3ee90c |
WebFS with WebDataClone (score:INFINITY) (with-rsc-role:Master)
|
|
rpm-build |
3ee90c |
WebSite with WebFS (score:INFINITY)
|
|
rpm-build |
3ee90c |
Ticket Constraints:
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs -f fs_cfg resource show
|
|
rpm-build |
3ee90c |
ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-1
|
|
rpm-build |
3ee90c |
WebSite (ocf::heartbeat:apache): Started pcmk-1
|
|
rpm-build |
3ee90c |
Master/Slave Set: WebDataClone [WebData]
|
|
rpm-build |
3ee90c |
Masters: [ pcmk-1 ]
|
|
rpm-build |
3ee90c |
Slaves: [ pcmk-2 ]
|
|
rpm-build |
3ee90c |
WebFS (ocf::heartbeat:Filesystem): Stopped
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
After reviewing the new configuration, upload it and watch the
|
|
rpm-build |
3ee90c |
cluster put it into effect.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs cluster cib-push fs_cfg --config
|
|
rpm-build |
3ee90c |
CIB updated
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs status
|
|
rpm-build |
3ee90c |
Cluster name: mycluster
|
|
rpm-build |
3ee90c |
Stack: corosync
|
|
rpm-build |
3ee90c |
Current DC: pcmk-2 (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum
|
|
rpm-build |
3ee90c |
Last updated: Mon Sep 10 18:02:24 2018
|
|
rpm-build |
3ee90c |
Last change: Mon Sep 10 18:02:14 2018 by root via cibadmin on pcmk-1
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
2 nodes configured
|
|
rpm-build |
3ee90c |
5 resources configured
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Online: [ pcmk-1 pcmk-2 ]
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Full list of resources:
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-1
|
|
rpm-build |
3ee90c |
WebSite (ocf::heartbeat:apache): Started pcmk-1
|
|
rpm-build |
3ee90c |
Master/Slave Set: WebDataClone [WebData]
|
|
rpm-build |
3ee90c |
Masters: [ pcmk-1 ]
|
|
rpm-build |
3ee90c |
Slaves: [ pcmk-2 ]
|
|
rpm-build |
3ee90c |
WebFS (ocf::heartbeat:Filesystem): Started pcmk-1
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Daemon Status:
|
|
rpm-build |
3ee90c |
corosync: active/disabled
|
|
rpm-build |
3ee90c |
pacemaker: active/disabled
|
|
rpm-build |
3ee90c |
pcsd: active/enabled
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
== Test Cluster Failover ==
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Previously, we used `pcs cluster stop pcmk-1` to stop all cluster
|
|
rpm-build |
3ee90c |
services on *pcmk-1*, failing over the cluster resources, but there is another
|
|
rpm-build |
3ee90c |
way to safely simulate node failure.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
We can put the node into _standby mode_. Nodes in this state continue to
|
|
rpm-build |
3ee90c |
run corosync and pacemaker but are not allowed to run resources. Any resources
|
|
rpm-build |
3ee90c |
found active there will be moved elsewhere. This feature can be particularly
|
|
rpm-build |
3ee90c |
useful when performing system administration tasks such as updating packages
|
|
rpm-build |
3ee90c |
used by cluster resources.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Put the active node into standby mode, and observe the cluster move all
|
|
rpm-build |
3ee90c |
the resources to the other node. The node's status will change to indicate that
|
|
rpm-build |
3ee90c |
it can no longer host resources, and eventually all the resources will move.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs cluster standby pcmk-1
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs status
|
|
rpm-build |
3ee90c |
Cluster name: mycluster
|
|
rpm-build |
3ee90c |
Stack: corosync
|
|
rpm-build |
3ee90c |
Current DC: pcmk-2 (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum
|
|
rpm-build |
3ee90c |
Last updated: Mon Sep 10 18:04:22 2018
|
|
rpm-build |
3ee90c |
Last change: Mon Sep 10 18:03:43 2018 by root via cibadmin on pcmk-1
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
2 nodes configured
|
|
rpm-build |
3ee90c |
5 resources configured
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Node pcmk-1: standby
|
|
rpm-build |
3ee90c |
Online: [ pcmk-2 ]
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Full list of resources:
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-2
|
|
rpm-build |
3ee90c |
WebSite (ocf::heartbeat:apache): Started pcmk-2
|
|
rpm-build |
3ee90c |
Master/Slave Set: WebDataClone [WebData]
|
|
rpm-build |
3ee90c |
Masters: [ pcmk-2 ]
|
|
rpm-build |
3ee90c |
Stopped: [ pcmk-1 ]
|
|
rpm-build |
3ee90c |
WebFS (ocf::heartbeat:Filesystem): Started pcmk-2
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Daemon Status:
|
|
rpm-build |
3ee90c |
corosync: active/disabled
|
|
rpm-build |
3ee90c |
pacemaker: active/disabled
|
|
rpm-build |
3ee90c |
pcsd: active/enabled
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Once we've done everything we needed to on pcmk-1 (in this case nothing,
|
|
rpm-build |
3ee90c |
we just wanted to see the resources move), we can allow the node to be a
|
|
rpm-build |
3ee90c |
full cluster member again.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs cluster unstandby pcmk-1
|
|
rpm-build |
3ee90c |
[root@pcmk-1 ~]# pcs status
|
|
rpm-build |
3ee90c |
Cluster name: mycluster
|
|
rpm-build |
3ee90c |
Stack: corosync
|
|
rpm-build |
3ee90c |
Current DC: pcmk-2 (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum
|
|
rpm-build |
3ee90c |
Last updated: Mon Sep 10 18:05:22 2018
|
|
rpm-build |
3ee90c |
Last change: Mon Sep 10 18:05:21 2018 by root via cibadmin on pcmk-1
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
2 nodes configured
|
|
rpm-build |
3ee90c |
5 resources configured
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Online: [ pcmk-1 pcmk-2 ]
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Full list of resources:
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-2
|
|
rpm-build |
3ee90c |
WebSite (ocf::heartbeat:apache): Started pcmk-2
|
|
rpm-build |
3ee90c |
Master/Slave Set: WebDataClone [WebData]
|
|
rpm-build |
3ee90c |
Masters: [ pcmk-2 ]
|
|
rpm-build |
3ee90c |
Slaves: [ pcmk-1 ]
|
|
rpm-build |
3ee90c |
WebFS (ocf::heartbeat:Filesystem): Started pcmk-2
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Daemon Status:
|
|
rpm-build |
3ee90c |
corosync: active/disabled
|
|
rpm-build |
3ee90c |
pacemaker: active/disabled
|
|
rpm-build |
3ee90c |
pcsd: active/enabled
|
|
rpm-build |
3ee90c |
----
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
Notice that *pcmk-1* is back to the *Online* state, and that the cluster resources
|
|
rpm-build |
3ee90c |
stay where they are due to our resource stickiness settings configured earlier.
|
|
rpm-build |
3ee90c |
|
|
rpm-build |
3ee90c |
.Note
|
|
rpm-build |
3ee90c |
[NOTE]
|
|
rpm-build |
3ee90c |
====
|
|
rpm-build |
3ee90c |
Since Fedora 29 and CentOS 8.0, the commands for controlling standby mode are
|
|
rpm-build |
3ee90c |
`pcs node standby` and `pcs node unstandby`.
|
|
rpm-build |
3ee90c |
====
|