一、环境说明：
测试环境使用虚拟机2台，OS：centos6.4。内核：2.6.32-358.el6.x86_64
系统安装采用最小化安装。
主机名和IP
| hostname |
ip address |
remarks |
| pg1-rg |
172.26.184.38 |
primary |
| pg2-rg |
172.26.184.39 |
secondary |
| 172.26.184.42 |
VIP |
Disk
2台server都是2块磁盘。xvda做操作系统使用，xvdb和xvdc分区/dev/xvdb1、/dev/xvdc1做drbd使用。
| Disk |
partition | remarks |
| xvda | /dev/xvda1 |
boot |
| xvda | /dev/xvda2 |
os |
| xvdb |
/dev/xvdb1 | drbd |
| xvdc | /dev/xvdc1 | drbd（扩容） |
二、系统初始化
关闭selinux
vi /etc/sysconfig/selinux
找到行 SELINUX=enforcing
修改为 SELINUX=disabled
关闭iptables
chkconfig ipables off
service iptables stop
设置主机名解析
vi /etc/hosts
文件最后加入以下：
172.26.184.38 pg1-rg
172.26.184.39 pg2-rg配置安装源--这里配置的是本地yum repository
vi /etc/yum.repos.d/custom.repo
[centos]
name=centos6.4iso
baseurl=http://172.26.184.241/centos/6.4/os/x86_64
enabled=1
gpgcheck=0[epel]
name=epel
baseurl=http://172.26.184.241/epel/6/x86_64/
enabled=1
gpgcheck=0[postgresql91]
name=postgresql91
baseurl=http://172.26.184.241/postgresql91
enabled=1
gpgcheck=0
三、DRBD安装配置（2台主机)
安装依赖软件包
yum install kernel-devel kernel-headers flex wget make gcc glibc perl -y
下载最新drbd源码包，并解压
cd /tmp
wget http://oss.linbit.com/drbd/drbd-latest.tar.gz
tar -zxvf drbd-latest.tar.gz -C /usr/local/src/
编译安装drbd
cd /usr/local/src/drbd-8.4.4
./configure --prefix=/opt/drbd --localstatedir=/var --sysconfdir=/etc --with-km --with-rgmanagermake && make install
安装drbd模块
cd /usr/local/src/drbd-8.4.4/drbd
make clean
make
make install
cp drbd.ko /lib/modules/`uname -r`/kernel/lib/
modprobe drbd
lsmod | grep drbd #如果看到以下信息，说明内核加载drbd模块成功。drbd 340679 8
libcrc32c 1246 1 drbd配置drbd
cd /etc/drbd.d
全局配置
vi global_common.conf
global { usage-count yes; # minor-count dialog-refresh disable-ip-verification } common { syncer { rate 100M; } handlers { # These are EXAMPLE handlers only. # They may have severe implications,# like hard resetting the node under certain circumstances. # Be careful when chosing your poison. # pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f"; # pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f"; # local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f"; # fence-peer "/usr/lib/drbd/crm-fence-peer.sh"; # split-brain "/usr/lib/drbd/notify-split-brain.sh root"; # out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root"; # before-resync-target "/usr/lib/drbd/snapshot-resync-target-lvm.sh -p 15 -- -c 16k"; # after-resync-target /usr/lib/drbd/unsnapshot-resync-target-lvm.sh; } startup { # wfc-timeout degr-wfc-timeout outdated-wfc-timeout wait-after-sb degr-wfc-timeout 120; # 2 minutes. } options { # cpu-mask on-no-data-accessible } disk { # size max-bio-bvecs on-io-error fencing disk-barrier disk-flushes # disk-drain md-flushes resync-rate resync-after al-extents # c-plan-ahead c-delay-target c-fill-target c-max-rate # c-min-rate disk-timeout } net { # protocol timeout max-epoch-size max-buffers unplug-watermark # connect-int ping-int sndbuf-size rcvbuf-size ko-count # allow-two-primaries cram-hmac-alg shared-secret after-sb-0pri # after-sb-1pri after-sb-2pri always-asbp rr-conflict # ping-timeout data-integrity-alg tcp-cork on-congestion # congestion-fill congestion-extents csums-alg verify-alg # use-rle cram-hmac-alg "sha1"; shared-secret "FooFunFactory"; after-sb-0pri disconnect; after-sb-1pri disconnect; after-sb-2pri disconnect; rr-conflict disconnect; } }资源配置
vi drbd.res
resource r0 { protocol C; device /dev/drbd0; disk /dev/xvdb1; Meta-disk internal; on pg1-rg { address 172.26.184.38:7788; } on pg2-rg { address 172.26.184.39:7788; } } resource r1 { protocol C; device /dev/drbd1; disk /dev/xvdc1; Meta-disk internal; on pg1-rg { address 172.26.184.38:7789; } on pg2-rg { address 172.26.184.39:7789; } }创建节点启动drbd服务
drbdadm create-md r0
chkconfig --add drbd
service drbd startdrbd默认 2个都是secondary，所以在主节点设置为primary
drbdadm -- --overwrite-data-of-peer primary all （第一次设置主节点）
drbdadm primary all （以后设置节点为主节点）drbd常用命令
查看资源的连接状态
drbdadm cstate all查看节点状态
drbdadm role all
查看同步进度
cat /proc/drbd 或者 /opt/drbd/sbin/drbd-overview
#############报错信息#############
报错1:
configure: error: Cannot build utils without flex,either install flex or pass the --without-utils option.
解决办法:
yum -y intalls flex
报错2：
make: [check-kdir] error 1
解决办法：yum install -y kernel-devel
报错3：
SORRY,kernel makefile not found.
You need to tell me a correct KDIR,
Or install the neccessary kernel source packages.
报错4：'drbd' not defined in your config (for this host).
这个因为没有加载到drbd模块
启动报错：
Command 'drbdMeta 0 v08 /dev/sdb1 internal create-md' terminated with exit code 40
这是因为sdb1已经有文件系统了，已经有数据存在了
解决方法：
[root@node1 ~]# dd if=/dev/zero bs=1M count=1 of=/dev/sdb1
请检查这个KDIR目录指的路径是否正确
##########################
四、postgresql安装（以下在主机操作，备机仅需安装postgresql）
准备数据目录（格式化drbd设备）
pvcreate /dev/drbd0
vgcreate drbd /dev/drbd0
lvcreate -l +100%FREE -n pgsql drbd
mkfs.ext4 /dev/drbd/pgsql
mkdir -p /var/lib/pgsql/9.1/data
mount /dev/drbd/pgsql /var/lib/pgsql/9.1/data安装postgresql（本文主要介绍psql的高可用性，psql配置略过）
yum install postgresql91-server -y
chown -R postgres.postgres /var/lib/pgsql/
chmod -R 700 /var/lib/pgsql/
service postgresql-9.1 initdb
service postgresql-9.1 start
五、RHCS套件安装配置
yum groupinstall "High Availability" -y
编辑配置文件
vi /etc/cluster/cluster.conf
<?xml version="1.0"?> <cluster config_version="32" name="pgsql"> <clusternodes> <clusternode name="pg1-rg" nodeid="1"/> <clusternode name="pg2-rg" nodeid="2"/> </clusternodes> <cman expected_votes="1" two_node="1"/> <rm> <resources> <ip address="172.26.184.42/24" sleeptime="10"/> <drbd name="r0" resource="r0"/> <drbd name="r1" resource="r1"/> <script file="/etc/init.d/postgresql-9.1" name="postgresql91"/> <fs device="/dev/drbd/pgsql" fsid="27557" fstype="ext4" mountpoint="/var/lib/pgsql/9.1/data" name="pgsql(mount)"/> <script file="/etc/init.d/drbd_cluster.sh" name="switch drbd"/> </resources> <failoverdomains> <failoverdomain name="FO" nofailback="1" ordered="1"> <failoverdomainnode name="pg1-rg" priority="1"/> <failoverdomainnode name="pg2-rg" priority="2"/> </failoverdomain> </failoverdomains> <service domain="FO" exclusive="1" name="postgres" recovery="relocate"> <ip ref="172.26.184.42/24"/> <script ref="switch drbd"> <fs ref="pgsql(mount)"> <script ref="postgresql91"/> </fs> </script> </service> </rm> <fencedevices> <fencedevice agent="fence_ilo" ipaddr="xxx.xxx.xxx.xxx" ipport="xx" login="username" name="ilo_fence" passwd="password"/> </fencedevices> <logging to_syslog="no"/> </cluster>
cluster.conf配置中，在切换DRBD时需要自己编写故障切换脚本，详见附件。
vi /etc/init.d/drbd_cluster.sh #内容见附件
chmod +x /etc/init.d/drbd_cluster.sh
启动cluster
service cman start
service rgmanager start
检查点状态
clustat -l
模拟故障切换
在主机（pg1-rg）上确认状态。
df
检查postgresql服务状态
在pg1-rg上停止postgresql-9.1服务
service postgresql-9.1 stop
一段时间后，pg2-rg自动接管服务。
