CentOS 6.5上部署drbd

前端之家收集整理的这篇文章主要介绍了CentOS 6.5上部署drbd前端之家小编觉得挺不错的,现在分享给大家,也给大家做个参考。

下面开始在CentOS 6.5上安装并配置drbd,环境的话继续使用之前安装完heartbeat的两台主机,同时也是为后面实现heartbeat存储和数据库高可用做准备,所以如果需要单独操作,请按照之前安装heartbeat的准备工作进行配置,然后根据下面的步骤进行也可以。

1、环境准备

这里我以其中一台为例,介绍准备工作包括哪些点:

#检查防火墙是否关闭(或者开启7788端口)

[root@heartbeat01 ~]# iptables -L -n

Chain INPUT (policy ACCEPT)

target prot opt source destination


Chain FORWARD (policy ACCEPT)

target prot opt source destination


Chain OUTPUT (policy ACCEPT)

target prot opt source destination

#检查SELinux是否禁用

[root@heartbeat01 ~]# getenforce

Disabled

#检查是否已经添加了时间同步定时任务

[root@heartbeat01 ~]# crontab -l

0 * * * * /usr/sbin/ntpdate 210.72.145.44 64.147.116.229 time.nist.gov

#检查hosts文件中是否有两个节点的记录

[root@heartbeat01 ~]# cat /etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4

::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.49.133 heartbeat01.contoso.com heartbeat01

192.168.49.134 heartbeat02.contoso.com heartbeat02

#检查是否更换国内yum源

[root@heartbeat01 ~]# ll /etc/yum.repos.d/

total 32

-rw-r--r-- 1 root root 2006 Sep 18 2014 CentOS-Base.repo

-rw-r--r--. 1 root root 1926 Nov 27 2013 CentOS-Base.repo.bak_2016-07-28

-rw-r--r--. 1 root root 638 Nov 27 2013 CentOS-Debuginfo.repo.bak_2016-07-28

-rw-r--r--. 1 root root 630 Nov 27 2013 CentOS-Media.repo.bak_2016-07-28

-rw-r--r--. 1 root root 3664 Nov 27 2013 CentOS-Vault.repo.bak_2016-07-28

-rw-r--r--. 1 root root 120 Jul 25 18:24 cobbler-config.repo.bak_2016-07-28

另外,跟heartbeat不同的是,drbd需要两块硬盘,所以这里我们还需要添加一块硬盘,我是在虚拟机操作的,直接添加一块2G的硬盘。

[root@heartbeat01 ~]# fdisk -l


Disk /dev/sda: 21.5 GB,21474836480 bytes

255 heads,63 sectors/track,2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x000693e3


Device Boot Start End Blocks Id System

/dev/sda1 * 1 64 512000 83 Linux

Partition 1 does not end on cylinder boundary.

/dev/sda2 64 2611 20458496 8e Linux LVM


Disk /dev/sdb: 2147 MB,2147483648 bytes

255 heads,261 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x00000000

2、安装drbd

drbd的安装有多种方式,可以源码编译安装,也可以使用rpm包进行安装,当然在centos上也可以使用yum安装,这里我采用yum安装的方式。

rpm-Uvhhttp://www.elrepo.org/elrepo-release-6-6.el6.elrepo.noarch.rpm
yumupdate-y
yum-yinstallkernel*
yum-yinstalldrbd83-utilskmod-drbd83
modprobedrbd

因为更新了内核,所以需要重启才能加载drbd模块,所以使用yum安装完kernel之后需要重启两台服务器,如果不重启则会出现以下问题:

[root@heartbeat01 ~]# uname -r

2.6.32-431.el6.x86_64

[root@heartbeat01 ~]# lsmod|grep drbd

[root@heartbeat01 ~]# modprobe drbd

FATAL: Module drbd not found.

[root@heartbeat01 ~]# echo $?

1

重启之后重试:

[root@heartbeat01 ~]# uname -r

2.6.32-642.4.2.el6.x86_64

[root@heartbeat01 ~]# lsmod |grep drbd

drbd 332493 0

[root@heartbeat01 ~]# /sbin/modprobe drbd

[root@heartbeat01 ~]# echo $?

0

3、准备drbd设备

先对新添加的磁盘进行分区操作,这里仍然以其中一台为例。

[root@heartbeat01 ~]# fdisk -cu /dev/sdb

Device contains neither a valid DOS partition table,nor Sun,SGI or OSF disklabel

Building a new DOS disklabel with disk identifier 0x198bc436.

Changes will remain in memory only,until you decide to write them.

After that,of course,the prevIoUs content won't be recoverable.


Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)


Command (m for help): p


Disk /dev/sdb: 2147 MB,2147483648 bytes

255 heads,261 cylinders,total 4194304 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x198bc436


Device Boot Start End Blocks Id System


Command (m for help): n

Command action

e extended

p primary partition (1-4)

p

Partition number (1-4): 1

First sector (2048-4194303,default 2048):

Using default value 2048

Last sector,+sectors or +size{K,M,G} (2048-4194303,default 4194303):

Using default value 4194303


Command (m for help): w

The partition table has been altered!


Calling ioctl() to re-read partition table.

Syncing disks.

4、编辑drbd配置文件

根据drbd的官方文档,drbd的配置文件是/etc/drbd.conf,但是这个文件中只有简单的2行内容,它引用了/etc/drbd.d/下的global_common.conf和*.res文件,这点和Nginx、apache的配置文件类似,我们可以把多个resource相同的定义放到global中去,然后添加不同的res文件,从而更有效和方便的管理drbd的配置文件

下面是/etc/drbd.conf的默认配置:

[root@heartbeat01 ~]# egrep -v "#|^$" /etc/drbd.conf

include "drbd.d/global_common.conf";

include "drbd.d/*.res";

下面是/etc/drbd.d/global_common.conf的默认配置:

[root@heartbeat01 ~]# egrep -v "#|^$" /etc/drbd.d/global_common.conf

global {

usage-count yes;

}

common {

protocol C;

handlers {

}

startup {

}

disk {

}

net {

}

syncer {

}

}

下面是我新建的一个配置文件test.res的内容

resource test

{

startup {

wfc-timeout 30;

outdated-wfc-timeout 20;

degr-wfc-timeout 30;

}

net {

cram-hmac-alg sha1;

shared-secret sync_disk;

}

syncer {

rate 10M;

al-extents 257;

on-no-data-accessible io-error;

}

on heartbeat01.contoso.com {

device /dev/drbd0;

disk /dev/sdb1;

address 192.168.49.133:7788;

flexible-Meta-disk internal;

}

on heartbeat02.contoso.com {

device /dev/drbd0;

disk /dev/sdb1;

address 192.168.49.134:7788;

Meta-disk internal;

}

}

然后可以将此配置文件使用scp拷贝到节点2上去,两边的配置文件需要保持一致。

[root@heartbeat01 drbd.d]# scp test.res heartbeat02:/etc/drbd.d/

The authenticity of host 'heartbeat02 (192.168.49.134)' can't be established.

RSA key fingerprint is f9:ce:14:5d:cd:bb:3c:b4:0d:0b:fc:21:3a:92:43:6b.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'heartbeat02,192.168.49.134' (RSA) to the list of known hosts.

root@heartbeat02's password:

test.res 100% 444 0.4KB/s 00:00

5、启动drbd

drbdadmcreate-mdclusterdb#初始化Metadata数据存储
servicedrbdstart#启动drbd服务

上面的操作需要到两台server上运行,这里以其中一台为例演示结果:

[root@heartbeat01 drbd.d]# drbdadm create-md test

Writing Meta data...

initializing activity log

NOT initialized bitmap

New drbd Meta data block successfully created.

success

[root@heartbeat01 drbd.d]# service drbd start

Starting DRBD resources: [ d(test) s(test) n(test) ]..........

***************************************************************

DRBD's startup script waits for the peer node(s) to appear.

- In case this node was already a degraded cluster before the

reboot the timeout is 30 seconds. [degr-wfc-timeout]

- If the peer was available before the reboot the timeout will

expire after 30 seconds. [wfc-timeout]

(These values are for resource 'test'; 0 sec -> wait forever)

To abort waiting enter 'yes' [ 11]:

.

这里给出很多信息,大多是因为对端节点并没有启动,所以配置文件中的等待时间生效,但是在节点1启动后,我们再启动节点2时的信息就没有这么多了,因为节点1已经启动,所以无须等待。

[root@heartbeat02 drbd.d]# service drbd start

Starting DRBD resources: [ d(test) s(test) n(test) ].

此时,两个节点上都是Secondary的状态,所以我们需要手动的指定哪个节点为primary。

[root@heartbeat01 drbd.d]# cat /proc/drbd

version: 8.3.16 (api:88/proto:86-97)

GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R6,2014-11-24 14:51:37

0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r-----

ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:2096028

[root@heartbeat02 drbd.d]# cat /proc/drbd

version: 8.3.16 (api:88/proto:86-97)

GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R6,2014-11-24 14:51:37

0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r-----

ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:2096028

在要作为主节点的server上执行如下命令:

drbdadm----overwrite-data-of-peerprimaryall(是两个--,不是打错)

这里我在节点1上执行该命令。

[root@heartbeat01 drbd.d]# drbdadm -- --overwrite-data-of-peer primary all

再次看一下两个节点的状态:

[root@heartbeat01 drbd.d]# cat /proc/drbd

version: 8.3.16 (api:88/proto:86-97)

GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R6,2014-11-24 14:51:37

0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----

ns:217088 nr:0 dw:0 dr:217752 al:0 bm:13 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:1878940

[=>..................] sync'ed: 10.6% (1878940/2096028)K

finish: 0:03:01 speed: 10,344 (10,336) K/sec

[root@heartbeat02 drbd.d]# cat /proc/drbd

version: 8.3.16 (api:88/proto:86-97)

GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R6,2014-11-24 14:51:37

0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r-----

ns:0 nr:485248 dw:485248 dr:0 al:0 bm:29 lo:1 pe:1 ua:0 ap:0 ep:1 wo:f oos:1610780

[===>................] sync'ed: 23.3% (1610780/2096028)K

finish: 0:02:34 speed: 10,424 (10,324) want: 10,240 K/sec

可以看到现在heartbeat01是Primary的状态,而heartbeat02是Secondary的状态,而且已经开始从主上面同步数据到从节点上了。

[root@heartbeat01 drbd.d]# cat /proc/drbd

version: 8.3.16 (api:88/proto:86-97)

GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R6,2014-11-24 14:51:37

0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----

ns:2096028 nr:0 dw:0 dr:2096692 al:0 bm:128 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

同步完成,两个节点都是UpToDate的状态。

6、创建文件系统并挂载

下面在主节点上为drbd设置创建文件系统

[root@heartbeat01 drbd.d]# mkfs.ext4 /dev/drbd0

mke2fs 1.41.12 (17-May-2010)

Filesystem label=

OS type: Linux

Block size=4096 (log=2)

Fragment size=4096 (log=2)

Stride=0 blocks,Stripe width=0 blocks

131072 inodes,524007 blocks

26200 blocks (5.00%) reserved for the super user

First data block=0

Maximum filesystem blocks=536870912

16 block groups

32768 blocks per group,32768 fragments per group

8192 inodes per group

Superblock backups stored on blocks:

32768,98304,163840,229376,294912


Writing inode tables: done

Creating journal (8192 blocks): done

Writing superblocks and filesystem accounting information: done


This filesystem will be automatically checked every 22 mounts or

180 days,whichever comes first. Use tune2fs -c or -i to override.

然后创建挂载点,将drbd设备挂载到挂载点从而进行访问。

注意:从节点不需要为drbd创建文件系统,也无须挂载,在主节点挂掉之后,可以直接转换角色,然后将文件系统挂载到挂载点进行使用。

[root@heartbeat01 drbd.d]# mkdir /data

[root@heartbeat01 drbd.d]# mount /dev/drbd0 /data

[root@heartbeat01 drbd.d]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/VolGroup-lv_root

19G 1.6G 16G 10% /

tmpfs 238M 0 238M 0% /dev/shm

/dev/sda1 477M 83M 369M 19% /boot

/dev/drbd0 2.0G 3.0M 1.9G 1% /data

7、切换Primary和Secondary节点测试

wKiom1fmparjLybeAAGlgNf_BJw335.png-wh_50

这里,我创建了99个txt文件在/data目录下,等下看切换了主从节点之后,能否在从节点上看到这些文件

下面在主节点(heartbeat01)上操作:

首先要取消挂载:

[root@heartbeat01 drbd.d]# umount /data

[root@heartbeat01 drbd.d]# mount -n

/dev/mapper/VolGroup-lv_root on / type ext4 (rw)

proc on /proc type proc (rw)

sysfs on /sys type sysfs (rw)

devpts on /dev/pts type devpts (rw,gid=5,mode=620)

tmpfs on /dev/shm type tmpfs (rw)

/dev/sda1 on /boot type ext4 (rw)

none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)

后执行下面的命令切换角色:

[root@heartbeat01 drbd.d]# drbdadm secondary test

此时看到主节点已经是Secondary的角色了,而从节点暂时没有处理,也还是Secondary的角色。

[root@heartbeat01 drbd.d]# cat /proc/drbd

version: 8.3.16 (api:88/proto:86-97)

GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R6,2014-11-24 14:51:37

0: cs:Connected ro:Secondary/Secondary ds:UpToDate/UpToDate C r-----

ns:2162432 nr:0 dw:66404 dr:2097405 al:23 bm:128 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

下面到从节点(heartbeat02)上操作:

[root@heartbeat02 drbd.d]# drbdadm primarytest

或者

[root@heartbeat02 drbd.d]# drbdadm -- --overwrite-data-of-peer primary all

[root@heartbeat02 drbd.d]# cat /proc/drbd

version: 8.3.16 (api:88/proto:86-97)

GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R6,2014-11-24 14:51:37

0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----

ns:0 nr:2162432 dw:2162432 dr:664 al:0 bm:128 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

此时,从节点(heartbeat02)已经是primary状态了。

[root@heartbeat02 drbd.d]# mkdir /data

[root@heartbeat02 drbd.d]# mount /dev/drbd0 /data

wKiom1fmp-njYqn5AAF406snnDE057.png-wh_50

创建目录并挂载,已经在heartbeat02的/data目录看到之前创建的99个txt文件了,说明drbd已经将数据同步成功了。

猜你在找的CentOS相关文章