一、基础结构:
二、存储规划:
1、 GRID集群组件磁盘组
+dggrid: 1个,由三个10G磁盘组成normal
2、 数据库安装磁盘组
+dgsystem:用于数据库基本表空间,控制文件,参数文件等
+dgrecovery:用于归档与闪回日志空间
三、IP规划:
oraclenode1:
publicip : bond0 :192.168.10.103
vip:192.168.10.105
privateip :bond2 :192.168.14.3
oraclenode2:
publicip : bond0 :192.168.10.104
vip:192.168.10.106
privateip :bond2 :192.168.14.4
scanip :192.168.10.107
四、软件版本:
操作系统:CentOS 7.2
数据库:ORACLE12c R1
集群管理软件:ORACLEGRID 12.0.1.2
五、系统环境配置:
注意:如下配置除非特别说明,否则两个节点都需要操作
1、安装软件依赖
yuminstallbinutilscompat-libcap1compat-libstdc++-33compat-libstdc++-33.i686gccgcc-c++glibcglibc.i686glibc-develglibc-devel.i686kshlibgcclibgcc.i686libstdc++libstdc++.i686libstdc++-devellibstdc++-devel.i686libaiolibaio.i686libaio-devellibaio-devel.i686libXextlibXext.i686libXtstlibXtst.i686libX11libX11.i686libXaulibXau.i686libxcblibxcb.i686libXilibXi.i686makesysstatunixODBCunixODBC-develreadlinelibtermcap-develpdksh-y
#在两台主机修改host文件,添加如下内容: vim/etc/hosts #publicipbond0 192.168.10.103oraclenode1 192.168.10.104oraclenode2 #privateipbond2 192.168.14.3oraclenode1pri 192.168.14.4oraclenode2pri #vipip 192.168.10.105oraclenode1vip 192.168.10.106oraclenode2vip #scanip 192.168.10.107oraclenodescan
3、关闭selinux和配置防火墙
setenforce0 firewall-cmd--set-defaults-zone=trusted
#在两个节点增加用户与组: groupadd-g60001oinstall groupadd-g60002dba groupadd-g60003oper groupadd-g60004backupdba groupadd-g60005dgdba groupadd-g60006kmdba groupadd-g60007asmdba groupadd-g60008asmoper groupadd-g60009asmadmin useradd-u61001-goinstall-Gasmadmin,asmdba,dba,asmopergrid useradd-u61002-goinstall-Gdba,backupdba,dgdba,kmdba,asmadmin,oper,asmdbaoracle echo"grid"|passwd--stdingrid echo"oracle"|passwd--stdinoracle
5、添加目录
mkdir-p/data/oracle/app/grid mkdir-p/data/oracle/app/12.1.0.2/grid chown-Rgrid:oinstall/data/oracle mkdir-p/data/oracle/app/oraInventory chown-Rgrid:oinstall/data/oracle/app/oraInventory mkdir-p/data/oracle/app/oracle chown-Roracle:oinstall/data/oracle/app/oracle chmod-R775/data/oracle
6、修改操作系统参数
vim/etc/security/limits.conf #ORACLESETTING gridsoftnproc2047 gridhardnproc16384 gridsoftnofile1024 gridhardnofile65536 gridsoftstack10240 gridhardstack32768 oraclesoftnproc2047 oraclehardnproc16384 oraclesoftnofile1024 oraclehardnofile65536 oraclesoftstack10240 oraclehardstack32768 vim/etc/security/limits.d/20-nproc.conf #Changethis *softnproc1024 #Tothis *-nproc16384 vim/etc/pam.d/login #ORACLESETTING sessionrequiredpam_limits.so MEMTOTAL=$(free-b|sed-n'2p'|awk'{print$2}') SHMMAX=$(expr$MEMTOTAL\*4/5) SHMMNI=4096 SHMALL=$(expr$MEMTOTAL/\(4\*1024\)) cp/etc/sysctl.conf/etc/sysctl.conf.bak cat>>/etc/sysctl.conf<<EOF fs.aio-max-nr=1048576 fs.file-max=6815744 kernel.shmmax=$SHMMAX kernel.shmall=$SHMALL kernel.shmmni=$SHMMNI kernel.sem=25032000100128 net.ipv4.ip_local_port_range=900065500 net.core.rmem_default=262144 net.core.rmem_max=4194304 net.core.wmem_default=262144 net.core.wmem_max=1048576 kernel.panic_on_oops=1 EOF #kernel.shmmax大于共享内存区,小于物理内存 #kernel.shmall物理内存/4K 让配置生效: sysctl�Cp #使用centos7.2安装grid时,需要修改这个参数,不然会报错 vim/etc/systemd/logind.conf RemoveIPC=no systemctldaemon-reload systemctlrestartsystemcd-logind
7、配置用户环境变量
su-grid #grid用户的环境变量如下: vim~/.bash_profile PS1="[`whoami`@`hostname`:"'$PWD]$' exportPS1 umask022 exportTMP=/tmp exportLANG=en_US exportTMPDIR=$TMP exportORACLE_HOSTNAME=oraclenode1 ORACLE_SID=+ASM1;exportORACLE_SID ORACLE_TERM=xterm;exportORACLE_TERM ORACLE_BASE=/data/oracle/app/grid;exportORACLE_BASE ORACLE_HOME=/data/oracle/app/12.1.0.2/grid;exportORACLE_HOME NLS_DATE_FORMAT="yyyy-mm-ddHH24:MI:SS";exportNLS_DATE_FORMAT PATH=.:$PATH:$HOME/bin:$ORACLE_HOME/bin;exportPATH THREADS_FLAG=native;exportTHREADS_FLAG if[$USER="oracle"]||[$USER="grid"];then if[$SHELL="/bin/ksh"];then ulimit-p16384 ulimit-n65536 else ulimit-u16384-n65536 fi umask022 fi su-oracle #oracle用户的环境变量如下: vim~/.bash_profile PS1="[`whoami`@`hostname`:"'$PWD]$' exportPS1 exportTMP=/tmp exportLANG=en_US exportTMPDIR=$TMP exportORACLE_HOSTNAME=oraclenode1 exportORACLE_UNQNAME=orcldb ORACLE_BASE=/data/oracle/app/oracle;exportORACLE_BASE ORACLE_HOME=$ORACLE_BASE/product/12.1.0.2/db_1;exportORACLE_HOME ORACLE_SID=orcldb1;exportORACLE_SID ORACLE_TERM=xterm;exportORACLE_TERM NLS_DATE_FORMAT="yyyy-mm-ddHH24:MI:SS";exportNLS_DATE_FORMAT NLS_LANG=AMERICAN_AMERICA.ZHS16GBK;exportNLS_LANG PATH=.:$PATH:$HOME/bin:$ORACLE_BASE/product/12.1.0.2/db_1/bin:$ORACLE_HOME/bin;exportPATH THREADS_FLAG=native;exportTHREADS_FLAG if[$USER="oracle"]||[$USER="grid"];then if[$SHELL="/bin/ksh"];then ulimit-p16384 ulimit-n65536 else ulimit-u16384-n65536 fi umask022 fi #节点二: su-grid #grid的环境变量 vim~./bash_profile PS1="[`whoami`@`hostname`:"'$PWD]$' exportPS1 umask022 exportTMP=/tmp exportLANG=en_US exportTMPDIR=$TMP exportORACLE_HOSTNAME=oraclenode2 ORACLE_SID=+ASM2;exportORACLE_SID ORACLE_TERM=xterm;exportORACLE_TERM ORACLE_BASE=/data/oracle/app/grid;exportORACLE_BASE ORACLE_HOME=/data/oracle/app/12.1.0.2/grid;exportORACLE_HOME NLS_DATE_FORMAT="yyyy-mm-ddHH24:MI:SS";exportNLS_DATE_FORMAT PATH=.:$PATH:$HOME/bin:$ORACLE_HOME/bin;exportPATH THREADS_FLAG=native;exportTHREADS_FLAG if[$USER="oracle"]||[$USER="grid"];then if[$SHELL="/bin/ksh"];then ulimit-p16384 ulimit-n65536 else ulimit-u16384-n65536 fi umask022 fi su-oracle vim~/.bash_profile PS1="[`whoami`@`hostname`:"'$PWD]$' exportPS1 exportTMP=/tmp exportLANG=en_US exportTMPDIR=$TMP exportORACLE_HOSTNAME=oraclenode2 exportORACLE_UNQNAME=orcldb ORACLE_BASE=/data/oracle/app/oracle;exportORACLE_BASE ORACLE_HOME=$ORACLE_BASE/product/12.1.0.2/db_1;exportORACLE_HOME ORACLE_SID=orcldb2;exportORACLE_SID ORACLE_TERM=xterm;exportORACLE_TERM NLS_DATE_FORMAT="yyyy-mm-ddHH24:MI:SS";exportNLS_DATE_FORMAT NLS_LANG=AMERICAN_AMERICA.ZHS16GBK;exportNLS_LANG PATH=.:$PATH:$HOME/bin:$ORACLE_BASE/product/12.1.0.2/db_1/bin:$ORACLE_HOME/bin;exportPATH THREADS_FLAG=native;exportTHREADS_FLAG if[$USER="oracle"]||[$USER="grid"];then if[$SHELL="/bin/ksh"];then ulimit-p16384 ulimit-n65536 else ulimit-u16384-n65536 fi umask022 fi
8、配置ssh无密登录(两节点)
su-grid ssh-keygen ssh-copy-id-i~/.ssh/id_rsa.pubgrid@192.168.10.103 ssh-copy-id-i~/.ssh/id_rsa.pubgrid@192.168.10.104 su-oracle ssh-keygen ssh-copy-id-i~/.ssh/id_rsa.puboracle@192.168.10.103 ssh-copy-id-i~/.ssh/id_rsa.puboracle@192.168.10.104
9、配置共享存储
存储划分以下卷: grid0110G grid0210G grid0310G system300G recovery800G data4T 配置多路径wwid和别名: 前提:服务器已经连接到存储,并识别到LUN a、配置/etc/multipath.conf文件 defaults{ polling_interval10 path_selector"round-robin0" path_grouping_policymultibus prioalua path_checkerreadsector0 rr_min_io100 max_fds8192 rr_weightpriorities failbackimmediate no_path_retryfail user_friendly_namesyes find_multipathsyes } blacklist{ devnode"^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode"^sd[a]" } b、执行如下命令 modprobedm-multipath multipath-F multipathdm-multipath multipathdm-round-robin 执行完成上述命令之后,会在/etc/multipath目录下生成一个bindings这的一个文件,包含了存储的uuid和别名 mpatha36000d310036222000000000000000003 mpathb36000d310036222000000000000000004 mpathc36000d310036222000000000000000008 mpathd36000d310036222000000000000000007 mpathe36000d310036222000000000000000006 mpathf36000d310036222000000000000000005 mpathg36000d310036222000000000000000009 c、修改别名方便识别 multipaths{ multipath{ wwid36000d310036222000000000000000004 aliasdgdata path_grouping_policymultibus } multipath{ wwid36000d310036222000000000000000005 aliasdggrid1 path_grouping_policymultibus } multipath{ wwid36000d310036222000000000000000006 aliasdggrid2 path_grouping_policymultibus } multipath{ wwid36000d310036222000000000000000009 aliasdgrecovery path_grouping_policymultibus } multipath{ wwid36000d310036222000000000000000007 aliasdggrid3 path_grouping_policymultibus } multipath{ wwid36000d310036222000000000000000008 aliasdgsystem path_grouping_policymultibus } } 重启服务生效 systemctlreloadmultipath d、修改udev权限 vim/etc/udev/rules/12-mulitpath-privs.rules ENV{DM_NAME}=="dg*",OWNER:="grid",GROUP:="asmadmin",MODE:="660" 重启服务器生效,并检查重启后权限、别名是否正常
10、安装cvuqdisk包并验证
解压安装包 unziplinuxamd64_12102_grid_1of2.zip unziplinuxamd64_12102_grid_2of2.zip cd/soft/grid/rpm exportCVUQDISK_GRP=oinstall rpm-ivhcvuqdisk-1.0.9-1.rpm su-grid ./runcluvfy.shstage-precrsinst-noraclenode1,oraclenode2-fixup-verbose ./runcluvfy.shstage-posthwos-noraclenode1,oraclenode2-verbose #检查不通过的配置
11、安装grid
#远程调用图形界面安装vnc ./runInstaller
图形化界面安装,按照前面规划填写相关配置,值得注意的是硬盘位置需要修改:使用多路径修改为/dev/mapper/dg*
grid安装完成,使用crs_stat-t查看集群状态,使用ocrcheck、crsctl query css votedisk查看ocr磁盘状态
12、创建asm磁盘组
su-grid asmca #按照自己的规划,添加相应的磁盘组,并和存储卷关联
数据库软件的安装和实例创建,可以参考http://ld0381.blog.51cto.com/3318114/1923076上的内容,集群的配置和单实例步骤基本上是一致的。