1、Oracle备库down机
微软云上的oracle备库意外ping不通了，各种报警上来了，联系微软云后台工程师，他们看了下，大概是说文件损坏了，信息如下：@H_301_15@
Checking all file systems.@H_301_15@ [/sbin/fsck.ext4 (1) -- /] fsck.ext4 -a /dev/sda1@H_301_15@ /dev/sda1: clean,134941/1905008 files,2416626/7608064 blocks@H_301_15@ [/sbin/fsck.ext4 (1) -- /oracle] fsck.ext4 -a /dev/sdc1@H_301_15@ fsck.ext4: No such file or directory while trying to open /dev/sdc1@H_301_15@ /dev/sdc1:@H_301_15@ The superblock could not be read or does not describe a correct ext2@H_301_15@ filesystem. If the device is valid and it really contains an ext2@H_301_15@ filesystem (and not swap or ufs or something else),then the superblock@H_301_15@ is corrupt,and you might try running e2fsck with an alternate superblock:@H_301_15@ e2fsck -b 8193 <device>@H_301_15@ *** An error occurred during the file system check.@H_301_15@ *** Dropping you to a shell; the system will reboot@H_301_15@ *** when you leave the shell.@H_301_15@ Give root password for maintenance@H_301_15@ (or type Control-D to continue):@H_301_15@ 系统在启动的过程中无法识别/dev/sdc1的superblock，未能正常启动到login界面，sdc1的superblock损坏。@H_301_15@ |
2、尝试界面恢复
给azure云的工程师建议，直接在管理界面关闭，然后再启动，试试看。不知道他们是否采纳了，过了3个小时候后，他们跟我说一句恢复了，oracle备库服务器已经启动起来了。@H_301_15@
然后我远程登录进去，一看数据文件目录还在，就准备startup，不过startup报错，如下所示@H_301_15@
ORACLE instance started.@H_301_15@ Total System Global Area 1.1825E+10 bytes@H_301_15@ Fixed Size 2217024 bytes@H_301_15@ Variable Size 6106909632 bytes@H_301_15@ Database Buffers 5704253440 bytes@H_301_15@ Redo Buffers 11849728 bytes@H_301_15@ Database mounted.@H_301_15@ ORA-10458: standby database requires recovery@H_301_15@ ORA-01196: file 1 is inconsistent due to a Failed media recovery session@H_301_15@ ORA-01110: data file 1: '/home/oradata/powerdes/system01.dbf'@H_301_15@ |
看到是书籍不一致，所以在启动的时候，尝试介质恢复失败，在system01.dbf这个数据文件上面恢复失败了。@H_301_15@
3、启动归档传输
启动归档，必须在mount状态下执行：@H_301_15@
sql> alter database recover managed standby database using current logfile disconnect from session;@H_301_15@ Database altered.@H_301_15@ |
4、实时观察日志应用
实时观察，查看主库备库归档传输完成：@H_301_15@
| # 主库查看：@H_301_15@ sql> select sequence#,applied from v$archived_log order by sequence#;@H_301_15@ ......@H_301_15@ SEQUENCE# APPLIED@H_301_15@ ---------- ---------@H_301_15@ 16850 NO@H_301_15@ 16850 NO@H_301_15@ 16851 NO@H_301_15@ 16851 NO@H_301_15@ 16852 NO@H_301_15@ 16852 NO@H_301_15@ 16853 NO@H_301_15@ 16853 NO@H_301_15@ 16854 NO@H_301_15@ 16854 NO@H_301_15@ 16855 NO@H_301_15@ SEQUENCE# APPLIED@H_301_15@ ---------- ---------@H_301_15@ 16855 NO@H_301_15@ 5600 rows selected.@H_301_15@ # 备库查看：@H_301_15@ sql> select sequence#,applied from v$archived_log order by sequence#;@H_301_15@ ......@H_301_15@ ---------- ---------@H_301_15@ 16844 NO@H_301_15@ 16845 NO@H_301_15@ 16846 NO@H_301_15@ 16847 NO@H_301_15@ 16848 NO@H_301_15@ 16849 NO@H_301_15@ 16850 NO@H_301_15@ 16851 NO@H_301_15@ 16852 NO@H_301_15@ 16853 NO@H_301_15@ 16854 NO@H_301_15@ SEQUENCE# APPLIED@H_301_15@ ---------- ---------@H_301_15@ 16855 NO@H_301_15@ 5600 rows selected.@H_301_15@ |
但是归档传输完成了，只是表面从主库的归档日志已经完全拉到备库了，但是日志应用还没有结束，还在执行中，去alert后台日志就可以看到信息：@H_301_15@
Mon Nov 21 19:35:31 2016@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16715_906253421.dbf@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16716_906253421.dbf@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16717_906253421.dbf@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16718_906253421.dbf@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16719_906253421.dbf@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16720_906253421.dbf@H_301_15@ Mon Nov 21 19:35:41 2016@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16721_906253421.dbf@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16722_906253421.dbf@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16723_906253421.dbf@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16724_906253421.dbf@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16725_906253421.dbf@H_301_15@ Mon Nov 21 19:35:58 2016@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16726_906253421.dbf@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16727_906253421.dbf@H_301_15@ Mon Nov 21 19:36:09 2016@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16728_906253421.dbf@H_301_15@ |
也可以查看dg状态，看到正在应用日志中@H_301_15@
sql> col value for a80@H_301_15@ sql> set linesize 2000@H_301_15@ sql> col name for a50@H_301_15@ sql> col value for a80@H_301_15@ sql> select name,value from v$dataguard_stats;@H_301_15@ NAME VALUE@H_301_15@ -------------------------------------------------- --------------------------------------------------------------------------------@H_301_15@ transport lag +00 00:00:00@H_301_15@ apply lag +02 01:23:55@H_301_15@ apply finish time +00 00:06:31.000@H_301_15@ estimated startup time 11@H_301_15@ |
等日志应用完成后，再查看后台alert log可以看到：@H_301_15@
Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16853_906253421.dbf@H_301_15@ Mon Nov 21 19:50:21 2016@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16854_906253421.dbf@H_301_15@ Mon Nov 21 19:50:31 2016@H_301_15@ Media Recovery Log /oracle/app/oracle/flash_recovery_area/archivelog1_16855_906253421.dbf@H_301_15@ Media Recovery Waiting for thread 1 sequence 16856 (in transit)@H_301_15@ Recovery of Online Redo Log: Thread 1 Group 4 Seq 16856 Reading mem 0@H_301_15@ Mem# 0: /home/oradata/powerdes/redo_dg_021.log@H_301_15@ @H_301_15@ |
5、打开数据库
sql> alter database open;@H_301_15@ alter database open@H_301_15@ *@H_301_15@ ERROR at line 1:@H_301_15@ ORA-10456: cannot open standby database; media recovery session may be in@H_301_15@ progress@H_301_15@ sql> select name,open_mode from v$database;@H_301_15@ NAME OPEN_MODE@H_301_15@ --------- --------------------@H_301_15@ POWERDES MOUNTED@H_301_15@ sql> alter database recover managed standby database cancel;@H_301_15@ Database altered.@H_301_15@ sql> alter database open;@H_301_15@ Database altered.@H_301_15@ sql> alter database recover managed standby database using current logfile disconnect;@H_301_15@ Database altered.@H_301_15@ |
