VCS引起的oracle数据库异常重启一例

前端之家收集整理的这篇文章主要介绍了VCS引起的oracle数据库异常重启一例前端之家小编觉得挺不错的,现在分享给大家,也给大家做个参考。

1. 环境描述

操作系统版本:SUSELinuxEnterprise Server 10 sp2 (x86_64)

数据库版本:Oracle11.1.0.7.16

VCS版本:5.1

blog地址:http://blog.csdn.net/hw_libo/article/details/41171561


2. 问题现象及分析

凌晨3:46左右,手机短信收到数据库异常告警。

(1)查看数据库alert日志

  1. SunNov1603:46:512014
  2. StoppingbackgroundprocessSMCO
  3. StoppingbackgroundprocessFBDA
  4. Shuttingdowninstance:furtherlogonsdisabled
  5. SunNov1603:46:542014
  6. kkjcre1p:unabletospawnjobqslaveprocess,slot0,error1089
  7. StoppingbackgroundprocessCJQ0
  8. StoppingbackgroundprocessQMNC
  9. StoppingbackgroundprocessMMNL
  10. StoppingbackgroundprocessMMON
  11. SunNov1603:47:032014
  12. Shuttingdowninstance(immediate)
  13. Licensehighwatermark=366
  14. SunNov1603:51:522014
  15. USER(ospid:8654):terminatingtheinstance
  16. SunNov1603:51:522014
  17. ORA-1092:opiodrabortingprocessunknownospid(28656_47367034883680)
  18. LGWRwaitingforinstancetermination

并未发现较为明显的异常原因。

同时也查看了那个时间段的报告,发现数据库压力不大,也没有什么异常的sql存在。


(2)检查OS资源使用


系统资源使用正常,IO也不高。


(3)查看messages日志

copy
Nov1603:42:46boscodbAgentFramework[10894]:VCSERRORV-16-2-13027Thread(4149214112)Resource(ora_vg)-monitorproceduredidnotcompletewithintheexpectedtime.
  • Nov1603:42:46boscodbHad[10258]:VCSERRORV-16-2-13027(boscodb)Resource(ora_vg)-monitorproceduredidnotcompletewithintheexpectedtime.
  • Nov1603:43:01boscodbsu:(toroot)rootonnone
  • ……
  • Nov1603:45:39boscodbsu:(tooracle)rootonnone
  • Nov1603:45:47boscodbAgentFramework[10894]:VCSERRORV-16-2-13210Thread(4154813344)Agentiscallingcleanforresource(ora_vg)because4successiveinvocationsofthemonitorproceduredidnotcompletewithintheexpectedtime.
  • Nov1603:45:47boscodbHad[10258]:VCSERRORV-16-2-13210(boscodb)Agentiscallingcleanforresource(ora_vg)because4successiveinvocationsofthemonitorproceduredidnotcompletewithintheexpectedtime.
  • Nov1603:46:48boscodbAgentFramework[10894]:VCSERRORV-16-2-13006Thread(4153760672)Resource(ora_vg):cleanproceduredidnotcompletewithintheexpectedtime.
  • Nov1603:46:48boscodbHad[10258]:VCSERRORV-16-2-13006(boscodb)Resource(ora_vg):cleanproceduredidnotcompletewithintheexpectedtime.
  • Nov1603:46:49boscodbAgentFramework[10899]:VCSERRORV-16-2-13067Thread(4155194272)Agentiscallingcleanforresource(ora_archive)becausetheresourcebecameOFFLINEunexpectedly,onitsown.
  • Nov1603:46:50boscodbHad[10258]:VCSERRORV-16-2-13067(boscodb)Agentiscallingcleanforresource(ora_archive)becausetheresourcebecameOFFLINEunexpectedly,onitsown.
  • Nov1603:46:51boscodbAgentFramework[10899]:VCSERRORV-16-2-13068Thread(4155194272)Resource(ora_archive)-cleancompletedsuccessfully.
  • 原来是VCS出现了错误,监控vg超时了,这些错误信息在VCS的日志中也有记录。


    3. 解决办法

    检查了监控vg的moniter脚本,发现也没有异常,而且脚本在其他库,压力都比较大的都不曾出问题。

    当前的配置为:


    决定暂时将监控时长加大到90s,可以使用两种方法:

    (1)可以修改main.cf,但需要重启VCS,无操作时间窗口

    (2)命令动态修改

    haconf -makerw

    hares -modify ora_vg MonitorTimeout 90 ## 调整监控时长为90s,默认是60s

    haconf -dump


    blog地址:http://blog.csdn.net/hw_libo/article/details/41171561

    -- Bosco QQ:375612082

    ---- END ----

    猜你在找的Oracle相关文章