最近两天生产数据库一直在报ORA-24756错误，查了MOS上的文档，看到一篇类似的，说是BUG(Bug 19201866 - RECO reports ORA-24756 repeatedly into trace file (文档 ID 19201866.8))，但是HP-UX上的11.2.0.4没有解决需要升级到12.2版本，文档中还说重启实例也可以解决，但是不现实。

于是查报错时的trace文件都是如下的内容

***2017-02-0409:17:56.058
ERROR,tran=42.13.2709343,ose=0:
ORA-24756:
......

看到tran=42.13.2709343，印象中是分布式事务，于是查DBA_2PC_PENDINGS视图

SYS@db1>COLLOCAL_TRAN_IDFORMATA13
SYS@db1>COLGLOBAL_TRAN_IDFORMATA90
SYS@db1>COLSTATEFORMATA10
SYS@db1>COLMIXEDFORMATA3
SYS@db1>COLHOSTFORMATA10
SYS@db1>COLCOMMIT#FORMATA15
SYS@db1>SETLINESIZE240
SYS@db1>SELECTLOCAL_TRAN_ID,GLOBAL_TRAN_ID,STATE,FAIL_TIME,FORCE_TIME,RETRY_TIME,MIXED,HOST,COMMIT#
2FROMDBA_2PC_PENDING
3/

LOCAL_TRAN_IDGLOBAL_TRAN_IDSTATEFAIL_TIMEFORCE_TIMERETRY_TIMEMIXHOSTCOMMIT#
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
42.13.27093431096044365.31302E3235352E3233322E32332E746D313438363137313036383638333230333633collecting2017020409:17:552017020601:12:41nobosbpm4s764631398601

看到LOCAL_TRAN_ID和trace文件中的tran一致，刚开始的设想是正确的。其中FAIL_TIME是20170204 09:17:55对应到后台的alert日志中，看到这个时间前后的报错信息

SatFeb0409:17:502017
Error22trappedin2PContransaction42.13.2709343.Cleaningup.
Errorstackreturnedtouser:
ORA-02050:transaction42.13.2709343rolledback,someremoteDBsmaybein-doubt
ORA-00022:invalidsessionID;accessdenied
ORA-02063:precedinglinefromLINK_DB2
SatFeb0409:17:562017
DISTRIBTRAN41544f4d.31302E3235352E3233322E32332E746D313438363137313036383638333230333633
islocaltran42.13.2709343(hex=2a.0d.29575f)
insertpendingcollectingtran,scn=764631398601(hex=b2.079538c9)
SatFeb0409:17:562017
Errorsinfile/oracle11g/app/oracle/diag/rdbms/db1/db1/trace/db1_reco_23402.trc:
ORA-24756:

这种分布式事务故障可能会锁定数据导致其他事务报ORA-01591报错或者一直占用UNDO段不能被别的事务重用。需要手工处理这种故障。

SYS@db1>EXECUTEDBMS_TRANSACTION.PURGE_LOST_DB_ENTRY('42.13.2709343');

PL/sqlproceduresuccessfullycompleted.

SYS@db1>commit;

Commitcomplete.

SYS@db1>SELECTLOCAL_TRAN_ID,COMMIT#
2FROMDBA_2PC_PENDING
3/

norowsselected

清理完毕。

管理分布式事务的官方文档：http://docs.oracle.com/cd/E11882_01/server.112/e25494/ds_txnman.htm#ADMIN12252

MOS文档：How to Purge a Distributed Transaction from a Database (文档 ID 159377.1)ORA-30019 When Executing Dbms_transaction.Purge_lost_db_entry (文档 ID 290405.1)