晚上给几个测试库打补丁，停完监听和实例后查看已经没有Oracle的进程了，于是执行opatch apply。但执行报如下错误：
$opatchapply OracleInterimPatchInstallerversion11.2.0.3.12 Copyright(c)2017,OracleCorporation.Allrightsreserved. OracleHome:/opt/oracle/app/oracle/product/11.2.4/db_1 CentralInventory:/opt/oracle/app/oracle/oraInventory from:/opt/oracle/app/oracle/product/11.2.4/db_1/oraInst.loc OPatchversion:11.2.0.3.12 OUIversion:11.2.0.4.0 Logfilelocation:/opt/oracle/app/oracle/product/11.2.4/db_1/cfgtoollogs/opatch/opatch2017-03-29_00-16-10AM_1.log Verifyingenvironmentandperformingprerequisitechecks... Prerequisitecheck"CheckActiveFilesAndExecutables"Failed. Thedetailsare: Followingexecutablesareactive: /opt/oracle/app/oracle/product/11.2.4/db_1/lib/libclntsh.so.11.1 UtilSessionFailed:Prerequisitecheck"CheckActiveFilesAndExecutables"Failed. Logfilelocation:/opt/oracle/app/oracle/product/11.2.4/db_1/cfgtoollogs/opatch/opatch2017-03-29_00-16-10AM_1.log
Oracle的相关进程已经没有了
[bbtst3soracle@tst3/home/oracle] $ps-ef|grepora oracle86088606002:04:59?0:00sshd:oracle@pts/2 oracle879619274002:05:24?0:00sleep15 oracle1927413Dec20?967:38/bin/sh./OSWatcher.sh15192 oracle88258610002:05:35pts/20:00grepora oracle866819319002:05:03?0:00sleep60 oracle88248610202:05:35pts/20:00ps-ef oracle86108608002:04:59pts/20:00-ksh oracle19319192740Dec20?23:31/bin/sh./OSWatcherFM.sh192/opt/bboss/tst3/oracle/arch/oswatcher/oswbb/archive root860621662002:04:56?0:00sshd:oracle[priv]
于是使用fuser查看
$fuser/opt/oracle/app/oracle/product/11.2.4/db_1/lib/libclntsh.so.11.1 /opt/oracle/app/oracle/product/11.2.4/db_1/lib/libclntsh.so.11.1:29888m
看到文件确实被占用了，但不知道是被哪个进程占用的，再使用lsof命令
$./lsof/opt/oracle/app/oracle/product/11.2.4/db_1/lib/libclntsh.so.11.1
执行完后没有结果，这就没办法查了。
于是请教了同事，他用使用root用户去执行lsof命令，而且只指定到最上一层的目录去查
#lsof/opt COMMANDPIDUSERFDTYPEDEVICESIZE/OFFNODENAME RaterServ29888rate3memREG64,0x61080064885874/opt/oracle/app/oracle/product/11.2.4/db_1/lib/libnnz11.so RaterServ29888rate3memREG64,0x6121875400324102/opt/oracle/app/oracle/product/11.2.4/db_1/lib/libclntsh.so.11.1
从上面的输出结果可以看出来确实是有进程在占用，进程号是29888。
再查29888对就的进程名
# ps -ef |grep 29888
找到是应用的一个服务，与测试沟通后，这个进程可以直接kill，于是
kill -9 29888
再次执行opatch apply成功。
