Secondly from GI PSU 11.2.0.3.5 and up the GI PSU have a separate GI and DB portions, which means there's patch number GI component number and DB component number. These components must be substitued in the commands used for manually applying the patch and these commands now come in generic form such as
<GI_HOME>/OPatch/opatch napply -oh <GI_HOME> -local <UNZIPPED_PATCH_LOCATION>/<GI_components_number> <GI_HOME>/OPatch/opatch apply -oh <GI_HOME> -local <UNZIPPED_PATCH_LOCATION>/<DB_PSU_number>Manually applying patch gives the DBA the control and also to closely examine what goes on.
"opatch auto" has it's merits as well. One command to patch both GI and DB Homes. It brings down the clusterware stack and start it up once finished. One draw back is that even if one component fails rest of the patching will continue and one has to come back to it at the end of the patching. Manual patching is more interactive allowing any errors/failures to be corrected before continuing with the rest of the patching.
Auto patching is run as root and switch (or execute as) between the appropriate user (grid or oracle) to the relevant home as it apply the patch. Opatch auto also need an ocm response file. How To Create An OCM Response File For Opatch Silent Installation [ID 966023.1]
Output on the shell where opatch auto is run only shows the outcome of each patchs' apply process (pass or fail).
[root@rhel6m1 patches]# /opt/app/11.2.0/grid/OPatch/opatch auto /usr/local/patches -ocmrf ocm.rsp Executing /usr/bin/perl /opt/app/11.2.0/grid/OPatch/crs/patch112.pl -patchdir /usr/local -patchn patches -ocmrf ocm.rsp -paramfile /opt/app/11.2.0/grid/crs/install/crsconfig_params opatch auto log file location is /opt/app/11.2.0/grid/OPatch/crs/../../cfgtoollogs/opatchauto2013-01-17_11-33-35.log Detected Oracle Clusterware install Using configuration parameter file: /opt/app/11.2.0/grid/crs/install/crsconfig_params patch /usr/local/patches/15876003/custom/server/15876003 apply successful for home /opt/app/oracle/product/11.2.0/dbhome_1 patch /usr/local/patches/14727310 apply successful for home /opt/app/oracle/product/11.2.0/dbhome_1 Successfully unlock /opt/app/11.2.0/grid patch /usr/local/patches/15876003 apply successful for home /opt/app/11.2.0/grid patch /usr/local/patches/14727310 apply successful for home /opt/app/11.2.0/grid ACFS-9300: ADVM/ACFS distribution files found. ACFS-9312: Existing ADVM/ACFS installation detected. ACFS-9314: Removing previous ADVM/ACFS installation. ACFS-9315: Previous ADVM/ACFS components successfully removed. ACFS-9307: Installing requested ADVM/ACFS software. ACFS-9308: Loading installed ADVM/ACFS drivers. ACFS-9321: Creating udev for ADVM/ACFS. ACFS-9323: Creating module dependencies - this may take some time. ACFS-9154: Loading 'oracleoks.ko' driver. ACFS-9154: Loading 'oracleadvm.ko' driver. ACFS-9154: Loading 'oracleacfs.ko' driver. ACFS-9327: Verifying ADVM/ACFS devices. ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'. ACFS-9156: Detecting control device '/dev/ofsctl'. ACFS-9309: ADVM/ACFS installation correctness verified. CRS-4123: Oracle High Availability Services has been started.After apply query the lsinventory for the PSU
$GI_HOME/OPatch/opatch lsinventory -local -bugs_fixed | grep "GRID INFRASTRUCTURE PATCH SET UPDATE" $ORACLE_HOME/OPatch/opatch lsinventory -local -bugs_fixed | grep "DATABASE PATCH SET UPDATE"It all comes down to personal preference, manual or auto but it won't be a surprise if eventually "auto" becomes the "recommended" way while manual is only for "troubleshooting".
Below is a full log of applying GI PSU 11.2.0.3.5 on RAC system running on RHEL6 which shows the behind the scene work of opatch auto such as alternating between grid and oracle user.
2013-01-17 11:33:35: Using Oracle CRS home /opt/app/11.2.0/grid 2013-01-17 11:33:35: Checking parameters from paramfile /opt/app/11.2.0/grid/crs/install/crsconfig_params to validate installer variables 2013-01-17 11:33:35: The configuration parameter file /opt/app/11.2.0/grid/crs/install/crsconfig_params is valid 2013-01-17 11:33:35: ### Printing the configuration values from files: 2013-01-17 11:33:35: /opt/app/11.2.0/grid/crs/install/crsconfig_params 2013-01-17 11:33:35: /opt/app/11.2.0/grid/OPatch/crs/s_crsconfig_defs 2013-01-17 11:33:35: ASM_AU_SIZE=1 2013-01-17 11:33:35: ASM_DISCOVERY_STRING=/dev/sd* 2013-01-17 11:33:35: ASM_DISKS=/dev/sdb1,/dev/sdc1,/dev/sdd1 2013-01-17 11:33:35: ASM_DISK_GROUP=CLUSTER_DG 2013-01-17 11:33:35: ASM_REDUNDANCY=NORMAL 2013-01-17 11:33:35: ASM_SPFILE= 2013-01-17 11:33:35: ASM_UPGRADE=false 2013-01-17 11:33:35: CLSCFG_MISSCOUNT= 2013-01-17 11:33:35: CLUSTER_GUID= 2013-01-17 11:33:35: CLUSTER_NAME=rhel6m-cluster 2013-01-17 11:33:35: CRFHOME="/opt/app/11.2.0/grid" 2013-01-17 11:33:35: CRS_LIMIT_CORE=unlimited 2013-01-17 11:33:35: CRS_LIMIT_MEMLOCK=unlimited 2013-01-17 11:33:35: CRS_LIMIT_OPENFILE=65536 2013-01-17 11:33:35: CRS_LIMIT_STACK=2048 2013-01-17 11:33:35: CRS_NODEVIPS='rhel6m1-vip/255.255.255.0/eth0,rhel6m2-vip/255.255.255.0/eth0' 2013-01-17 11:33:35: CRS_STORAGE_OPTION=1 2013-01-17 11:33:35: CSS_LEASEDURATION=400 2013-01-17 11:33:35: DIRPREFIX= 2013-01-17 11:33:35: DISABLE_OPROCD=0 2013-01-17 11:33:35: EXTERNAL_ORACLE=/opt/oracle 2013-01-17 11:33:35: EXTERNAL_ORACLE_BIN=/opt/oracle/bin 2013-01-17 11:33:35: GNS_ADDR_LIST= 2013-01-17 11:33:35: GNS_ALLOW_NET_LIST= 2013-01-17 11:33:35: GNS_CONF=false 2013-01-17 11:33:35: GNS_DENY_ITF_LIST= 2013-01-17 11:33:35: GNS_DENY_NET_LIST= 2013-01-17 11:33:35: GNS_DOMAIN_LIST= 2013-01-17 11:33:35: GPNPCONFIGDIR=/opt/app/11.2.0/grid 2013-01-17 11:33:35: GPNPGCONFIGDIR=/opt/app/11.2.0/grid 2013-01-17 11:33:35: GPNP_PA= 2013-01-17 11:33:35: HOST_NAME_LIST=rhel6m1,rhel6m2 2013-01-17 11:33:35: ID=/etc/init.d 2013-01-17 11:33:35: INIT=/sbin/init 2013-01-17 11:33:35: ISROLLING=true 2013-01-17 11:33:35: IT=/etc/inittab 2013-01-17 11:33:35: JLIBDIR=/opt/app/11.2.0/grid/jlib 2013-01-17 11:33:35: JREDIR=/opt/app/11.2.0/grid/jdk/jre/ 2013-01-17 11:33:35: LANGUAGE_ID=AMERICAN_AMERICA.AL32UTF8 2013-01-17 11:33:35: MSGFILE=/var/adm/messages 2013-01-17 11:33:35: NETWORKS="eth0"/192.168.0.0:public,"eth1"/192.168.1.0:cluster_interconnect 2013-01-17 11:33:35: NEW_HOST_NAME_LIST= 2013-01-17 11:33:35: NEW_NODEVIPS='rhel6m1-vip/255.255.255.0/eth0,rhel6m2-vip/255.255.255.0/eth0' 2013-01-17 11:33:35: NEW_NODE_NAME_LIST= 2013-01-17 11:33:35: NEW_PRIVATE_NAME_LIST= 2013-01-17 11:33:35: NODELIST=rhel6m1,rhel6m2 2013-01-17 11:33:35: NODE_NAME_LIST=rhel6m1,rhel6m2 2013-01-17 11:33:35: OCFS_CONFIG= 2013-01-17 11:33:35: OCRCONFIG=/etc/oracle/ocr.loc 2013-01-17 11:33:35: OCRCONFIGDIR=/etc/oracle 2013-01-17 11:33:35: OCRID= 2013-01-17 11:33:35: OCRLOC=ocr.loc 2013-01-17 11:33:35: OCR_LOCATIONS=NO_VAL 2013-01-17 11:33:35: OLASTGASPDIR=/etc/oracle/lastgasp 2013-01-17 11:33:35: OLD_CRS_HOME= 2013-01-17 11:33:35: OLRCONFIG=/etc/oracle/olr.loc 2013-01-17 11:33:35: OLRCONFIGDIR=/etc/oracle 2013-01-17 11:33:35: OLRLOC=olr.loc 2013-01-17 11:33:35: OPROCDCHECKDIR=/etc/oracle/oprocd/check 2013-01-17 11:33:35: OPROCDDIR=/etc/oracle/oprocd 2013-01-17 11:33:35: OPROCDFATALDIR=/etc/oracle/oprocd/fatal 2013-01-17 11:33:35: OPROCDSTOPDIR=/etc/oracle/oprocd/stop 2013-01-17 11:33:35: ORACLE_BASE=/opt/app/oracle 2013-01-17 11:33:35: ORACLE_HOME=/opt/app/11.2.0/grid 2013-01-17 11:33:35: ORACLE_OWNER=grid 2013-01-17 11:33:35: ORA_ASM_GROUP=asmadmin 2013-01-17 11:33:35: ORA_DBA_GROUP=oinstall 2013-01-17 11:33:35: PRIVATE_NAME_LIST= 2013-01-17 11:33:35: RCALLDIR=/etc/rc.d/rc0.d /etc/rc.d/rc1.d /etc/rc.d/rc2.d /etc/rc.d/rc3.d /etc/rc.d/rc4.d /etc/rc.d/rc5.d /etc/rc.d/rc6.d 2013-01-17 11:33:35: RCKDIR=/etc/rc.d/rc0.d /etc/rc.d/rc1.d /etc/rc.d/rc2.d /etc/rc.d/rc3.d /etc/rc.d/rc4.d /etc/rc.d/rc6.d 2013-01-17 11:33:35: RCSDIR=/etc/rc.d/rc3.d /etc/rc.d/rc5.d 2013-01-17 11:33:35: RC_KILL=K15 2013-01-17 11:33:35: RC_KILL_OLD=K96 2013-01-17 11:33:35: RC_KILL_OLD2=K19 2013-01-17 11:33:35: RC_START=S96 2013-01-17 11:33:35: REUSEDG=false 2013-01-17 11:33:35: SCAN_NAME=rhel6m-scan 2013-01-17 11:33:35: SCAN_PORT=1521 2013-01-17 11:33:35: SCRBASE=/etc/oracle/scls_scr 2013-01-17 11:33:35: SILENT=false 2013-01-17 11:33:35: SO_EXT=so 2013-01-17 11:33:35: SRVCFGLOC=srvConfig.loc 2013-01-17 11:33:35: SRVCONFIG=/var/opt/oracle/srvConfig.loc 2013-01-17 11:33:35: SRVCONFIGDIR=/var/opt/oracle 2013-01-17 11:33:35: TZ=Europe/London 2013-01-17 11:33:35: USER_IGNORED_PREREQ=false 2013-01-17 11:33:35: VNDR_CLUSTER=false 2013-01-17 11:33:35: VOTING_DISKS=NO_VAL 2013-01-17 11:33:35: ### Printing other configuration values ### 2013-01-17 11:33:35: CLSCFG_EXTRA_PARMS= 2013-01-17 11:33:35: HAS_GROUP=oinstall 2013-01-17 11:33:35: HAS_USER=root 2013-01-17 11:33:35: HOST=rhel6m1 2013-01-17 11:33:35: OLR_DIRECTORY=/opt/app/11.2.0/grid/cdata 2013-01-17 11:33:35: OLR_LOCATION=/opt/app/11.2.0/grid/cdata/rhel6m1.olr 2013-01-17 11:33:35: ORA_CRS_HOME=/opt/app/11.2.0/grid 2013-01-17 11:33:35: SUPERUSER=root 2013-01-17 11:33:35: UNLOCK=0 2013-01-17 11:33:35: VF_DISCOVERY_STRING= 2013-01-17 11:33:35: crscfg_trace=1 2013-01-17 11:33:35: crscfg_trace_file=/opt/app/11.2.0/grid/OPatch/crs/../../cfgtoollogs/opatchauto2013-01-17_11-33-35.log 2013-01-17 11:33:35: hosts= 2013-01-17 11:33:35: osdfile=/opt/app/11.2.0/grid/OPatch/crs/s_crsconfig_defs 2013-01-17 11:33:35: parameters_valid=1 2013-01-17 11:33:35: paramfile=/opt/app/11.2.0/grid/crs/install/crsconfig_params 2013-01-17 11:33:35: platform_family=unix 2013-01-17 11:33:35: srvctl_trc_suff=0 2013-01-17 11:33:35: user_is_superuser=1 2013-01-17 11:33:35: ### Printing of configuration values complete ### 2013-01-17 11:33:35: No -patchfile specified, assuming the patch is already uncompressed 2013-01-17 11:33:35: Oracle user for /opt/app/11.2.0/grid is grid 2013-01-17 11:33:35: silent mode option is -silent -ocmrf ocm.rsp 2013-01-17 11:33:35: Bundle.xml content is <bundle type = "GI_BUNDLE"> <entities> <entity location="15876003"> <target type="crs"/> <target type="siha"/> </entity> <entity location="15876003/custom/server/15876003"> <target type="rac" /> <target type="sidb"/> </entity> <entity location="14727310"> <target type="crs"/> <target type="rac"/> <target type="sidb"/> <target type="siha"/> </entity> </entities> </bundle> 2013-01-17 11:33:35: The patch ids are 15876003 14727310 2013-01-17 11:33:35: The patch ids are 15876003 14727310 2013-01-17 11:33:35: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch query -get_patch_type /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid 2013-01-17 11:33:35: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch query -get_patch_type /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid ' 2013-01-17 11:33:39: Removing file /tmp/fileSKWEXl 2013-01-17 11:33:39: Successfully removed file: /tmp/fileSKWEXl 2013-01-17 11:33:39: /bin/su successfully executed 2013-01-17 11:33:39: output is This patch is a "legacy_bundle_top" patch. 2013-01-17 11:33:39: Patch type is "legacy_bundle_top" 2013-01-17 11:33:39: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch query -get_patch_type /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid 2013-01-17 11:33:39: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch query -get_patch_type /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid ' 2013-01-17 11:33:40: Removing file /tmp/file1oyqPq 2013-01-17 11:33:40: Successfully removed file: /tmp/file1oyqPq 2013-01-17 11:33:40: /bin/su exited with rc=0 75 2013-01-17 11:33:40: output is 2013-01-17 11:33:40: Patch type is 2013-01-17 11:33:40: GI patches are /usr/local/patches/15876003 /usr/local/patches/14727310 2013-01-17 11:33:40: DB patches are /usr/local/patches/15876003/custom/server/15876003 /usr/local/patches/14727310 2013-01-17 11:33:40: Running /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1 2013-01-17 11:33:40: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1 2013-01-17 11:33:40: Command output: > ************************************************************** > rhel6m1: > CRS-4537: Cluster Ready Services is online > CRS-4529: Cluster Synchronization Services is online > CRS-4533: Event Manager is online > ************************************************************** >End Command output 2013-01-17 11:33:40: Looking for configured databases on node rhel6m1 2013-01-17 11:33:41: Databases configured on node rhel6m1 are: std11g2 2013-01-17 11:33:41: Determining ORACLE_HOME paths for configured databases 2013-01-17 11:33:41: Executing cmd: /opt/app/11.2.0/grid/bin/srvctl config database -d std11g2 2013-01-17 11:33:43: Command output: > Database unique name: std11g2 > Database name: std11g2 > Oracle home: /opt/app/oracle/product/11.2.0/dbhome_1 > Oracle user: oracle > Spfile: +DATA/std11g2/spfilestd11g2.ora > Domain: > Start options: open > Stop options: immediate > Database role: PRIMARY > Management policy: AUTOMATIC > Server pools: std11g2 > Database instances: std11g21,std11g22 > Disk Groups: DATA,FLASH > Mount point paths: > Services: myservice > Type: RAC > Database is administrator managed >End Command output 2013-01-17 11:33:43: output is Oracle home: /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:33:43: Oracle home for database std11g2 is /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:33:43: Oracle Home /opt/app/oracle/product/11.2.0/dbhome_1 is configured with Database(s)-> std11g2 2013-01-17 11:33:43: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle 2013-01-17 11:33:43: oracle home list is /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:33:43: Processing oracle home /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:33:43: Opening file /etc/oracle/ocr.loc 2013-01-17 11:33:43: Value (FALSE) is set for key=local_only 2013-01-17 11:33:43: Home type of /opt/app/oracle/product/11.2.0/dbhome_1 is DB 2013-01-17 11:33:43: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle 2013-01-17 11:33:43: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle 2013-01-17 11:33:43: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch version -oh /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:33:43: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch version -oh /opt/app/oracle/product/11.2.0/dbhome_1 ' 2013-01-17 11:33:44: Removing file /tmp/fileTZwL3G 2013-01-17 11:33:44: Successfully removed file: /tmp/fileTZwL3G 2013-01-17 11:33:44: /bin/su successfully executed 2013-01-17 11:33:44: opatch version in oracle home /opt/app/oracle/product/11.2.0/dbhome_1 is 11.2.0.3.0 2013-01-17 11:33:44: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/15876003/custom/server/15876003 -version 11.2.0.3.0 -oh /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:33:44: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/15876003/custom/server/15876003 -version 11.2.0.3.0 -oh /opt/app/oracle/product/11.2.0/dbhome_1 ' 2013-01-17 11:33:49: Removing file /tmp/filei5JgK0 2013-01-17 11:33:49: Successfully removed file: /tmp/filei5JgK0 2013-01-17 11:33:49: /bin/su successfully executed 2013-01-17 11:33:49: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/14727310 -version 11.2.0.3.0 -oh /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:33:49: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/14727310 -version 11.2.0.3.0 -oh /opt/app/oracle/product/11.2.0/dbhome_1 ' 2013-01-17 11:33:52: Removing file /tmp/filekjM3wv 2013-01-17 11:33:52: Successfully removed file: /tmp/filekjM3wv 2013-01-17 11:33:52: /bin/su successfully executed 2013-01-17 11:33:52: Status of opatch version check for /opt/app/oracle/product/11.2.0/dbhome_1 is 1 2013-01-17 11:33:52: Opatch version check passed for oracle home /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:33:52: Processing oracle home /opt/app/11.2.0/grid 2013-01-17 11:33:52: Opening file /etc/oracle/ocr.loc 2013-01-17 11:33:52: Value (FALSE) is set for key=local_only 2013-01-17 11:33:52: Home type of /opt/app/11.2.0/grid is CRS 2013-01-17 11:33:52: Oracle user for /opt/app/11.2.0/grid is grid 2013-01-17 11:33:52: Oracle user for /opt/app/11.2.0/grid is grid 2013-01-17 11:33:52: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch version -oh /opt/app/11.2.0/grid 2013-01-17 11:33:52: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch version -oh /opt/app/11.2.0/grid ' 2013-01-17 11:33:53: Removing file /tmp/fileQxFgZ8 2013-01-17 11:33:53: Successfully removed file: /tmp/fileQxFgZ8 2013-01-17 11:33:53: /bin/su successfully executed 2013-01-17 11:33:53: opatch version in oracle home /opt/app/11.2.0/grid is 11.2.0.3.0 2013-01-17 11:33:53: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/15876003 -version 11.2.0.3.0 -oh /opt/app/11.2.0/grid 2013-01-17 11:33:53: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/15876003 -version 11.2.0.3.0 -oh /opt/app/11.2.0/grid ' 2013-01-17 11:33:57: Removing file /tmp/file1noRAO 2013-01-17 11:33:57: Successfully removed file: /tmp/file1noRAO 2013-01-17 11:33:57: /bin/su successfully executed 2013-01-17 11:33:57: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/14727310 -version 11.2.0.3.0 -oh /opt/app/11.2.0/grid 2013-01-17 11:33:57: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/14727310 -version 11.2.0.3.0 -oh /opt/app/11.2.0/grid ' 2013-01-17 11:34:00: Removing file /tmp/file3hDQCF 2013-01-17 11:34:00: Successfully removed file: /tmp/file3hDQCF 2013-01-17 11:34:00: /bin/su successfully executed 2013-01-17 11:34:00: Status of opatch version check for /opt/app/11.2.0/grid is 1 2013-01-17 11:34:00: Opatch version check passed for oracle home /opt/app/11.2.0/grid 2013-01-17 11:34:00: Opatch version check passed for all oracle homes 2013-01-17 11:34:00: Opening file /etc/oracle/ocr.loc 2013-01-17 11:34:00: Value (FALSE) is set for key=local_only 2013-01-17 11:34:00: The cluster nodes are rhel6m1 rhel6m2 2013-01-17 11:34:00: checking if path /opt/app/oracle/product/11.2.0/dbhome_1 is shared 2013-01-17 11:34:00: Running as user grid: /opt/app/11.2.0/grid/bin/cluvfy comp ssa -t software -s /opt/app/oracle/product/11.2.0/dbhome_1 -n rhel6m1,rhel6m2 -display_status 2013-01-17 11:34:00: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/bin/cluvfy comp ssa -t software -s /opt/app/oracle/product/11.2.0/dbhome_1 -n rhel6m1,rhel6m2 -display_status ' 2013-01-17 11:35:02: Removing file /tmp/filejzzDOD 2013-01-17 11:35:02: Successfully removed file: /tmp/filejzzDOD 2013-01-17 11:35:02: /bin/su exited with rc=0 1 2013-01-17 11:35:02: return code for shared check is 256 2013-01-17 11:35:02: output of sharedness check is Verifying shared storage accessibility Checking shared storage accessibility... "/opt/app/oracle/product/11.2.0/dbhome_1" is not shared Shared storage check failed on nodes "rhel6m2,rhel6m1" Verification of shared storage accessibility was unsuccessful on all the specified nodes. NODE_STATUS::rhel6m2:VFAIL NODE_STATUS::rhel6m1:VFAIL OVERALL_STATUS::VFAIL 2013-01-17 11:35:02: the ishared value is 0 2013-01-17 11:35:02: The oracle home /opt/app/oracle/product/11.2.0/dbhome_1 is not shared 2013-01-17 11:35:02: Opening file /etc/oracle/ocr.loc 2013-01-17 11:35:02: Value (FALSE) is set for key=local_only 2013-01-17 11:35:02: The cluster nodes are rhel6m1 rhel6m2 2013-01-17 11:35:02: checking if path /opt/app/11.2.0/grid/crs/install is shared 2013-01-17 11:35:02: Running as user grid: /opt/app/11.2.0/grid/bin/cluvfy comp ssa -t software -s /opt/app/11.2.0/grid/crs/install -n rhel6m1,rhel6m2 -display_status 2013-01-17 11:35:02: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/bin/cluvfy comp ssa -t software -s /opt/app/11.2.0/grid/crs/install -n rhel6m1,rhel6m2 -display_status ' 2013-01-17 11:35:21: Removing file /tmp/file4Xaqdc 2013-01-17 11:35:21: Successfully removed file: /tmp/file4Xaqdc 2013-01-17 11:35:21: /bin/su exited with rc=0 1 2013-01-17 11:35:21: return code for shared check is 256 2013-01-17 11:35:21: output of sharedness check is Verifying shared storage accessibility Checking shared storage accessibility... "/opt/app/11.2.0/grid/crs/install" is not shared Shared storage check failed on nodes "rhel6m2,rhel6m1" Verification of shared storage accessibility was unsuccessful on all the specified nodes. NODE_STATUS::rhel6m2:VFAIL NODE_STATUS::rhel6m1:VFAIL OVERALL_STATUS::VFAIL 2013-01-17 11:35:21: the ishared value is 0 2013-01-17 11:35:21: The oracle home /opt/app/11.2.0/grid is not shared 2013-01-17 11:35:21: Processing oracle home /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:35:21: Opening file /etc/oracle/ocr.loc 2013-01-17 11:35:21: Value (FALSE) is set for key=local_only 2013-01-17 11:35:21: Home type of /opt/app/oracle/product/11.2.0/dbhome_1 is DB 2013-01-17 11:35:21: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle 2013-01-17 11:35:21: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/15876003/custom/server/15876003 -oh /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:35:21: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/15876003/custom/server/15876003 -oh /opt/app/oracle/product/11.2.0/dbhome_1 ' 2013-01-17 11:35:24: Removing file /tmp/filedswEgx 2013-01-17 11:35:24: Successfully removed file: /tmp/filedswEgx 2013-01-17 11:35:24: /bin/su successfully executed 2013-01-17 11:35:24: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/14727310 -oh /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:35:24: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/14727310 -oh /opt/app/oracle/product/11.2.0/dbhome_1 ' 2013-01-17 11:35:28: Removing file /tmp/filehlvsO1 2013-01-17 11:35:28: Successfully removed file: /tmp/filehlvsO1 2013-01-17 11:35:28: /bin/su successfully executed 2013-01-17 11:35:28: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle 2013-01-17 11:35:28: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/15876003/custom/server/15876003 -oh /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:35:28: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/15876003/custom/server/15876003 -oh /opt/app/oracle/product/11.2.0/dbhome_1 ' 2013-01-17 11:35:35: Removing file /tmp/fileBBS2qF 2013-01-17 11:35:35: Successfully removed file: /tmp/fileBBS2qF 2013-01-17 11:35:35: /bin/su successfully executed 2013-01-17 11:35:35: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/14727310 -oh /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:35:35: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/14727310 -oh /opt/app/oracle/product/11.2.0/dbhome_1 ' 2013-01-17 11:35:42: Removing file /tmp/file1i2Z7B 2013-01-17 11:35:42: Successfully removed file: /tmp/file1i2Z7B 2013-01-17 11:35:42: /bin/su successfully executed 2013-01-17 11:35:42: Status of component/conflict check for /opt/app/oracle/product/11.2.0/dbhome_1 is 1 2013-01-17 11:35:42: Conflict check passes for oracle home /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:35:42: Processing oracle home /opt/app/11.2.0/grid 2013-01-17 11:35:42: Opening file /etc/oracle/ocr.loc 2013-01-17 11:35:42: Value (FALSE) is set for key=local_only 2013-01-17 11:35:42: Home type of /opt/app/11.2.0/grid is CRS 2013-01-17 11:35:42: Oracle user for /opt/app/11.2.0/grid is grid 2013-01-17 11:35:42: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid 2013-01-17 11:35:42: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid ' 2013-01-17 11:35:45: Removing file /tmp/filekTThuP 2013-01-17 11:35:45: Successfully removed file: /tmp/filekTThuP 2013-01-17 11:35:45: /bin/su successfully executed 2013-01-17 11:35:45: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid 2013-01-17 11:35:45: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid ' 2013-01-17 11:35:48: Removing file /tmp/filersHlyc 2013-01-17 11:35:48: Successfully removed file: /tmp/filersHlyc 2013-01-17 11:35:48: /bin/su successfully executed 2013-01-17 11:35:48: Oracle user for /opt/app/11.2.0/grid is grid 2013-01-17 11:35:48: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid 2013-01-17 11:35:48: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid ' 2013-01-17 11:35:56: Removing file /tmp/fileMd0bDH 2013-01-17 11:35:56: Successfully removed file: /tmp/fileMd0bDH 2013-01-17 11:35:56: /bin/su successfully executed 2013-01-17 11:35:56: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid 2013-01-17 11:35:56: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid ' 2013-01-17 11:36:03: Removing file /tmp/fileoLmZhv 2013-01-17 11:36:03: Successfully removed file: /tmp/fileoLmZhv 2013-01-17 11:36:03: /bin/su successfully executed 2013-01-17 11:36:03: Status of component/conflict check for /opt/app/11.2.0/grid is 1 2013-01-17 11:36:03: Conflict check passes for oracle home /opt/app/11.2.0/grid 2013-01-17 11:36:03: Conflict check passed for all oracle homes 2013-01-17 11:36:03: Processing oracle home /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:36:03: Opening file /etc/oracle/ocr.loc 2013-01-17 11:36:03: Value (FALSE) is set for key=local_only 2013-01-17 11:36:03: Home type of /opt/app/oracle/product/11.2.0/dbhome_1 is DB 2013-01-17 11:36:03: Performing DB patch 2013-01-17 11:36:03: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle 2013-01-17 11:36:03: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/15876003/custom/server/15876003 -oh /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:36:03: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/15876003/custom/server/15876003 -oh /opt/app/oracle/product/11.2.0/dbhome_1 ' 2013-01-17 11:36:07: Removing file /tmp/file82jfJB 2013-01-17 11:36:07: Successfully removed file: /tmp/file82jfJB 2013-01-17 11:36:07: /bin/su successfully executed 2013-01-17 11:36:07: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/14727310 -oh /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:36:07: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/14727310 -oh /opt/app/oracle/product/11.2.0/dbhome_1 ' 2013-01-17 11:36:11: Removing file /tmp/filegeYkOS 2013-01-17 11:36:11: Successfully removed file: /tmp/filegeYkOS 2013-01-17 11:36:11: /bin/su successfully executed 2013-01-17 11:36:11: Status of Applicable check for /opt/app/oracle/product/11.2.0/dbhome_1 is 1 2013-01-17 11:36:11: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl query crs activeversion 2013-01-17 11:36:11: Command output: > Oracle Clusterware active version on the cluster is [11.2.0.3.0] >End Command output 2013-01-17 11:36:11: crs version is 11 2 0 3 0 2013-01-17 11:36:11: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/bin/srvctl stop home -o /opt/app/oracle/product/11.2.0/dbhome_1 -s /opt/app/oracle/product/11.2.0/dbhome_1/srvm/admin/stophome.txt -n rhel6m1 2013-01-17 11:36:11: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/bin/srvctl stop home -o /opt/app/oracle/product/11.2.0/dbhome_1 -s /opt/app/oracle/product/11.2.0/dbhome_1/srvm/admin/stophome.txt -n rhel6m1 ' 2013-01-17 11:36:29: Removing file /tmp/filegXdp4j 2013-01-17 11:36:29: Successfully removed file: /tmp/filegXdp4j 2013-01-17 11:36:29: /bin/su successfully executed 2013-01-17 11:36:29: /opt/app/oracle/product/11.2.0/dbhome_1/bin/srvctl stop home -o /opt/app/oracle/product/11.2.0/dbhome_1 -s /opt/app/oracle/product/11.2.0/dbhome_1/srvm/admin/stophome.txt -n rhel6m1 output is 2013-01-17 11:36:29: Stopped resources from datbase home /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:36:29: Running as user oracle: /usr/local/patches/15876003/custom/server/15876003/custom/scripts/prepatch.sh -dbhome /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:36:29: s_run_as_user2: Running /bin/su oracle -c ' /usr/local/patches/15876003/custom/server/15876003/custom/scripts/prepatch.sh -dbhome /opt/app/oracle/product/11.2.0/dbhome_1 ' 2013-01-17 11:36:29: Removing file /tmp/filePZl0Xw 2013-01-17 11:36:29: Successfully removed file: /tmp/filePZl0Xw 2013-01-17 11:36:29: /bin/su successfully executed 2013-01-17 11:36:29: Running as user oracle: true 2013-01-17 11:36:29: s_run_as_user2: Running /bin/su oracle -c ' true ' 2013-01-17 11:36:29: Removing file /tmp/fileLQZBdK 2013-01-17 11:36:29: Successfully removed file: /tmp/fileLQZBdK 2013-01-17 11:36:29: /bin/su successfully executed 2013-01-17 11:36:29: prepatch execution for DB home ... success 2013-01-17 11:36:29: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle 2013-01-17 11:36:29: Executing command /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply /usr/local/patches/15876003/custom/server/15876003 -local -silent -ocmrf ocm.rsp -oh /opt/app/oracle/product/11.2.0/dbhome_1 as oracle 2013-01-17 11:36:29: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply /usr/local/patches/15876003/custom/server/15876003 -local -silent -ocmrf ocm.rsp -oh /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:36:29: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply /usr/local/patches/15876003/custom/server/15876003 -local -silent -ocmrf ocm.rsp -oh /opt/app/oracle/product/11.2.0/dbhome_1 ' 2013-01-17 11:37:17: Removing file /tmp/file2D3exX 2013-01-17 11:37:17: Successfully removed file: /tmp/file2D3exX 2013-01-17 11:37:17: /bin/su successfully executed 2013-01-17 11:37:17: status of apply patch is 0 2013-01-17 11:37:17: The apply patch output is Oracle Interim Patch Installer version 11.2.0.3.0 Copyright (c) 2012, Oracle Corporation. All rights reserved. Oracle Home : /opt/app/oracle/product/11.2.0/dbhome_1 Central Inventory : /opt/app/oraInventory from : /opt/app/oracle/product/11.2.0/dbhome_1/oraInst.loc OPatch version : 11.2.0.3.0 OUI version : 11.2.0.3.0 Log file location : /opt/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2013-01-17_11-36-30AM_1.log Verifying environment and performing prerequisite checks... Conflicts/Supersets for each patch are: Patch : 15876003 Bug Superset of 14275572 Super set bugs are: 14275572, 13919095, 13696251, 13348650, 12659561, 13039908, 13825231, 13036424, 12794268, 13011520, 13569812, 12758736, 13000491, 13498267, 13077654, 13001901, 13550689, 13430715, 13806545, 11675721, 14082976, 12771830, 12538907, 13947200, 13066371, 13483672, 12594616, 13540563, 12897651, 12897902, 13241779, 12896850, 12726222, 12829429, 12728585, 13079948, 12876314, 13090686, 12925041, 12995950, 13251796, 12650672, 12398492, 12848480, 13582411, 13652088, 12990582, 13857364, 12975811, 12917897, 13082238, 12947871, 13037709, 13371153, 12878750, 10114953, 11772838, 13058611, 13001955, 11836951, 12965049, 13440962, 12765467, 13727853, 13425727, 12885323, 13965075, 13339443, 12784559, 13332363, 13074261, 12971251, 13811209, 12709476, 13460353, 13523527, 12857064, 13719731, 13396284, 12899169, 13111013, 13323698, 12867511, 12639013, 12959140, 13085732, 12829917, 10317921, 13843080, 12934171, 12849377, 12349553, 13924431, 13869978, 12680491, 12914824, 13789135, 12730342, 13334158, 12950823, 10418841, 13355963, 13531373, 13776758, 12720728, 13620816, 13002015, 13023609, 13024624, 12791719 Patches [ 14275572 ] will be rolled back. Do you want to proceed? [y|n] Y (auto-answered by -silent) User Responded with: Y OPatch continues with these patches: 15876003 Do you want to proceed? [y|n] Y (auto-answered by -silent) User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/opt/app/oracle/product/11.2.0/dbhome_1') Is the local system ready for patching? [y|n] Y (auto-answered by -silent) User Responded with: Y Backing up files... Applying interim patch '15876003' to OH '/opt/app/oracle/product/11.2.0/dbhome_1' Rolling back interim patch '14275572' from OH '/opt/app/oracle/product/11.2.0/dbhome_1' Patching component oracle.rdbms, 11.2.0.3.0... RollbackSession removing interim patch '14275572' from inventory OPatch back to application of the patch '15876003' after auto-rollback. Patching component oracle.rdbms, 11.2.0.3.0... Verifying the update... Patch 15876003 successfully applied. OPatch Session completed with warnings. Log file location: /opt/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2013-01-17_11-36-30AM_1.log OPatch completed with warnings. 2013-01-17 11:37:17: patch /usr/local/patches/15876003/custom/server/15876003 apply successful for home /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:37:17: Executing command /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply /usr/local/patches/14727310 -local -silent -ocmrf ocm.rsp -oh /opt/app/oracle/product/11.2.0/dbhome_1 as oracle 2013-01-17 11:37:17: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply /usr/local/patches/14727310 -local -silent -ocmrf ocm.rsp -oh /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:37:17: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply /usr/local/patches/14727310 -local -silent -ocmrf ocm.rsp -oh /opt/app/oracle/product/11.2.0/dbhome_1 ' 2013-01-17 11:39:59: Removing file /tmp/file5tU7W9 2013-01-17 11:39:59: Successfully removed file: /tmp/file5tU7W9 2013-01-17 11:39:59: /bin/su successfully executed 2013-01-17 11:39:59: status of apply patch is 0 2013-01-17 11:39:59: The apply patch output is Oracle Interim Patch Installer version 11.2.0.3.0 Copyright (c) 2012, Oracle Corporation. All rights reserved. Oracle Home : /opt/app/oracle/product/11.2.0/dbhome_1 Central Inventory : /opt/app/oraInventory from : /opt/app/oracle/product/11.2.0/dbhome_1/oraInst.loc OPatch version : 11.2.0.3.0 OUI version : 11.2.0.3.0 Log file location : /opt/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2013-01-17_11-37-18AM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 14727310 Do you want to proceed? [y|n] Y (auto-answered by -silent) User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/opt/app/oracle/product/11.2.0/dbhome_1') Is the local system ready for patching? [y|n] Y (auto-answered by -silent) User Responded with: Y Backing up files... Applying sub-patch '14727310' to OH '/opt/app/oracle/product/11.2.0/dbhome_1' Patching component oracle.rdbms, 11.2.0.3.0... Patching component oracle.rdbms.dbscripts, 11.2.0.3.0... Patching component oracle.rdbms.deconfig, 11.2.0.3.0... Patching component oracle.rdbms.rsf, 11.2.0.3.0... Patching component oracle.sdo.locator, 11.2.0.3.0... Patching component oracle.sysman.console.db, 11.2.0.3.0... Patching component oracle.sysman.oms.core, 10.2.0.4.4... Verifying the update... Composite patch 14727310 successfully applied. Log file location: /opt/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2013-01-17_11-37-18AM_1.log OPatch succeeded. 2013-01-17 11:39:59: patch /usr/local/patches/14727310 apply successful for home /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:39:59: Running as user oracle: /usr/local/patches/15876003/custom/server/15876003/custom/scripts/postpatch.sh -dbhome /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:39:59: s_run_as_user2: Running /bin/su oracle -c ' /usr/local/patches/15876003/custom/server/15876003/custom/scripts/postpatch.sh -dbhome /opt/app/oracle/product/11.2.0/dbhome_1 ' 2013-01-17 11:39:59: Removing file /tmp/fileeOwh17 2013-01-17 11:39:59: Successfully removed file: /tmp/fileeOwh17 2013-01-17 11:39:59: /bin/su successfully executed 2013-01-17 11:39:59: Running as user oracle: true 2013-01-17 11:39:59: s_run_as_user2: Running /bin/su oracle -c ' true ' 2013-01-17 11:39:59: Removing file /tmp/file3qIa16 2013-01-17 11:39:59: Successfully removed file: /tmp/file3qIa16 2013-01-17 11:39:59: /bin/su successfully executed 2013-01-17 11:39:59: postpatch execution for DB home ... success 2013-01-17 11:39:59: Processing oracle home /opt/app/11.2.0/grid 2013-01-17 11:39:59: Opening file /etc/oracle/ocr.loc 2013-01-17 11:39:59: Value (FALSE) is set for key=local_only 2013-01-17 11:39:59: Home type of /opt/app/11.2.0/grid is CRS 2013-01-17 11:39:59: Unlock crshome... 2013-01-17 11:39:59: Exclude file used is /opt/app/11.2.0/grid/OPatch/crs/installPatch.excl 2013-01-17 11:39:59: Home location in olr.loc is 2013-01-17 11:39:59: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stop crs -f 2013-01-17 11:41:59: Command output: > CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel6m1' > CRS-2673: Attempting to stop 'ora.crsd' on 'rhel6m1' > CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rhel6m1' > CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rhel6m1' > CRS-2673: Attempting to stop 'ora.CLUSTER_DG.dg' on 'rhel6m1' > CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rhel6m1' > CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rhel6m1' > CRS-2673: Attempting to stop 'ora.FLASH.dg' on 'rhel6m1' > CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rhel6m1' succeeded > CRS-2673: Attempting to stop 'ora.rhel6m1.vip' on 'rhel6m1' > CRS-2677: Stop of 'ora.rhel6m1.vip' on 'rhel6m1' succeeded > CRS-2672: Attempting to start 'ora.rhel6m1.vip' on 'rhel6m2' > CRS-2677: Stop of 'ora.DATA.dg' on 'rhel6m1' succeeded > CRS-2677: Stop of 'ora.registry.acfs' on 'rhel6m1' succeeded > CRS-2676: Start of 'ora.rhel6m1.vip' on 'rhel6m2' succeeded > CRS-2677: Stop of 'ora.FLASH.dg' on 'rhel6m1' succeeded > CRS-2677: Stop of 'ora.CLUSTER_DG.dg' on 'rhel6m1' succeeded > CRS-2673: Attempting to stop 'ora.asm' on 'rhel6m1' > CRS-2677: Stop of 'ora.asm' on 'rhel6m1' succeeded > CRS-2673: Attempting to stop 'ora.ons' on 'rhel6m1' > CRS-2677: Stop of 'ora.ons' on 'rhel6m1' succeeded > CRS-2673: Attempting to stop 'ora.net1.network' on 'rhel6m1' > CRS-2677: Stop of 'ora.net1.network' on 'rhel6m1' succeeded > CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rhel6m1' has completed > CRS-2677: Stop of 'ora.crsd' on 'rhel6m1' succeeded > CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rhel6m1' > CRS-2673: Attempting to stop 'ora.mdnsd' on 'rhel6m1' > CRS-2673: Attempting to stop 'ora.crf' on 'rhel6m1' > CRS-2673: Attempting to stop 'ora.ctssd' on 'rhel6m1' > CRS-2673: Attempting to stop 'ora.evmd' on 'rhel6m1' > CRS-2673: Attempting to stop 'ora.asm' on 'rhel6m1' > CRS-2677: Stop of 'ora.crf' on 'rhel6m1' succeeded > CRS-2677: Stop of 'ora.mdnsd' on 'rhel6m1' succeeded > CRS-2677: Stop of 'ora.evmd' on 'rhel6m1' succeeded > CRS-2677: Stop of 'ora.asm' on 'rhel6m1' succeeded > CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rhel6m1' > CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rhel6m1' succeeded > CRS-2677: Stop of 'ora.ctssd' on 'rhel6m1' succeeded > CRS-2673: Attempting to stop 'ora.cssd' on 'rhel6m1' > CRS-2677: Stop of 'ora.drivers.acfs' on 'rhel6m1' succeeded > CRS-2677: Stop of 'ora.cssd' on 'rhel6m1' succeeded > CRS-2673: Attempting to stop 'ora.gipcd' on 'rhel6m1' > CRS-2677: Stop of 'ora.gipcd' on 'rhel6m1' succeeded > CRS-2673: Attempting to stop 'ora.gpnpd' on 'rhel6m1' > CRS-2677: Stop of 'ora.gpnpd' on 'rhel6m1' succeeded > CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel6m1' has completed > CRS-4133: Oracle High Availability Services has been stopped. >End Command output 2013-01-17 11:41:59: /opt/app/11.2.0/grid/bin/crsctl stop crs -f 2013-01-17 11:41:59: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1 2013-01-17 11:42:00: Command output: > CRS-4639: Could not contact Oracle High Availability Services > CRS-4000: Command Check failed, or completed with errors. >End Command output 2013-01-17 11:42:01: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check has 2013-01-17 11:42:02: Command output: > CRS-4639: Could not contact Oracle High Availability Services >End Command output 2013-01-17 11:44:10: Waiting for complete CRS stack to stop 2013-01-17 11:44:10: Invoking removeproc to clean oracle client procs 2013-01-17 11:44:10: Executing cmd: /sbin/fuser -k /opt/app/11.2.0/grid/bin/crsctl.bin 2013-01-17 11:44:10: fuser command output for /opt/app/11.2.0/grid/bin/crsctl.bin is 2013-01-17 11:44:10: Oracle user for /opt/app/11.2.0/grid is grid 2013-01-17 11:44:10: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid 2013-01-17 11:44:10: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid ' 2013-01-17 11:44:14: Removing file /tmp/fileCAOkny 2013-01-17 11:44:14: Successfully removed file: /tmp/fileCAOkny 2013-01-17 11:44:14: /bin/su successfully executed 2013-01-17 11:44:14: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid 2013-01-17 11:44:14: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid ' 2013-01-17 11:44:17: Removing file /tmp/filepYiTfa 2013-01-17 11:44:17: Successfully removed file: /tmp/filepYiTfa 2013-01-17 11:44:17: /bin/su successfully executed 2013-01-17 11:44:17: Status of Applicable check for /opt/app/11.2.0/grid is 1 2013-01-17 11:44:17: Oracle user for /opt/app/11.2.0/grid is grid 2013-01-17 11:44:17: Executing command /opt/app/11.2.0/grid/OPatch/opatch napply /usr/local/patches/15876003 -local -silent -ocmrf ocm.rsp -oh /opt/app/11.2.0/grid as grid 2013-01-17 11:44:17: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch napply /usr/local/patches/15876003 -local -silent -ocmrf ocm.rsp -oh /opt/app/11.2.0/grid 2013-01-17 11:44:17: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch napply /usr/local/patches/15876003 -local -silent -ocmrf ocm.rsp -oh /opt/app/11.2.0/grid ' 2013-01-17 11:48:18: Removing file /tmp/fileftFTxU 2013-01-17 11:48:18: Successfully removed file: /tmp/fileftFTxU 2013-01-17 11:48:18: /bin/su successfully executed 2013-01-17 11:48:18: status of apply patch is 0 2013-01-17 11:48:18: The apply patch output is Oracle Interim Patch Installer version 11.2.0.3.0 Copyright (c) 2012, Oracle Corporation. All rights reserved. Oracle Home : /opt/app/11.2.0/grid Central Inventory : /opt/app/oraInventory from : /opt/app/11.2.0/grid/oraInst.loc OPatch version : 11.2.0.3.0 OUI version : 11.2.0.3.0 Log file location : /opt/app/11.2.0/grid/cfgtoollogs/opatch/opatch2013-01-17_11-44-18AM_1.log Verifying environment and performing prerequisite checks... Conflicts/Supersets for each patch are: Patch : 15876003 Bug Superset of 14275572 Super set bugs are: 14275572, 13919095, 13696251, 13348650, 12659561, 13039908, 13825231, 13036424, 12794268, 13011520, 13569812, 12758736, 13000491, 13498267, 13077654, 13001901, 13550689, 13430715, 13806545, 11675721, 14082976, 12771830, 12538907, 13947200, 13066371, 13483672, 12594616, 13540563, 12897651, 12897902, 13241779, 12896850, 12726222, 12829429, 12728585, 13079948, 12876314, 13090686, 12925041, 12995950, 13251796, 12650672, 12398492, 12848480, 13582411, 13652088, 12990582, 13857364, 12975811, 12917897, 13082238, 12947871, 13037709, 13371153, 12878750, 10114953, 11772838, 13058611, 13001955, 11836951, 12965049, 13440962, 12765467, 13727853, 13425727, 12885323, 13965075, 13339443, 12784559, 13332363, 13074261, 12971251, 13811209, 12709476, 13460353, 13523527, 12857064, 13719731, 13396284, 12899169, 13111013, 13323698, 12867511, 12639013, 12959140, 13085732, 12829917, 10317921, 13843080, 12934171, 12849377, 12349553, 13924431, 13869978, 12680491, 12914824, 13789135, 12730342, 13334158, 12950823, 10418841, 13355963, 13531373, 13776758, 12720728, 13620816, 13002015, 13023609, 13024624, 12791719, 13886023, 13255295, 13821454, 12782756, 14625969, 14152875, 14186070, 12873909, 14214257, 12914722, 13243172, 12842804, 13045518, 12765868, 12772345, 12663376, 13345868, 14059576, 13683090, 12932852, 13889047, 12695029, 13146560, 13038806, 14251904, 14070200, 13820621, 14304758, 13396356, 13697828, 13258062, 12834777, 12996572, 13941934, 13657366, 13019958, 12810890, 13888719, 13502441, 13726162, 13880925, 14153867, 13506114, 12820045, 13604057, 12823838, 13877508, 12823042, 14494305, 13582706, 13617861, 12825835, 13263435, 13025879, 13853089, 14009845, 13410987, 13570879, 13637590, 12827493, 13247273, 13068077 Patches [ 14275572 ] will be rolled back. Do you want to proceed? [y|n] Y (auto-answered by -silent) User Responded with: Y OPatch continues with these patches: 15876003 Do you want to proceed? [y|n] Y (auto-answered by -silent) User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/opt/app/11.2.0/grid') Is the local system ready for patching? [y|n] Y (auto-answered by -silent) User Responded with: Y Backing up files... Applying interim patch '15876003' to OH '/opt/app/11.2.0/grid' Rolling back interim patch '14275572' from OH '/opt/app/11.2.0/grid' Patching component oracle.crs, 11.2.0.3.0... Patching component oracle.usm, 11.2.0.3.0... RollbackSession removing interim patch '14275572' from inventory OPatch back to application of the patch '15876003' after auto-rollback. Patching component oracle.crs, 11.2.0.3.0... Patching component oracle.usm, 11.2.0.3.0... Verifying the update... Patch 15876003 successfully applied. OPatch Session completed with warnings. Log file location: /opt/app/11.2.0/grid/cfgtoollogs/opatch/opatch2013-01-17_11-44-18AM_1.log OPatch completed with warnings. 2013-01-17 11:48:18: patch /usr/local/patches/15876003 apply successful for home /opt/app/11.2.0/grid 2013-01-17 11:48:18: Executing command /opt/app/11.2.0/grid/OPatch/opatch napply /usr/local/patches/14727310 -local -silent -ocmrf ocm.rsp -oh /opt/app/11.2.0/grid as grid 2013-01-17 11:48:18: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch napply /usr/local/patches/14727310 -local -silent -ocmrf ocm.rsp -oh /opt/app/11.2.0/grid 2013-01-17 11:48:18: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch napply /usr/local/patches/14727310 -local -silent -ocmrf ocm.rsp -oh /opt/app/11.2.0/grid ' 2013-01-17 11:49:39: Removing file /tmp/file5fgSjJ 2013-01-17 11:49:39: Successfully removed file: /tmp/file5fgSjJ 2013-01-17 11:49:39: /bin/su successfully executed 2013-01-17 11:49:39: status of apply patch is 0 2013-01-17 11:49:39: The apply patch output is Oracle Interim Patch Installer version 11.2.0.3.0 Copyright (c) 2012, Oracle Corporation. All rights reserved. Oracle Home : /opt/app/11.2.0/grid Central Inventory : /opt/app/oraInventory from : /opt/app/11.2.0/grid/oraInst.loc OPatch version : 11.2.0.3.0 OUI version : 11.2.0.3.0 Log file location : /opt/app/11.2.0/grid/cfgtoollogs/opatch/opatch2013-01-17_11-48-19AM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 14727310 Do you want to proceed? [y|n] Y (auto-answered by -silent) User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/opt/app/11.2.0/grid') Is the local system ready for patching? [y|n] Y (auto-answered by -silent) User Responded with: Y Backing up files... Applying sub-patch '14727310' to OH '/opt/app/11.2.0/grid' ApplySession: Optional component(s) [ oracle.sysman.console.db, 11.2.0.3.0 ] , [ oracle.sysman.oms.core, 10.2.0.4.4 ] not present in the Oracle Home or a higher version is found. Patching component oracle.rdbms, 11.2.0.3.0... Patching component oracle.rdbms.dbscripts, 11.2.0.3.0... Patching component oracle.rdbms.deconfig, 11.2.0.3.0... Patching component oracle.rdbms.rsf, 11.2.0.3.0... Patching component oracle.sdo.locator, 11.2.0.3.0... Verifying the update... find: `./crf/admin/run/crfmond': Permission denied find: `./crf/admin/run/crflogd': Permission denied Composite patch 14727310 successfully applied. Log file location: /opt/app/11.2.0/grid/cfgtoollogs/opatch/opatch2013-01-17_11-48-19AM_1.log OPatch succeeded. 2013-01-17 11:49:39: patch /usr/local/patches/14727310 apply successful for home /opt/app/11.2.0/grid 2013-01-17 11:49:39: Performing Post patch actions 2013-01-17 11:49:39: norestart flag is set to 2013-01-17 11:49:39: Opening file /etc/oracle/ocr.loc 2013-01-17 11:49:39: Value (FALSE) is set for key=local_only 2013-01-17 11:49:39: Performing Post patch actions for Grid Home /opt/app/11.2.0/grid 2013-01-17 11:49:39: Executing cmd: /opt/app/11.2.0/grid/rdbms/install/rootadd_rdbms.sh 2013-01-17 11:49:39: setrdbmsfileperms succeeded 2013-01-17 11:49:39: Patching Oracle Clusterware 2013-01-17 11:49:39: norestart flag is set to 2013-01-17 11:49:39: Executing cmd: /bin/rpm -q sles-release 2013-01-17 11:49:39: Command output: > package sles-release is not installed >End Command output 2013-01-17 11:49:39: init file = /opt/app/11.2.0/grid/crs/init/init.ohasd 2013-01-17 11:49:39: Copying file /opt/app/11.2.0/grid/crs/init/init.ohasd to /etc/init.d directory 2013-01-17 11:49:39: Setting init.ohasd permission in /etc/init.d directory 2013-01-17 11:49:39: init file = /opt/app/11.2.0/grid/crs/init/ohasd 2013-01-17 11:49:39: Copying file /opt/app/11.2.0/grid/crs/init/ohasd to /etc/init.d directory 2013-01-17 11:49:39: Setting ohasd permission in /etc/init.d directory 2013-01-17 11:49:39: Executing cmd: /bin/rpm -q sles-release 2013-01-17 11:49:39: Command output: > package sles-release is not installed >End Command output 2013-01-17 11:49:39: Removing "/etc/rc.d/rc3.d/S96ohasd" 2013-01-17 11:49:39: Removing file /etc/rc.d/rc3.d/S96ohasd 2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc3.d/S96ohasd 2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc3.d/S96ohasd" pointing to /etc/init.d/ohasd 2013-01-17 11:49:39: Removing "/etc/rc.d/rc5.d/S96ohasd" 2013-01-17 11:49:39: Removing file /etc/rc.d/rc5.d/S96ohasd 2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc5.d/S96ohasd 2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc5.d/S96ohasd" pointing to /etc/init.d/ohasd 2013-01-17 11:49:39: Removing "/etc/rc.d/rc0.d/K15ohasd" 2013-01-17 11:49:39: Removing file /etc/rc.d/rc0.d/K15ohasd 2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc0.d/K15ohasd 2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc0.d/K15ohasd" pointing to /etc/init.d/ohasd 2013-01-17 11:49:39: Removing "/etc/rc.d/rc1.d/K15ohasd" 2013-01-17 11:49:39: Removing file /etc/rc.d/rc1.d/K15ohasd 2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc1.d/K15ohasd 2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc1.d/K15ohasd" pointing to /etc/init.d/ohasd 2013-01-17 11:49:39: Removing "/etc/rc.d/rc2.d/K15ohasd" 2013-01-17 11:49:39: Removing file /etc/rc.d/rc2.d/K15ohasd 2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc2.d/K15ohasd 2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc2.d/K15ohasd" pointing to /etc/init.d/ohasd 2013-01-17 11:49:39: Removing "/etc/rc.d/rc3.d/K15ohasd" 2013-01-17 11:49:39: Removing file /etc/rc.d/rc3.d/K15ohasd 2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc3.d/K15ohasd 2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc3.d/K15ohasd" pointing to /etc/init.d/ohasd 2013-01-17 11:49:39: Removing "/etc/rc.d/rc4.d/K15ohasd" 2013-01-17 11:49:39: Removing file /etc/rc.d/rc4.d/K15ohasd 2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc4.d/K15ohasd 2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc4.d/K15ohasd" pointing to /etc/init.d/ohasd 2013-01-17 11:49:39: Removing "/etc/rc.d/rc6.d/K15ohasd" 2013-01-17 11:49:39: Removing file /etc/rc.d/rc6.d/K15ohasd 2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc6.d/K15ohasd 2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc6.d/K15ohasd" pointing to /etc/init.d/ohasd 2013-01-17 11:49:39: The file ohasd has been successfully linked to the RC directories 2013-01-17 11:49:39: Executing /opt/app/11.2.0/grid/bin/acfsroot install 2013-01-17 11:49:39: Executing cmd: /opt/app/11.2.0/grid/bin/acfsroot install 2013-01-17 11:51:21: Command output: > ACFS-9300: ADVM/ACFS distribution files found. > ACFS-9312: Existing ADVM/ACFS installation detected. > ACFS-9314: Removing previous ADVM/ACFS installation. > ACFS-9315: Previous ADVM/ACFS components successfully removed. > ACFS-9307: Installing requested ADVM/ACFS software. > ACFS-9308: Loading installed ADVM/ACFS drivers. > ACFS-9321: Creating udev for ADVM/ACFS. > ACFS-9323: Creating module dependencies - this may take some time. > ACFS-9154: Loading 'oracleoks.ko' driver. > ACFS-9154: Loading 'oracleadvm.ko' driver. > ACFS-9154: Loading 'oracleacfs.ko' driver. > ACFS-9327: Verifying ADVM/ACFS devices. > ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'. > ACFS-9156: Detecting control device '/dev/ofsctl'. > ACFS-9309: ADVM/ACFS installation correctness verified. >End Command output 2013-01-17 11:51:21: /opt/app/11.2.0/grid/bin/acfsroot install ... success 2013-01-17 11:51:21: USM driver install status is 1 2013-01-17 11:51:21: Validate crsctl command 2013-01-17 11:51:21: Validating /opt/app/11.2.0/grid/bin/crsctl 2013-01-17 11:51:21: Starting Oracle Clusterware 2013-01-17 11:51:21: Executing /opt/app/11.2.0/grid/bin/crsctl start crs 2013-01-17 11:51:21: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl start crs 2013-01-17 11:51:28: Command output: > CRS-4123: Oracle High Availability Services has been started. >End Command output 2013-01-17 11:51:28: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource 2013-01-17 11:51:30: Command output: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4000: Command Status failed, or completed with errors. >End Command output 2013-01-17 11:51:30: Waiting for Oracle CRSD and EVMD to start 2013-01-17 11:51:35: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource 2013-01-17 11:51:36: Command output: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4000: Command Status failed, or completed with errors. >End Command output 2013-01-17 11:51:36: Waiting for Oracle CRSD and EVMD to start 2013-01-17 11:51:41: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource 2013-01-17 11:51:43: Command output: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4000: Command Status failed, or completed with errors. >End Command output 2013-01-17 11:51:43: Waiting for Oracle CRSD and EVMD to start 2013-01-17 11:51:48: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource 2013-01-17 11:51:50: Command output: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4000: Command Status failed, or completed with errors. >End Command output 2013-01-17 11:51:50: Waiting for Oracle CRSD and EVMD to start 2013-01-17 11:51:55: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource 2013-01-17 11:51:57: Command output: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4000: Command Status failed, or completed with errors. >End Command output 2013-01-17 11:51:57: Waiting for Oracle CRSD and EVMD to start 2013-01-17 11:52:02: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource 2013-01-17 11:52:04: Command output: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4000: Command Status failed, or completed with errors. >End Command output 2013-01-17 11:52:04: Waiting for Oracle CRSD and EVMD to start 2013-01-17 11:52:09: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource 2013-01-17 11:52:10: Command output: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4000: Command Status failed, or completed with errors. >End Command output 2013-01-17 11:52:10: Waiting for Oracle CRSD and EVMD to start 2013-01-17 11:52:15: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource 2013-01-17 11:52:17: Command output: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4000: Command Status failed, or completed with errors. >End Command output 2013-01-17 11:52:17: Waiting for Oracle CRSD and EVMD to start 2013-01-17 11:52:22: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource 2013-01-17 11:52:24: Command output: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4000: Command Status failed, or completed with errors. >End Command output 2013-01-17 11:52:24: Waiting for Oracle CRSD and EVMD to start 2013-01-17 11:52:29: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource 2013-01-17 11:52:29: Command output: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4000: Command Status failed, or completed with errors. >End Command output 2013-01-17 11:52:29: Waiting for Oracle CRSD and EVMD to start 2013-01-17 11:52:34: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource 2013-01-17 11:52:34: Command output: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4000: Command Status failed, or completed with errors. >End Command output 2013-01-17 11:52:34: Waiting for Oracle CRSD and EVMD to start 2013-01-17 11:52:39: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource 2013-01-17 11:52:40: Command output: > NAME=ora.CLUSTER_DG.dg > TYPE=ora.diskgroup.type > TARGET=ONLINE , ONLINE > STATE=ONLINE on rhel6m1, ONLINE on rhel6m2 > > NAME=ora.DATA.dg > TYPE=ora.diskgroup.type > TARGET=ONLINE , ONLINE > STATE=ONLINE on rhel6m1, ONLINE on rhel6m2 > > NAME=ora.FLASH.dg > TYPE=ora.diskgroup.type > TARGET=ONLINE , ONLINE > STATE=ONLINE on rhel6m1, ONLINE on rhel6m2 > > NAME=ora.LISTENER.lsnr > TYPE=ora.listener.type > TARGET=ONLINE , ONLINE > STATE=OFFLINE, ONLINE on rhel6m2 > > NAME=ora.LISTENER_SCAN1.lsnr > TYPE=ora.scan_listener.type > TARGET=ONLINE > STATE=ONLINE on rhel6m2 > > NAME=ora.asm > TYPE=ora.asm.type > TARGET=ONLINE , ONLINE > STATE=ONLINE on rhel6m1, ONLINE on rhel6m2 > > NAME=ora.cvu > TYPE=ora.cvu.type > TARGET=ONLINE > STATE=ONLINE on rhel6m2 > > NAME=ora.gsd > TYPE=ora.gsd.type > TARGET=OFFLINE, OFFLINE > STATE=OFFLINE, OFFLINE > > NAME=ora.net1.network > TYPE=ora.network.type > TARGET=ONLINE , ONLINE > STATE=ONLINE on rhel6m1, ONLINE on rhel6m2 > > NAME=ora.oc4j > TYPE=ora.oc4j.type > TARGET=ONLINE > STATE=ONLINE on rhel6m2 > > NAME=ora.ons > TYPE=ora.ons.type > TARGET=ONLINE , ONLINE > STATE=ONLINE on rhel6m1, ONLINE on rhel6m2 > > NAME=ora.registry.acfs > TYPE=ora.registry.acfs.type > TARGET=ONLINE , ONLINE > STATE=ONLINE on rhel6m1, ONLINE on rhel6m2 > > NAME=ora.rhel6m1.vip > TYPE=ora.cluster_vip_net1.type > TARGET=ONLINE > STATE=ONLINE on rhel6m1 > > NAME=ora.rhel6m2.vip > TYPE=ora.cluster_vip_net1.type > TARGET=ONLINE > STATE=ONLINE on rhel6m2 > > NAME=ora.scan1.vip > TYPE=ora.scan_vip.type > TARGET=ONLINE > STATE=ONLINE on rhel6m2 > > NAME=ora.std11g2.db > TYPE=ora.database.type > TARGET=OFFLINE, ONLINE > STATE=OFFLINE, ONLINE on rhel6m2 > > NAME=ora.std11g2.myservice.svc > TYPE=ora.service.type > TARGET=OFFLINE, ONLINE > STATE=OFFLINE, ONLINE on rhel6m2 > >End Command output 2013-01-17 11:52:40: Running /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1 2013-01-17 11:52:40: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1 2013-01-17 11:52:40: Command output: > ************************************************************** > rhel6m1: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4529: Cluster Synchronization Services is online > CRS-4533: Event Manager is online > ************************************************************** >End Command output 2013-01-17 11:52:40: Checking the status of cluster 2013-01-17 11:52:45: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1 2013-01-17 11:52:45: Command output: > ************************************************************** > rhel6m1: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4529: Cluster Synchronization Services is online > CRS-4533: Event Manager is online > ************************************************************** >End Command output 2013-01-17 11:52:45: Checking the status of cluster 2013-01-17 11:52:50: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1 2013-01-17 11:52:50: Command output: > ************************************************************** > rhel6m1: > CRS-4535: Cannot communicate with Cluster Ready Services > CRS-4529: Cluster Synchronization Services is online > CRS-4533: Event Manager is online > ************************************************************** >End Command output 2013-01-17 11:52:50: Checking the status of cluster 2013-01-17 11:52:55: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1 2013-01-17 11:52:55: Command output: > ************************************************************** > rhel6m1: > CRS-4537: Cluster Ready Services is online > CRS-4529: Cluster Synchronization Services is online > CRS-4533: Event Manager is online > ************************************************************** >End Command output 2013-01-17 11:52:55: Oracle CRS stack installed and running 2013-01-17 11:52:55: Performing Post patch actions 2013-01-17 11:52:55: norestart flag is set to 2013-01-17 11:52:55: Opening file /etc/oracle/ocr.loc 2013-01-17 11:52:55: Value (FALSE) is set for key=local_only 2013-01-17 11:52:55: Performing Post Patch start action for DB Home /opt/app/oracle/product/11.2.0/dbhome_1 2013-01-17 11:52:55: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource -c rhel6m1 2013-01-17 11:52:56: Command output: > NAME=ora.CLUSTER_DG.dg > TYPE=ora.diskgroup.type > TARGET=ONLINE > STATE=ONLINE > > NAME=ora.DATA.dg > TYPE=ora.diskgroup.type > TARGET=ONLINE > STATE=ONLINE > > NAME=ora.FLASH.dg > TYPE=ora.diskgroup.type > TARGET=ONLINE > STATE=ONLINE > > NAME=ora.LISTENER.lsnr > TYPE=ora.listener.type > TARGET=ONLINE > STATE=ONLINE > > NAME=ora.asm > TYPE=ora.asm.type > TARGET=ONLINE > STATE=ONLINE > > NAME=ora.gsd > TYPE=ora.gsd.type > TARGET=OFFLINE > STATE=OFFLINE > > NAME=ora.net1.network > TYPE=ora.network.type > TARGET=ONLINE > STATE=ONLINE > > NAME=ora.ons > TYPE=ora.ons.type > TARGET=ONLINE > STATE=ONLINE > > NAME=ora.registry.acfs > TYPE=ora.registry.acfs.type > TARGET=ONLINE > STATE=ONLINE > > NAME=ora.rhel6m1.vip > TYPE=ora.cluster_vip_net1.type > CARDINALITY_ID=1 > TARGET=ONLINE > STATE=ONLINE > >End Command output 2013-01-17 11:52:56: Server assignments completed. Ready to start databases 2013-01-17 11:52:56: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/bin/srvctl start home -o /opt/app/oracle/product/11.2.0/dbhome_1 -s /opt/app/oracle/product/11.2.0/dbhome_1/srvm/admin/stophome.txt -n rhel6m1 2013-01-17 11:52:56: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/bin/srvctl start home -o /opt/app/oracle/product/11.2.0/dbhome_1 -s /opt/app/oracle/product/11.2.0/dbhome_1/srvm/admin/stophome.txt -n rhel6m1 ' 2013-01-17 11:53:16: Removing file /tmp/filePCaf67 2013-01-17 11:53:16: Successfully removed file: /tmp/filePCaf67 2013-01-17 11:53:16: /bin/su successfully executed 2013-01-17 11:53:16: /opt/app/oracle/product/11.2.0/dbhome_1/bin/srvctl start home -o /opt/app/oracle/product/11.2.0/dbhome_1 -s /opt/app/oracle/product/11.2.0/dbhome_1/srvm/admin/stophome.txt -n rhel6m1 output is 2013-01-17 11:53:16: Started resources from datbase home /opt/app/oracle/product/11.2.0/dbhome_1