Showing posts with label PSU. Show all posts
Showing posts with label PSU. Show all posts

Monday, August 10, 2015

Applying of GI PSU 11.2.0.4.5 Fails Due to -customLogDir Opton Used by OPatch 11.2.0.3.10

Use of opatch auto option has become the default for applying PSU for both GI and Oracle home in an automated fashion. Applying 11.2.0.4.5 GI PSU (July 2015) for both GI and Oracle Home in a single instance setup failed with following message.
$ /opt/app/oracle/product/11.2.0/grid_4/OPatch/opatch version
OPatch Version: 11.2.0.3.10

# /opt/app/oracle/product/11.2.0/grid_4/OPatch/opatch auto ./20996923 -ocmrf ../ocm.rsp
Executing /opt/app/oracle/product/11.2.0/grid_4/perl/bin/perl /opt/app/oracle/product/11.2.0/grid_4/OPatch/crs/patch11203.pl -patchdir . -patchn 20996923 -ocmrf ../ocm.rsp -paramfile /opt/app/oracle/product/11.2.0/grid_4/crs/install/crsconfig_params
This is the main log file: /opt/app/oracle/product/11.2.0/grid_4/cfgtoollogs/opatchauto2015-08-10_14-06-35.log
This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/opt/app/oracle/product/11.2.0/grid_4/cfgtoollogs/opatchauto2015-08-10_14-06-35.report.log
2015-08-10 14:06:35: Starting Oracle Restart Patch Setup
Using configuration parameter file: /opt/app/oracle/product/11.2.0/grid_4/crs/install/crsconfig_params
The opatch Component check failed. This patch is not applicable for /opt/app/oracle/product/11.2.0/dbhome_4
The opatch Component check failed. This patch is not applicable for /opt/app/oracle/product/11.2.0/dbhome_4
Conflict check failed for oracle home  /opt/app/oracle/product/11.2.0/dbhome_4
Conflict check failed
ERROR: Conflict-Check has failed . Please refer to /opt/app/oracle/product/11.2.0/grid_4/cfgtoollogs/opatchauto2015-08-10_14-06-35.log for details
Though the error message says conflict check failed, it is not the real reason in this case. Examining the log file revealed following
2015-08-10 14:07:13: Status of opatch version check  for /opt/app/oracle/product/11.2.0/grid_4 is 1
2015-08-10 14:07:13: Opatch version check passed for oracle home  /opt/app/oracle/product/11.2.0/grid_4
2015-08-10 14:07:13: Opatch version check passed  for all oracle homes
2015-08-10 14:07:13: Processing oracle home /opt/app/oracle/product/11.2.0/dbhome_4
2015-08-10 14:07:13: Opening file /etc/oracle/ocr.loc
2015-08-10 14:07:13: Value (TRUE) is set for key=local_only
2015-08-10 14:07:13: Home type of /opt/app/oracle/product/11.2.0/dbhome_4 is DB
2015-08-10 14:07:13: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_4 is oracle
2015-08-10 14:07:13: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_4/OPatch/opatch prereq CheckComponents -ph ./20996923/20760982 -oh /opt/app/oracle/product/11.2.0/dbhome_4 
-customLogDir /opt/app/oracle/product/11.2.0/grid_4/cfgtoollogs/opatchauto/core
2015-08-10 14:07:13: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_4/OPatch/opatch prereq CheckComponents -ph ./20996923/20760982 -oh /opt/app/oracle/product/11.2.0/dbhome_4 
-customLogDir /opt/app/oracle/product/11.2.0/grid_4/cfgtoollogs/opatchauto/core '
2015-08-10 14:07:14: Removing file /tmp/fileWJrEf9
2015-08-10 14:07:14: Successfully removed file: /tmp/fileWJrEf9
2015-08-10 14:07:14: /bin/su exited with rc=1

2015-08-10 14:07:14: The opatch Component check failed. This patch is not applicable for /opt/app/oracle/product/11.2.0/dbhome_4
2015-08-10 14:07:14: The component check failed with following error
2015-08-10 14:07:14: Input custom log directory "/opt/app/oracle/product/11.2.0/grid_4/cfgtoollogs/opatchauto/core" does not exists or not a valid directory.

 OPatch failed with error code 1
From the output it could be seen that when opatch is checking components for oracle home patching it is setting a custom log directory which is inside the grid home. This wouldn't be a problem if both GI home and Oracle home are owned by the same user. But in this case role separation is used and the directories cfgtoollogs,opatchauto,core all have permission set to either 700 or 755 and ownership grid:oinstall. This makes oracle user unable to write to these directories and the failure of the patch apply.
Metalink notes 2022797.1 and 1964569.1 (These MOS notes are related to 12c) suggest to use -customLogDir option to specify a directory but this is not applicable for "opatch auto" option.



Two ways to overcome this problem. One is to grant the oinstall group the write permission to cfgtoollogs/opatchauto/core before the patch apply and revoke once the the patch is successfully applied.
Second option is to use a lower version OPatch, in this case using OPatch version 11.2.0.3.6 (minimum version required according to PSU's readme.html) enabled to successfully apply the patch.
As patch apply went ahead without a problem using a lower version of opatch mean that customLogDir option was introduced in higher version OPatch (11.2.0.3.10) without the consideration to possible failures that could arise due to use of role separation.

Useful Metalink Notes
opatchauto wasn't able to run ... because 'bash' command could not be found [ID 2022797.1]
"opatch lsinventory" on CRS/GI home fails with error code 15 [ID 1964569.1]
Posted by at
Labels: ,

Thursday, October 17, 2013

Patching 12c (12.1.0.1) RAC with October 2013 PSU

First critical patch update for 12c was released Oct 15 2013. This post looks at the difference in patching 12c RAC environment (with role separation) compared to 11.2 environment. The environment used for patching is the environment that was upgraded from 11.2 to 12c.
First thing to notice is the name of the patch. On the readme.html that is included in the patch it is referred to as "Oracle Grid Infrastructure System Patch" instead of "Oracle Grid Infrastructure Patch Set Update" (More jargon to converse with!). However in the PSU and CPU availability document (1571391.1) it is still referred to as PSU (GI 12.1.0.1.1 PSU Patch 17272829). "GI System Patch" is used throughout the readme.html document so it's pretty safe to assume that's how the 12c patches going to be referred from now on.
Opatch auto option has been merged into one single command called "opatchauto".
However it is still possible to apply the patch manually. But at the time of this post (16/10/2013) the document with instruction for manual patch apply/rollback (1591616.1) is not available on MOS though the readme.html mentions it (shouldn't this be available before patches are released?). When this become available follow it for manual patch apply. In mean time as a workaround generateSteps option could be used to list the steps used by opatchauto
/opt/app/12.1.0/grid/OPatch/opatchauto apply  /usr/local/patch/17272829  -ocmrf ocm.rsp  -generateSteps
OPatch 12.1.0.1.2 or later is needed to apply this patch. Installing new OPatch on GI_HOME causes the following
unzip p6880880_121010_Linux-x86-64.zip
  ..
  inflating: OPatch/operr
error:  cannot create PatchSearch.xml
        Permission denied
File PatchSearch.xml is to be copied (or unzipped) to GI_HOME outside the OPatch directory and since GI_HOME has restrictive permission unzipping as grid user causes the above error. The file could be copied manually as root user into GI_HOME or ignore the error (this caused no issue when installing the patch). Looking inside the PatchSearch.xml file it seem this might be used to get the OPatch from MOS (has urls of MOS and OPatch including CSI number). No such issue installing the new OPatch on ORACLE_HOME.
Next issue is related to patch location. Readme.html mentions to use "PATH_TO_PATCH_DIRECTORY" in the opatchauto command. PATH_TO_PATCH_DIRECTORY is the location where the patch was unzipped. This is same as the 11.2. However this location is not recognized by the opatchauto command and complains of the missing bundle.xml file.
[grid@rhel6m2 patches]$ pwd
/usr/local/patches  <<-- this becomes the PATH_TO_PATCH_DIRECTORY (same as 11.2 as shown here)
[grid@rhel6m2 patches]$ ls
p17027533_121010_Linux-x86-64.zip
[grid@rhel6m2 patches]$ unzip p17027533_121010_Linux-x86-64.zip
[grid@rhel6m2 patches]$ su <-- preparing to run opatchauto as root user
[root@rhel6m2 patches]# /opt/app/12.1.0/grid/OPatch/opatchauto apply /usr/local/patches -ocmrf ocm.rsp

Parameter Validation: Successful

Patch Collection failed: Invalid patch location "/usr/local/patches" as there is no bundle.xml file in it or its parent directory.

opatchauto failed with error code 2.
So using the location where the patch was unzipped doesn't work unlike 11.2. Give the full path to the patch directory
[root@rhel6m2 patches]# /opt/app/12.1.0/grid/OPatch/opatchauto apply /usr/local/patches/17272829 -ocmrf ocm.rsp

OPatchauto version : 12.1.0.1.2
OUI version        : 12.1.0.1.0
Running from       : /opt/app/12.1.0/grid

opatchauto log file: /opt/app/12.1.0/grid/cfgtoollogs/opatchauto/17272829/opatch_gi_2013-10-16_15-39-34_deploy.log

Parameter Validation: Successful
...
Apply of patch progress.
Also worth noting is that along with opatchauto keyword apply must be given without it a syntax error occurs
[root@rhel6m1 patches]# /opt/app/12.1.0/grid/OPatch/opatchauto /usr/local/patches/17272829 -ocmrf ocm.rsp
OPatch Automation Tool
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

Syntax Error... Unrecognized Command or Option (/usr/local/patches/17272829): 1st argument must be one of the following:
   apply
   rollback
   version
   ..
Section 2.3 on the readme.html does mention apply keyword in the commands but in 2.4 Patch installation section the apply key word missing. This is another difference compared to 11.2 where there was no apply key word when opatch auto option was used. Rollback commands on section 2.7 are also incorrectly listed. Correct rollback commands are listed on section 2.3.
The readme.html for GI system patch doesn't list any post installation task such as loading modified SQLs. This is automatically run as part of the patch apply. Once the patch is applied on the last node of the RAC the registry history is updated
SQL> select * from dba_registry_history;

ACTION_TIME                    ACTION     NAMESPACE  VERSION            ID BUNDLE_SER COMMENTS
------------------------------ ---------- ---------- ---------- ---------- ---------- ------------------------------
12-AUG-13 04.28.26.378432 PM   UPGRADE    SERVER     12.1.0.1.0                       Upgraded from 11.2.0.3.0
12-AUG-13 04.34.09.496894 PM   APPLY      SERVER     12.1.0.1            0 PSU        Patchset 12.1.0.0.0
16-OCT-13 04.05.54.514261 PM   APPLY      SERVER     12.1.0.1            1 PSU        PSU 12.1.0.1.1
SQL apply is logged in dba_registry_sqlpatch table
SQL> show con_name

CON_NAME
------------------------------
CDB$ROOT

SQL> select * from dba_registry_sqlpatch;

  PATCH_ID ACTION     STATUS          ACTION_TIME                    DESCRIPTIO LOGFILE
---------- ---------- --------------- ------------------------------ ---------- --------------------------------------------------------------------------------
  17027533 APPLY      SUCCESS         16-OCT-13 05.54.42.295071 PM   sqlpatch   /opt/app/oracle/product/12.1.0/dbhome_1/sqlpatch/17027533/17027533_apply_CDB12C_
                                                                                CDBROOT_2013Oct16_17_51_30.log
Each PDB will also have its own log file entry in the dba_registry_sqlpatch view
SQL> alter session set container=pdb12c;

Session altered.

SQL> show con_name

CON_NAME
------------------------------
PDB12C

SQL> select * from dba_registry_sqlpatch;

  PATCH_ID ACTION     STATUS          ACTION_TIME                    DESCRIPTIO LOGFILE
---------- ---------- --------------- ------------------------------ ---------- --------------------------------------------------------------------------------
  17027533 APPLY      END             16-OCT-13 05.54.44.488402 PM   sqlpatch   /opt/app/oracle/product/12.1.0/dbhome_1/sqlpatch/17027533/17027533_apply_CDB12C_
                                                                                PDB12C_2013Oct16_17_51_49.log
Even the pdb$seed database could be queried this way to confirm that it is also updated with the SQL changes made by the patch. Any new PDB created using the seed PDB also gets these modification and no patch post installation work is necessary.




Full output of running the opaatchauto is given below
[root@rhel6m1 17272829]# /opt/app/12.1.0/grid/OPatch/opatchauto apply `pwd` -ocmrf ../ocm.rsp
OPatch Automation Tool
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

OPatchauto version : 12.1.0.1.2
OUI version        : 12.1.0.1.0
Running from       : /opt/app/12.1.0/grid

opatchauto log file: /opt/app/12.1.0/grid/cfgtoollogs/opatchauto/17272829/opatch_gi_2013-10-16_14-23-50_deploy.log

Parameter Validation: Successful


Grid Infrastructure home:
/opt/app/12.1.0/grid
RAC home(s):
/opt/app/oracle/product/12.1.0/dbhome_1

Configuration Validation: Successful

Patch Location: /usr/local/patches/17272829
Grid Infrastructure Patch(es): 17027533 17077442 17303297
RAC Patch(es): 17027533 17077442

Patch Validation: Successful

Stopping RAC (/opt/app/oracle/product/12.1.0/dbhome_1) ... Successful
Following database(s) were stopped and will be restarted later during the session: std11g2

Applying patch(es) to "/opt/app/oracle/product/12.1.0/dbhome_1" ...
Patch "/usr/local/patches/17272829/17027533" successfully applied to "/opt/app/oracle/product/12.1.0/dbhome_1".
Patch "/usr/local/patches/17272829/17077442" successfully applied to "/opt/app/oracle/product/12.1.0/dbhome_1".

Stopping CRS ... Successful

Applying patch(es) to "/opt/app/12.1.0/grid" ...
Patch "/usr/local/patches/17272829/17027533" successfully applied to "/opt/app/12.1.0/grid".
Patch "/usr/local/patches/17272829/17077442" successfully applied to "/opt/app/12.1.0/grid".
Patch "/usr/local/patches/17272829/17303297" successfully applied to "/opt/app/12.1.0/grid".

Starting CRS ... Successful

Starting RAC (/opt/app/oracle/product/12.1.0/dbhome_1) ... Successful

SQL changes, if any, are applied successfully on the following database(s): std11g2

Apply Summary:
Following patch(es) are successfully installed:
GI Home: /opt/app/12.1.0/grid: 17027533, 17077442, 17303297
RAC Home: /opt/app/oracle/product/12.1.0/dbhome_1: 17027533, 17077442

On a system with PDBs that have dynamic services created for them, stopping RAC step will have the following output listing the service
Stopping RAC (/opt/app/oracle/product/12.1.0/dbhome_1) ... Successful
Following database(s) were stopped and will be restarted later during the session: -pdbsvc,cdb12c
If there are no services created for the PDBs then only the CDB is mentioned in the output
Stopping RAC (/opt/app/oracle/product/12.1.0/dbhome_1) ... Successful
Following database(s) were stopped and will be restarted later during the session: cdb12c

Apply has the option of analyze which says 
-analyze
              This option runs all the required prerequisite checks to confirm
              the patchability of the system without actually patching or
              affecting the system in any way.
Even though it says "runs all the required prerequisite checks to confirm the patchability" this seem not be the case. Analyze could suceed and actual patch apply could fail. 
[root@rhel12c2 patch]# /opt/app/12.1.0/grid/OPatch/opatchauto apply /usr/local/patch/17272829 -ocmrf ocm.rsp -analyze
OPatch Automation Tool
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

OPatchauto version : 12.1.0.1.2
OUI version        : 12.1.0.1.0
Running from       : /opt/app/12.1.0/grid

opatchauto log file: /opt/app/12.1.0/grid/cfgtoollogs/opatchauto/17272829/opatch_gi_2013-10-17_11-28-37_analyze.log

NOTE: opatchauto is running in ANALYZE mode. There will be no change to your system.

Parameter Validation: Successful

Grid Infrastructure home:
/opt/app/12.1.0/grid
RAC home(s):
/opt/app/oracle/product/12.1.0/dbhome_1

Configuration Validation: Successful

Patch Location: /usr/local/patch/17272829
Grid Infrastructure Patch(es): 17027533 17077442 17303297
RAC Patch(es): 17027533 17077442

Patch Validation: Successful

Analyzing patch(es) on "/opt/app/oracle/product/12.1.0/dbhome_1" ...
Patch "/usr/local/patch/17272829/17027533" successfully analyzed on "/opt/app/oracle/product/12.1.0/dbhome_1" for apply.
Patch "/usr/local/patch/17272829/17077442" successfully analyzed on "/opt/app/oracle/product/12.1.0/dbhome_1" for apply.

Analyzing patch(es) on "/opt/app/12.1.0/grid" ...
Patch "/usr/local/patch/17272829/17027533" successfully analyzed on "/opt/app/12.1.0/grid" for apply.
Patch "/usr/local/patch/17272829/17077442" successfully analyzed on "/opt/app/12.1.0/grid" for apply.
Patch "/usr/local/patch/17272829/17303297" successfully analyzed on "/opt/app/12.1.0/grid" for apply.

SQL changes, if any, are analyzed successfully on the following database(s): cdb12c

Apply Summary:
Following patch(es) are successfully analyzed:
GI Home: /opt/app/12.1.0/grid: 17027533, 17077442, 17303297
RAC Home: /opt/app/oracle/product/12.1.0/dbhome_1: 17027533, 17077442

opatchauto succeeded.

<<------ Running of actual patch command ----------->>
[root@rhel12c2 patch]# /opt/app/12.1.0/grid/OPatch/opatchauto apply /usr/local/patch/17272829 -ocmrf ocm.rsp
OPatch Automation Tool
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

OPatchauto version : 12.1.0.1.2
OUI version        : 12.1.0.1.0
Running from       : /opt/app/12.1.0/grid

opatchauto log file: /opt/app/12.1.0/grid/cfgtoollogs/opatchauto/17272829/opatch_gi_2013-10-17_11-32-12_deploy.log

Parameter Validation: Successful

Grid Infrastructure home:
/opt/app/12.1.0/grid
RAC home(s):
/opt/app/oracle/product/12.1.0/dbhome_1

Configuration Validation: Successful

Patch Location: /usr/local/patch/17272829
Grid Infrastructure Patch(es): 17027533 17077442 17303297
RAC Patch(es): 17027533 17077442

Patch Validation: Successful

Stopping RAC (/opt/app/oracle/product/12.1.0/dbhome_1) ... Successful
Following database(s) were stopped and will be restarted later during the session: -pdbsvc,cdb12c

Applying patch(es) to "/opt/app/oracle/product/12.1.0/dbhome_1" ...
Patch "/usr/local/patch/17272829/17027533" successfully applied to "/opt/app/oracle/product/12.1.0/dbhome_1".
Patch "/usr/local/patch/17272829/17077442" successfully applied to "/opt/app/oracle/product/12.1.0/dbhome_1".

Stopping CRS ... Successful

Applying patch(es) to "/opt/app/12.1.0/grid" ...
Command "/opt/app/12.1.0/grid/OPatch/opatch napply -phBaseFile /tmp/OraGI12Home1_patchList -local  -invPtrLoc /opt/app/12.1.0/grid/oraInst.loc -oh /opt/app/12.1.0/grid -silent -ocmrf /usr/local/patch/ocm.rsp" execution failed:
UtilSession failed:
Prerequisite check "CheckSystemSpace" failed.

Log file Location for the failed command: /opt/app/12.1.0/grid/cfgtoollogs/opatch/opatch2013-10-17_11-39-04AM_1.log

[WARNING] The local database instance 'cdb12c2' from '/opt/app/oracle/product/12.1.0/dbhome_1' is not running. SQL changes, if any,  will not be applied. Please refer to the log file for more details.
For more details, please refer to the log file "/opt/app/12.1.0/grid/cfgtoollogs/opatchauto/17272829/opatch_gi_2013-10-17_11-32-12_deploy.debug.log".

Apply Summary:
Following patch(es) are successfully installed:
RAC Home: /opt/app/oracle/product/12.1.0/dbhome_1: 17027533, 17077442

Following patch(es) failed to be installed:
GI Home: /opt/app/12.1.0/grid: 17027533, 17077442, 17303297

opatchauto failed with error code 2.
Log files list the failed steps and has commands that could be manually executed. 
-------------------Following steps still need to be executed-------------------

/opt/app/12.1.0/grid/OPatch/opatch napply -phBaseFile /tmp/OraGI12Home1_patchList -local  -invPtrLoc /opt/app/12.1.0/grid/oraInst.loc -oh /opt/app/12.1.0/grid -silent -ocmrf /usr/local/patch/ocm.rsp (TRIED BUT FAILED)

/opt/app/12.1.0/grid/rdbms/install/rootadd_rdbms.sh

/usr/bin/perl /opt/app/12.1.0/grid/crs/install/rootcrs.pl -postpatch
Executing the first command shows how much free disk space must be available before the patch apply
[grid@rhel12c2 patch]$ /opt/app/12.1.0/grid/OPatch/opatch napply -phBaseFile /tmp/OraGI12Home1_patchList -local  -invPtrLoc /opt/app/12.1.0/grid/oraInst.loc -oh /opt/app/12.1.0/grid -silent -ocmrf /usr/local/patch/ocm.rsp
Oracle Interim Patch Installer version 12.1.0.1.2
Copyright (c) 2013, Oracle Corporation.  All rights reserved.


Oracle Home       : /opt/app/12.1.0/grid
Central Inventory : /opt/app/oraInventory
   from           : /opt/app/12.1.0/grid/oraInst.loc
OPatch version    : 12.1.0.1.2
OUI version       : 12.1.0.1.0
Log file location : /opt/app/12.1.0/grid/cfgtoollogs/opatch/opatch2013-10-17_11-42-52AM_1.log

Verifying environment and performing prerequisite checks...
Prerequisite check "CheckSystemSpace" failed.
The details are:
Required amount of space(10578.277MB) is not available.
UtilSession failed:
Prerequisite check "CheckSystemSpace" failed.
Log file location: /opt/app/12.1.0/grid/cfgtoollogs/opatch/opatch2013-10-17_11-42-52AM_1.log

OPatch failed with error code 73

Unlike the RAC environment single instance database requires running the "loading modified SQLs" manually. 12c provides the datapatch tool for this purpose unlike in 11.2 where catbundle script was run for the same purpose. All databases (CDB and PDB) are updated.
[oracle@rhel6m1 OPatch]$ ./datapatch -verbose
SQL Patching tool version 12.1.0.1.0 on Mon Oct 21 16:58:15 2013
Copyright (c) 2013, Oracle.  All rights reserved.

Connecting to database...OK
Determining current state...
Currently installed SQL Patches:
  PDB CDB$ROOT:
  PDB PDB$SEED:
  PDB PDB12C:
  PDB PDB12CDI:
Currently installed C Patches: 17027533
For the following PDBs: CDB$ROOT
  Nothing to roll back
  The following patches will be applied: 17027533
For the following PDBs: PDB$SEED
  Nothing to roll back
  The following patches will be applied: 17027533
For the following PDBs: PDB12C
  Nothing to roll back
  The following patches will be applied: 17027533
For the following PDBs: PDB12CDI
  Nothing to roll back
  The following patches will be applied: 17027533
Adding patches to installation queue...
Installing patches...
Validating logfiles...
Patch 17027533 apply (pdb CDB$ROOT): SUCCESS
  logfile: /opt/app/oracle/product/12.1.0/dbhome_1/sqlpatch/17027533/17027533_apply_ENT12C_CDBROOT_2013Oct21_16_58_30.log (no errors)
Patch 17027533 apply (pdb PDB$SEED): SUCCESS
  logfile: /opt/app/oracle/product/12.1.0/dbhome_1/sqlpatch/17027533/17027533_apply_ENT12C_PDBSEED_2013Oct21_16_59_06.log (no errors)
Patch 17027533 apply (pdb PDB12C): SUCCESS
  logfile: /opt/app/oracle/product/12.1.0/dbhome_1/sqlpatch/17027533/17027533_apply_ENT12C_PDB12C_2013Oct21_16_59_32.log (no errors)
Patch 17027533 apply (pdb PDB12CDI): SUCCESS
  logfile: /opt/app/oracle/product/12.1.0/dbhome_1/sqlpatch/17027533/17027533_apply_ENT12C_PDB12CDI_2013Oct21_16_59_55.log (no errors)
SQL Patching tool complete on Mon Oct 21 17:00:30 2013
Each container could be queried to check the status of the apply.
SQL> show con_name

CON_NAME
------------------------------
CDB$ROOT
SQL> select * from dba_registry_sqlpatch;

  PATCH_ID ACTION          STATUS          ACTION_TIME                  DESCRIPTIO LOGFILE
---------- --------------- --------------- ---------------------------- ---------- ----------------------------------------------------------------------
  17027533 APPLY           SUCCESS         21-OCT-13 05.00.27.856979 PM sqlpatch   /opt/app/oracle/product/12.1.0/dbhome_1/sqlpatch/17027533/17027533_app
                                                                                   ly_ENT12C_CDBROOT_2013Oct21_16_58_30.log

SQL> ALTER SESSION SET container = pdb$seed;
Session altered.

SQL> show con_name

CON_NAME
------------------------------
PDB$SEED
SQL>  select * from dba_registry_sqlpatch;

  PATCH_ID ACTION          STATUS          ACTION_TIME                  DESCRIPTIO LOGFILE
---------- --------------- --------------- ---------------------------- ---------- ----------------------------------------------------------------------
  17027533 APPLY           SUCCESS         21-OCT-13 05.00.29.488402 PM sqlpatch   /opt/app/oracle/product/12.1.0/dbhome_1/sqlpatch/17027533/17027533_app
                                                                                   ly_ENT12C_PDBSEED_2013Oct21_16_59_06.log                       

SQL> ALTER SESSION SET container = pdb12c;
Session altered.
                       
SQL> show con_name

CON_NAME
------------------------------
PDB12C
SQL> select * from dba_registry_sqlpatch;

  PATCH_ID ACTION          STATUS          ACTION_TIME                  DESCRIPTIO LOGFILE
---------- --------------- --------------- ---------------------------- ---------- ----------------------------------------------------------------------
  17027533 APPLY           SUCCESS         21-OCT-13 05.00.30.823562 PM sqlpatch   /opt/app/oracle/product/12.1.0/dbhome_1/sqlpatch/17027533/17027533_app
                                                                                   ly_ENT12C_PDB12C_2013Oct21_16_59_32.log
                      
SQL> ALTER SESSION SET container = pdb12cdi;
Session altered.

SQL> show con_name

CON_NAME
------------------------------
PDB12CDI
SQL> select * from dba_registry_sqlpatch;

  PATCH_ID ACTION          STATUS          ACTION_TIME                  DESCRIPTIO LOGFILE
---------- --------------- --------------- ---------------------------- ---------- ----------------------------------------------------------------------
  17027533 APPLY           SUCCESS         21-OCT-13 05.00.30.996406 PM sqlpatch   /opt/app/oracle/product/12.1.0/dbhome_1/sqlpatch/17027533/17027533_app
                                                                                   ly_ENT12C_PDB12CDI_2013Oct21_16_59_55.log

Useful metalink notes
Known Patching Issues for the Oct 15 PSU, Oracle Database 12c R1 using opatchauto and EM [ID 1592252.1]

Update 17 January 2014
More Useful metalink notes
Supplemental Readme - Patch Installation and Deinstallation For 12.1.0.1.x GI PSU [ID 1591616.1]
Example: Manually Apply a 12c GI PSU in Cluster Environment [ID 1594184.1]
Example: Manually Apply a 12c GI PSU in Standalone Environment [ID 1595408.1]
Example: Applying a 12c GI PSU With opatchauto in GI Cluster or Standalone Environment [ID 1594183.1]
What's the sub-patches in 12c GI PSU [ID 1595371.1]
Posted by at
Labels: , , , , , ,

Thursday, April 25, 2013

April 2013 PSU (11.2.0.3) Manual Steps for Apply/Rollback Patch

April PSU for 11.2 also follows the same pattern as the January PSU when it comes to manual steps for applying the patch. The document ID listed for manual steps (1494646.1) is same one listed for January PSU and it is not updated with the information for the April PSU. This leaves the question of identifying which is the "GI_Components_number" and which is the "DB_PSU_number". There's no information in the GI PSU read me file to help with this fact. However there are indirect ways of identifying the GI component number and the DB PSU number.
1. When the April GI PSU (16083653 - 11.2.0.6) is extracted it will have two patch directories (16056266 and 16315641). One will have the same ID as the (unbundled) April DB PSU (16056266 - 11.2.0.6). Once this is identified then the other one is the GI component number (in this case it is 16315641). For this approach before manually applying the GI PSU one must also refer the unbundled DB PSU to get the DB PSU number (even though this is not needed in GI environment as it's bundled with the GI PSU patch).



2. Another method is to useopatch query -get_base_bug to identify the GI PSU component. This command only works with the GI component part of the patch and will return an error with the DB portion of the patch. For example running against the DB portion of the patch
$ORACLE_HOME/OPatch/opatch query -get_base_bug /usr/local/patches/16056266/
Oracle Interim Patch Installer version 11.2.0.3.4
...
Failed to load the patch object.  Possible causes are:
  The specified path is not an interim Patch shiphome
  Meta-data files are missing from the patch area
  Patch location = /usr/local/patches/16056266/
  Details = Input metadata files are missing.

Patch Location "/usr/local/patches/16056266/" doesn't point to a valid patch area.

OPatch failed with error code 75
However running against the GI component part of the patch will give the GI component number.
 $ORACLE_HOME/OPatch/opatch query -get_base_bug /usr/local/patches/16315641
Oracle Interim Patch Installer version 11.2.0.3.4
...
--------------------------------------------------------------------------------

 List of bugs to be fixed:
   16315641: GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.3.6 (GI COMPONENTS)
   15876003: GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.3.5 (GI COMPONENTS)
   14275572: GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.3.4 (INCLUDES DB PSU 11.2.0.3.4)
   13919095: GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.3.3 (INCLUDES DB PSU 11.2.0.3.3)
   13696251: GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.3.2 (INCLUDES DB PSU 11.2.0.3.2)
   13348650: GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.3.1 (INCLUDES DB PSU 11.2.0.3.1)
   12659561: INSTANCE DOES NOT REGISTER SERVICES WHEN SCAN FAILOVERED
   14305980: IMPROVE CRSD RESOURCE AUTO START PROCESSING
   14277586: INCORRECT SHUTDOWN REASON CODE IN FAN EVENTS - BREAKING CLIENTS
   ...
   ...
Once the GI component is known the remaining directory's number is the DB PSU number.

Related Post
January 2013 PSU (11.2.0.3) Manual Steps for Apply/Rollback Patch vs OPatch Auto
Posted by at
Labels: , ,

Sunday, January 20, 2013

January 2013 PSU (11.2.0.3) Manual Steps for Apply/Rollback Patch vs OPatch Auto

It seems future of patch applying is to be "auto" if the January 2013 PSU for 11.2.0.3 is anything to go by. First is that manual steps has been taken out of the main readme.html that comes with the patch to a supplemental document (1494646.1). So more documents to go through than to have all the necessary information in one place.
Secondly from GI PSU 11.2.0.3.5 and up the GI PSU have a separate GI and DB portions, which means there's patch number GI component number and DB component number. These components must be substitued in the commands used for manually applying the patch and these commands now come in generic form such as 
<GI_HOME>/OPatch/opatch napply -oh <GI_HOME> -local <UNZIPPED_PATCH_LOCATION>/<GI_components_number>
<GI_HOME>/OPatch/opatch apply -oh <GI_HOME> -local <UNZIPPED_PATCH_LOCATION>/<DB_PSU_number>
Manually applying patch gives the DBA the control and also to closely examine what goes on.
"opatch auto" has it's merits as well. One command to patch both GI and DB Homes. It brings down the clusterware stack and start it up once finished. One draw back is that even if one component fails rest of the patching will continue and one has to come back to it at the end of the patching. Manual patching is more interactive allowing any errors/failures to be corrected before continuing with the rest of the patching.
Auto patching is run as root and switch (or execute as) between the appropriate user (grid or oracle) to the relevant home as it apply the patch. Opatch auto also need an ocm response file. How To Create An OCM Response File For Opatch Silent Installation [ID 966023.1]
Output on the shell where opatch auto is run only shows the outcome of each patchs' apply process (pass or fail).
[root@rhel6m1 patches]# /opt/app/11.2.0/grid/OPatch/opatch auto /usr/local/patches -ocmrf ocm.rsp
Executing /usr/bin/perl /opt/app/11.2.0/grid/OPatch/crs/patch112.pl -patchdir /usr/local -patchn patches -ocmrf ocm.rsp -paramfile /opt/app/11.2.0/grid/crs/install/crsconfig_params
opatch auto log file location is /opt/app/11.2.0/grid/OPatch/crs/../../cfgtoollogs/opatchauto2013-01-17_11-33-35.log
Detected Oracle Clusterware install
Using configuration parameter file: /opt/app/11.2.0/grid/crs/install/crsconfig_params
patch /usr/local/patches/15876003/custom/server/15876003  apply successful for home  /opt/app/oracle/product/11.2.0/dbhome_1
patch /usr/local/patches/14727310  apply successful for home  /opt/app/oracle/product/11.2.0/dbhome_1
Successfully unlock /opt/app/11.2.0/grid
patch /usr/local/patches/15876003  apply successful for home  /opt/app/11.2.0/grid
patch /usr/local/patches/14727310  apply successful for home  /opt/app/11.2.0/grid
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9312: Existing ADVM/ACFS installation detected.
ACFS-9314: Removing previous ADVM/ACFS installation.
ACFS-9315: Previous ADVM/ACFS components successfully removed.
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies - this may take some time.
ACFS-9154: Loading 'oracleoks.ko' driver.
ACFS-9154: Loading 'oracleadvm.ko' driver.
ACFS-9154: Loading 'oracleacfs.ko' driver.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
ACFS-9156: Detecting control device '/dev/ofsctl'.
ACFS-9309: ADVM/ACFS installation correctness verified.
CRS-4123: Oracle High Availability Services has been started.
After apply query the lsinventory for the PSU
$GI_HOME/OPatch/opatch lsinventory -local -bugs_fixed | grep "GRID INFRASTRUCTURE PATCH SET UPDATE"
$ORACLE_HOME/OPatch/opatch lsinventory -local -bugs_fixed | grep "DATABASE PATCH SET UPDATE"
It all comes down to personal preference, manual or auto but it won't be a surprise if eventually "auto" becomes the "recommended" way while manual is only for "troubleshooting".




Below is a full log of applying GI PSU 11.2.0.3.5 on RAC system running on RHEL6 which shows the behind the scene work of opatch auto such as alternating between grid and oracle user.
2013-01-17 11:33:35: Using Oracle CRS home /opt/app/11.2.0/grid
2013-01-17 11:33:35: Checking parameters from paramfile /opt/app/11.2.0/grid/crs/install/crsconfig_params to validate installer variables
2013-01-17 11:33:35: The configuration parameter file /opt/app/11.2.0/grid/crs/install/crsconfig_params is valid
2013-01-17 11:33:35: ### Printing the configuration values from files:
2013-01-17 11:33:35:    /opt/app/11.2.0/grid/crs/install/crsconfig_params
2013-01-17 11:33:35:    /opt/app/11.2.0/grid/OPatch/crs/s_crsconfig_defs
2013-01-17 11:33:35: ASM_AU_SIZE=1
2013-01-17 11:33:35: ASM_DISCOVERY_STRING=/dev/sd*
2013-01-17 11:33:35: ASM_DISKS=/dev/sdb1,/dev/sdc1,/dev/sdd1
2013-01-17 11:33:35: ASM_DISK_GROUP=CLUSTER_DG
2013-01-17 11:33:35: ASM_REDUNDANCY=NORMAL
2013-01-17 11:33:35: ASM_SPFILE=
2013-01-17 11:33:35: ASM_UPGRADE=false
2013-01-17 11:33:35: CLSCFG_MISSCOUNT=
2013-01-17 11:33:35: CLUSTER_GUID=
2013-01-17 11:33:35: CLUSTER_NAME=rhel6m-cluster
2013-01-17 11:33:35: CRFHOME="/opt/app/11.2.0/grid"
2013-01-17 11:33:35: CRS_LIMIT_CORE=unlimited
2013-01-17 11:33:35: CRS_LIMIT_MEMLOCK=unlimited
2013-01-17 11:33:35: CRS_LIMIT_OPENFILE=65536
2013-01-17 11:33:35: CRS_LIMIT_STACK=2048
2013-01-17 11:33:35: CRS_NODEVIPS='rhel6m1-vip/255.255.255.0/eth0,rhel6m2-vip/255.255.255.0/eth0'
2013-01-17 11:33:35: CRS_STORAGE_OPTION=1
2013-01-17 11:33:35: CSS_LEASEDURATION=400
2013-01-17 11:33:35: DIRPREFIX=
2013-01-17 11:33:35: DISABLE_OPROCD=0
2013-01-17 11:33:35: EXTERNAL_ORACLE=/opt/oracle
2013-01-17 11:33:35: EXTERNAL_ORACLE_BIN=/opt/oracle/bin
2013-01-17 11:33:35: GNS_ADDR_LIST=
2013-01-17 11:33:35: GNS_ALLOW_NET_LIST=
2013-01-17 11:33:35: GNS_CONF=false
2013-01-17 11:33:35: GNS_DENY_ITF_LIST=
2013-01-17 11:33:35: GNS_DENY_NET_LIST=
2013-01-17 11:33:35: GNS_DOMAIN_LIST=
2013-01-17 11:33:35: GPNPCONFIGDIR=/opt/app/11.2.0/grid
2013-01-17 11:33:35: GPNPGCONFIGDIR=/opt/app/11.2.0/grid
2013-01-17 11:33:35: GPNP_PA=
2013-01-17 11:33:35: HOST_NAME_LIST=rhel6m1,rhel6m2
2013-01-17 11:33:35: ID=/etc/init.d
2013-01-17 11:33:35: INIT=/sbin/init
2013-01-17 11:33:35: ISROLLING=true
2013-01-17 11:33:35: IT=/etc/inittab
2013-01-17 11:33:35: JLIBDIR=/opt/app/11.2.0/grid/jlib
2013-01-17 11:33:35: JREDIR=/opt/app/11.2.0/grid/jdk/jre/
2013-01-17 11:33:35: LANGUAGE_ID=AMERICAN_AMERICA.AL32UTF8
2013-01-17 11:33:35: MSGFILE=/var/adm/messages
2013-01-17 11:33:35: NETWORKS="eth0"/192.168.0.0:public,"eth1"/192.168.1.0:cluster_interconnect
2013-01-17 11:33:35: NEW_HOST_NAME_LIST=
2013-01-17 11:33:35: NEW_NODEVIPS='rhel6m1-vip/255.255.255.0/eth0,rhel6m2-vip/255.255.255.0/eth0'
2013-01-17 11:33:35: NEW_NODE_NAME_LIST=
2013-01-17 11:33:35: NEW_PRIVATE_NAME_LIST=
2013-01-17 11:33:35: NODELIST=rhel6m1,rhel6m2
2013-01-17 11:33:35: NODE_NAME_LIST=rhel6m1,rhel6m2
2013-01-17 11:33:35: OCFS_CONFIG=
2013-01-17 11:33:35: OCRCONFIG=/etc/oracle/ocr.loc
2013-01-17 11:33:35: OCRCONFIGDIR=/etc/oracle
2013-01-17 11:33:35: OCRID=
2013-01-17 11:33:35: OCRLOC=ocr.loc
2013-01-17 11:33:35: OCR_LOCATIONS=NO_VAL
2013-01-17 11:33:35: OLASTGASPDIR=/etc/oracle/lastgasp
2013-01-17 11:33:35: OLD_CRS_HOME=
2013-01-17 11:33:35: OLRCONFIG=/etc/oracle/olr.loc
2013-01-17 11:33:35: OLRCONFIGDIR=/etc/oracle
2013-01-17 11:33:35: OLRLOC=olr.loc
2013-01-17 11:33:35: OPROCDCHECKDIR=/etc/oracle/oprocd/check
2013-01-17 11:33:35: OPROCDDIR=/etc/oracle/oprocd
2013-01-17 11:33:35: OPROCDFATALDIR=/etc/oracle/oprocd/fatal
2013-01-17 11:33:35: OPROCDSTOPDIR=/etc/oracle/oprocd/stop
2013-01-17 11:33:35: ORACLE_BASE=/opt/app/oracle
2013-01-17 11:33:35: ORACLE_HOME=/opt/app/11.2.0/grid
2013-01-17 11:33:35: ORACLE_OWNER=grid
2013-01-17 11:33:35: ORA_ASM_GROUP=asmadmin
2013-01-17 11:33:35: ORA_DBA_GROUP=oinstall
2013-01-17 11:33:35: PRIVATE_NAME_LIST=
2013-01-17 11:33:35: RCALLDIR=/etc/rc.d/rc0.d /etc/rc.d/rc1.d /etc/rc.d/rc2.d /etc/rc.d/rc3.d /etc/rc.d/rc4.d /etc/rc.d/rc5.d /etc/rc.d/rc6.d
2013-01-17 11:33:35: RCKDIR=/etc/rc.d/rc0.d /etc/rc.d/rc1.d /etc/rc.d/rc2.d /etc/rc.d/rc3.d /etc/rc.d/rc4.d /etc/rc.d/rc6.d
2013-01-17 11:33:35: RCSDIR=/etc/rc.d/rc3.d /etc/rc.d/rc5.d
2013-01-17 11:33:35: RC_KILL=K15
2013-01-17 11:33:35: RC_KILL_OLD=K96
2013-01-17 11:33:35: RC_KILL_OLD2=K19
2013-01-17 11:33:35: RC_START=S96
2013-01-17 11:33:35: REUSEDG=false
2013-01-17 11:33:35: SCAN_NAME=rhel6m-scan
2013-01-17 11:33:35: SCAN_PORT=1521
2013-01-17 11:33:35: SCRBASE=/etc/oracle/scls_scr
2013-01-17 11:33:35: SILENT=false
2013-01-17 11:33:35: SO_EXT=so
2013-01-17 11:33:35: SRVCFGLOC=srvConfig.loc
2013-01-17 11:33:35: SRVCONFIG=/var/opt/oracle/srvConfig.loc
2013-01-17 11:33:35: SRVCONFIGDIR=/var/opt/oracle
2013-01-17 11:33:35: TZ=Europe/London
2013-01-17 11:33:35: USER_IGNORED_PREREQ=false
2013-01-17 11:33:35: VNDR_CLUSTER=false
2013-01-17 11:33:35: VOTING_DISKS=NO_VAL
2013-01-17 11:33:35: ### Printing other configuration values ###
2013-01-17 11:33:35: CLSCFG_EXTRA_PARMS=
2013-01-17 11:33:35: HAS_GROUP=oinstall
2013-01-17 11:33:35: HAS_USER=root
2013-01-17 11:33:35: HOST=rhel6m1
2013-01-17 11:33:35: OLR_DIRECTORY=/opt/app/11.2.0/grid/cdata
2013-01-17 11:33:35: OLR_LOCATION=/opt/app/11.2.0/grid/cdata/rhel6m1.olr
2013-01-17 11:33:35: ORA_CRS_HOME=/opt/app/11.2.0/grid
2013-01-17 11:33:35: SUPERUSER=root
2013-01-17 11:33:35: UNLOCK=0
2013-01-17 11:33:35: VF_DISCOVERY_STRING=
2013-01-17 11:33:35: crscfg_trace=1
2013-01-17 11:33:35: crscfg_trace_file=/opt/app/11.2.0/grid/OPatch/crs/../../cfgtoollogs/opatchauto2013-01-17_11-33-35.log
2013-01-17 11:33:35: hosts=
2013-01-17 11:33:35: osdfile=/opt/app/11.2.0/grid/OPatch/crs/s_crsconfig_defs
2013-01-17 11:33:35: parameters_valid=1
2013-01-17 11:33:35: paramfile=/opt/app/11.2.0/grid/crs/install/crsconfig_params
2013-01-17 11:33:35: platform_family=unix
2013-01-17 11:33:35: srvctl_trc_suff=0
2013-01-17 11:33:35: user_is_superuser=1
2013-01-17 11:33:35: ### Printing of configuration values complete ###
2013-01-17 11:33:35: No -patchfile specified, assuming the patch is already uncompressed
2013-01-17 11:33:35: Oracle user for /opt/app/11.2.0/grid is grid
2013-01-17 11:33:35: silent mode option is -silent -ocmrf ocm.rsp
2013-01-17 11:33:35: Bundle.xml content is <bundle type = "GI_BUNDLE">
  <entities>
     <entity location="15876003">
       <target type="crs"/>
       <target type="siha"/>
     </entity>
     <entity location="15876003/custom/server/15876003">
       <target type="rac" />
       <target type="sidb"/>
     </entity>
     <entity location="14727310">
       <target type="crs"/>
       <target type="rac"/>
       <target type="sidb"/>
       <target type="siha"/>
     </entity>
   </entities>
 </bundle>

2013-01-17 11:33:35: The patch ids are 15876003 14727310
2013-01-17 11:33:35: The patch ids are 15876003 14727310
2013-01-17 11:33:35: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch query -get_patch_type /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid
2013-01-17 11:33:35: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch query -get_patch_type /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid '
2013-01-17 11:33:39: Removing file /tmp/fileSKWEXl
2013-01-17 11:33:39: Successfully removed file: /tmp/fileSKWEXl
2013-01-17 11:33:39: /bin/su successfully executed

2013-01-17 11:33:39: output is  This patch is a "legacy_bundle_top" patch.

2013-01-17 11:33:39: Patch type is "legacy_bundle_top"
2013-01-17 11:33:39: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch query -get_patch_type /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid
2013-01-17 11:33:39: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch query -get_patch_type /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid '
2013-01-17 11:33:40: Removing file /tmp/file1oyqPq
2013-01-17 11:33:40: Successfully removed file: /tmp/file1oyqPq
2013-01-17 11:33:40: /bin/su exited with rc=0
 75
2013-01-17 11:33:40: output is 
2013-01-17 11:33:40: Patch type is 
2013-01-17 11:33:40: GI patches are /usr/local/patches/15876003 /usr/local/patches/14727310
2013-01-17 11:33:40: DB patches are /usr/local/patches/15876003/custom/server/15876003 /usr/local/patches/14727310
2013-01-17 11:33:40: Running /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1
2013-01-17 11:33:40: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1
2013-01-17 11:33:40: Command output:
>  **************************************************************
>  rhel6m1:
>  CRS-4537: Cluster Ready Services is online
>  CRS-4529: Cluster Synchronization Services is online
>  CRS-4533: Event Manager is online
>  ************************************************************** 
>End Command output
2013-01-17 11:33:40: Looking for configured databases on node rhel6m1
2013-01-17 11:33:41: Databases configured on node rhel6m1 are: std11g2
2013-01-17 11:33:41: Determining ORACLE_HOME paths for configured databases
2013-01-17 11:33:41: Executing cmd: /opt/app/11.2.0/grid/bin/srvctl config database -d std11g2
2013-01-17 11:33:43: Command output:
>  Database unique name: std11g2
>  Database name: std11g2
>  Oracle home: /opt/app/oracle/product/11.2.0/dbhome_1
>  Oracle user: oracle
>  Spfile: +DATA/std11g2/spfilestd11g2.ora
>  Domain: 
>  Start options: open
>  Stop options: immediate
>  Database role: PRIMARY
>  Management policy: AUTOMATIC
>  Server pools: std11g2
>  Database instances: std11g21,std11g22
>  Disk Groups: DATA,FLASH
>  Mount point paths: 
>  Services: myservice
>  Type: RAC
>  Database is administrator managed 
>End Command output
2013-01-17 11:33:43: output is Oracle home: /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:33:43: Oracle home for database std11g2 is /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:33:43: Oracle Home /opt/app/oracle/product/11.2.0/dbhome_1 is configured with Database(s)-> std11g2
2013-01-17 11:33:43: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle
2013-01-17 11:33:43: oracle home list is /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:33:43: Processing oracle home /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:33:43: Opening file /etc/oracle/ocr.loc
2013-01-17 11:33:43: Value (FALSE) is set for key=local_only
2013-01-17 11:33:43: Home type of /opt/app/oracle/product/11.2.0/dbhome_1 is DB
2013-01-17 11:33:43: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle
2013-01-17 11:33:43: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle
2013-01-17 11:33:43: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch version -oh /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:33:43: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch version -oh /opt/app/oracle/product/11.2.0/dbhome_1 '
2013-01-17 11:33:44: Removing file /tmp/fileTZwL3G
2013-01-17 11:33:44: Successfully removed file: /tmp/fileTZwL3G
2013-01-17 11:33:44: /bin/su successfully executed

2013-01-17 11:33:44: opatch version in oracle home /opt/app/oracle/product/11.2.0/dbhome_1  is 11.2.0.3.0

2013-01-17 11:33:44: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/15876003/custom/server/15876003 -version 11.2.0.3.0 -oh /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:33:44: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/15876003/custom/server/15876003 -version 11.2.0.3.0 -oh /opt/app/oracle/product/11.2.0/dbhome_1 '
2013-01-17 11:33:49: Removing file /tmp/filei5JgK0
2013-01-17 11:33:49: Successfully removed file: /tmp/filei5JgK0
2013-01-17 11:33:49: /bin/su successfully executed

2013-01-17 11:33:49: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/14727310 -version 11.2.0.3.0 -oh /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:33:49: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/14727310 -version 11.2.0.3.0 -oh /opt/app/oracle/product/11.2.0/dbhome_1 '
2013-01-17 11:33:52: Removing file /tmp/filekjM3wv
2013-01-17 11:33:52: Successfully removed file: /tmp/filekjM3wv
2013-01-17 11:33:52: /bin/su successfully executed

2013-01-17 11:33:52: Status of opatch version check  for /opt/app/oracle/product/11.2.0/dbhome_1 is 1
2013-01-17 11:33:52: Opatch version check passed for oracle home  /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:33:52: Processing oracle home /opt/app/11.2.0/grid
2013-01-17 11:33:52: Opening file /etc/oracle/ocr.loc
2013-01-17 11:33:52: Value (FALSE) is set for key=local_only
2013-01-17 11:33:52: Home type of /opt/app/11.2.0/grid is CRS
2013-01-17 11:33:52: Oracle user for /opt/app/11.2.0/grid is grid
2013-01-17 11:33:52: Oracle user for /opt/app/11.2.0/grid is grid
2013-01-17 11:33:52: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch version -oh /opt/app/11.2.0/grid
2013-01-17 11:33:52: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch version -oh /opt/app/11.2.0/grid '
2013-01-17 11:33:53: Removing file /tmp/fileQxFgZ8
2013-01-17 11:33:53: Successfully removed file: /tmp/fileQxFgZ8
2013-01-17 11:33:53: /bin/su successfully executed

2013-01-17 11:33:53: opatch version in oracle home /opt/app/11.2.0/grid  is 11.2.0.3.0

2013-01-17 11:33:53: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/15876003 -version 11.2.0.3.0 -oh /opt/app/11.2.0/grid
2013-01-17 11:33:53: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/15876003 -version 11.2.0.3.0 -oh /opt/app/11.2.0/grid '
2013-01-17 11:33:57: Removing file /tmp/file1noRAO
2013-01-17 11:33:57: Successfully removed file: /tmp/file1noRAO
2013-01-17 11:33:57: /bin/su successfully executed

2013-01-17 11:33:57: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/14727310 -version 11.2.0.3.0 -oh /opt/app/11.2.0/grid
2013-01-17 11:33:57: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch util checkMinimumOPatchVersion -ph /usr/local/patches/14727310 -version 11.2.0.3.0 -oh /opt/app/11.2.0/grid '
2013-01-17 11:34:00: Removing file /tmp/file3hDQCF
2013-01-17 11:34:00: Successfully removed file: /tmp/file3hDQCF
2013-01-17 11:34:00: /bin/su successfully executed

2013-01-17 11:34:00: Status of opatch version check  for /opt/app/11.2.0/grid is 1
2013-01-17 11:34:00: Opatch version check passed for oracle home  /opt/app/11.2.0/grid
2013-01-17 11:34:00: Opatch version check passed  for all oracle homes
2013-01-17 11:34:00: Opening file /etc/oracle/ocr.loc
2013-01-17 11:34:00: Value (FALSE) is set for key=local_only
2013-01-17 11:34:00: The cluster nodes are rhel6m1 rhel6m2
2013-01-17 11:34:00: checking if path /opt/app/oracle/product/11.2.0/dbhome_1 is shared
2013-01-17 11:34:00: Running as user grid: /opt/app/11.2.0/grid/bin/cluvfy comp ssa -t software -s /opt/app/oracle/product/11.2.0/dbhome_1 -n rhel6m1,rhel6m2 -display_status
2013-01-17 11:34:00: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/bin/cluvfy comp ssa -t software -s /opt/app/oracle/product/11.2.0/dbhome_1 -n rhel6m1,rhel6m2 -display_status '
2013-01-17 11:35:02: Removing file /tmp/filejzzDOD
2013-01-17 11:35:02: Successfully removed file: /tmp/filejzzDOD
2013-01-17 11:35:02: /bin/su exited with rc=0
 1
2013-01-17 11:35:02: return code for shared check is 256
2013-01-17 11:35:02: output of sharedness check is 
 Verifying shared storage accessibility 
 
 Checking shared storage accessibility...
 
 "/opt/app/oracle/product/11.2.0/dbhome_1" is not shared
 
 
 Shared storage check failed on nodes "rhel6m2,rhel6m1"
 
 Verification of shared storage accessibility was unsuccessful on all the specified nodes. 
 NODE_STATUS::rhel6m2:VFAIL
 NODE_STATUS::rhel6m1:VFAIL
 OVERALL_STATUS::VFAIL

2013-01-17 11:35:02: the ishared value is 0
2013-01-17 11:35:02: The oracle home /opt/app/oracle/product/11.2.0/dbhome_1 is not shared
2013-01-17 11:35:02: Opening file /etc/oracle/ocr.loc
2013-01-17 11:35:02: Value (FALSE) is set for key=local_only
2013-01-17 11:35:02: The cluster nodes are rhel6m1 rhel6m2
2013-01-17 11:35:02: checking if path /opt/app/11.2.0/grid/crs/install is shared
2013-01-17 11:35:02: Running as user grid: /opt/app/11.2.0/grid/bin/cluvfy comp ssa -t software -s /opt/app/11.2.0/grid/crs/install -n rhel6m1,rhel6m2 -display_status
2013-01-17 11:35:02: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/bin/cluvfy comp ssa -t software -s /opt/app/11.2.0/grid/crs/install -n rhel6m1,rhel6m2 -display_status '
2013-01-17 11:35:21: Removing file /tmp/file4Xaqdc
2013-01-17 11:35:21: Successfully removed file: /tmp/file4Xaqdc
2013-01-17 11:35:21: /bin/su exited with rc=0
 1
2013-01-17 11:35:21: return code for shared check is 256
2013-01-17 11:35:21: output of sharedness check is 
 Verifying shared storage accessibility 
 
 Checking shared storage accessibility...
 
 "/opt/app/11.2.0/grid/crs/install" is not shared
 
 
 Shared storage check failed on nodes "rhel6m2,rhel6m1"
 
 Verification of shared storage accessibility was unsuccessful on all the specified nodes. 
 NODE_STATUS::rhel6m2:VFAIL
 NODE_STATUS::rhel6m1:VFAIL
 OVERALL_STATUS::VFAIL

2013-01-17 11:35:21: the ishared value is 0
2013-01-17 11:35:21: The oracle home /opt/app/11.2.0/grid is not shared
2013-01-17 11:35:21: Processing oracle home /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:35:21: Opening file /etc/oracle/ocr.loc
2013-01-17 11:35:21: Value (FALSE) is set for key=local_only
2013-01-17 11:35:21: Home type of /opt/app/oracle/product/11.2.0/dbhome_1 is DB
2013-01-17 11:35:21: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle
2013-01-17 11:35:21: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/15876003/custom/server/15876003 -oh /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:35:21: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/15876003/custom/server/15876003 -oh /opt/app/oracle/product/11.2.0/dbhome_1 '
2013-01-17 11:35:24: Removing file /tmp/filedswEgx
2013-01-17 11:35:24: Successfully removed file: /tmp/filedswEgx
2013-01-17 11:35:24: /bin/su successfully executed

2013-01-17 11:35:24: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/14727310 -oh /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:35:24: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/14727310 -oh /opt/app/oracle/product/11.2.0/dbhome_1 '
2013-01-17 11:35:28: Removing file /tmp/filehlvsO1
2013-01-17 11:35:28: Successfully removed file: /tmp/filehlvsO1
2013-01-17 11:35:28: /bin/su successfully executed

2013-01-17 11:35:28: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle
2013-01-17 11:35:28: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/15876003/custom/server/15876003 -oh /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:35:28: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/15876003/custom/server/15876003 -oh /opt/app/oracle/product/11.2.0/dbhome_1 '
2013-01-17 11:35:35: Removing file /tmp/fileBBS2qF
2013-01-17 11:35:35: Successfully removed file: /tmp/fileBBS2qF
2013-01-17 11:35:35: /bin/su successfully executed

2013-01-17 11:35:35: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/14727310 -oh /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:35:35: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/14727310 -oh /opt/app/oracle/product/11.2.0/dbhome_1 '
2013-01-17 11:35:42: Removing file /tmp/file1i2Z7B
2013-01-17 11:35:42: Successfully removed file: /tmp/file1i2Z7B
2013-01-17 11:35:42: /bin/su successfully executed

2013-01-17 11:35:42: Status of component/conflict check  for /opt/app/oracle/product/11.2.0/dbhome_1 is 1
2013-01-17 11:35:42:  Conflict check passes for oracle home  /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:35:42: Processing oracle home /opt/app/11.2.0/grid
2013-01-17 11:35:42: Opening file /etc/oracle/ocr.loc
2013-01-17 11:35:42: Value (FALSE) is set for key=local_only
2013-01-17 11:35:42: Home type of /opt/app/11.2.0/grid is CRS
2013-01-17 11:35:42: Oracle user for /opt/app/11.2.0/grid is grid
2013-01-17 11:35:42: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid
2013-01-17 11:35:42: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid '
2013-01-17 11:35:45: Removing file /tmp/filekTThuP
2013-01-17 11:35:45: Successfully removed file: /tmp/filekTThuP
2013-01-17 11:35:45: /bin/su successfully executed

2013-01-17 11:35:45: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid
2013-01-17 11:35:45: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch prereq CheckComponents -ph /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid '
2013-01-17 11:35:48: Removing file /tmp/filersHlyc
2013-01-17 11:35:48: Successfully removed file: /tmp/filersHlyc
2013-01-17 11:35:48: /bin/su successfully executed

2013-01-17 11:35:48: Oracle user for /opt/app/11.2.0/grid is grid
2013-01-17 11:35:48: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid
2013-01-17 11:35:48: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid '
2013-01-17 11:35:56: Removing file /tmp/fileMd0bDH
2013-01-17 11:35:56: Successfully removed file: /tmp/fileMd0bDH
2013-01-17 11:35:56: /bin/su successfully executed

2013-01-17 11:35:56: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid
2013-01-17 11:35:56: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch prereq CheckConflictAgainstOH -ph /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid '
2013-01-17 11:36:03: Removing file /tmp/fileoLmZhv
2013-01-17 11:36:03: Successfully removed file: /tmp/fileoLmZhv
2013-01-17 11:36:03: /bin/su successfully executed

2013-01-17 11:36:03: Status of component/conflict check  for /opt/app/11.2.0/grid is 1
2013-01-17 11:36:03:  Conflict check passes for oracle home  /opt/app/11.2.0/grid
2013-01-17 11:36:03: Conflict check passed  for all oracle homes
2013-01-17 11:36:03: Processing oracle home /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:36:03: Opening file /etc/oracle/ocr.loc
2013-01-17 11:36:03: Value (FALSE) is set for key=local_only
2013-01-17 11:36:03: Home type of /opt/app/oracle/product/11.2.0/dbhome_1 is DB
2013-01-17 11:36:03: Performing DB patch
2013-01-17 11:36:03: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle
2013-01-17 11:36:03: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/15876003/custom/server/15876003 -oh /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:36:03: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/15876003/custom/server/15876003 -oh /opt/app/oracle/product/11.2.0/dbhome_1 '
2013-01-17 11:36:07: Removing file /tmp/file82jfJB
2013-01-17 11:36:07: Successfully removed file: /tmp/file82jfJB
2013-01-17 11:36:07: /bin/su successfully executed

2013-01-17 11:36:07: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/14727310 -oh /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:36:07: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/14727310 -oh /opt/app/oracle/product/11.2.0/dbhome_1 '
2013-01-17 11:36:11: Removing file /tmp/filegeYkOS
2013-01-17 11:36:11: Successfully removed file: /tmp/filegeYkOS
2013-01-17 11:36:11: /bin/su successfully executed

2013-01-17 11:36:11: Status of Applicable  check  for /opt/app/oracle/product/11.2.0/dbhome_1 is 1
2013-01-17 11:36:11: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl query crs activeversion
2013-01-17 11:36:11: Command output:
>  Oracle Clusterware active version on the cluster is [11.2.0.3.0] 
>End Command output
2013-01-17 11:36:11: crs version is 11 2 0 3 0
2013-01-17 11:36:11: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/bin/srvctl stop home -o /opt/app/oracle/product/11.2.0/dbhome_1 -s /opt/app/oracle/product/11.2.0/dbhome_1/srvm/admin/stophome.txt -n rhel6m1
2013-01-17 11:36:11: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/bin/srvctl stop home -o /opt/app/oracle/product/11.2.0/dbhome_1 -s /opt/app/oracle/product/11.2.0/dbhome_1/srvm/admin/stophome.txt -n rhel6m1 '
2013-01-17 11:36:29: Removing file /tmp/filegXdp4j
2013-01-17 11:36:29: Successfully removed file: /tmp/filegXdp4j
2013-01-17 11:36:29: /bin/su successfully executed

2013-01-17 11:36:29: /opt/app/oracle/product/11.2.0/dbhome_1/bin/srvctl stop home -o /opt/app/oracle/product/11.2.0/dbhome_1 -s /opt/app/oracle/product/11.2.0/dbhome_1/srvm/admin/stophome.txt -n rhel6m1 output is 
2013-01-17 11:36:29: Stopped resources from datbase home /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:36:29: Running as user oracle: /usr/local/patches/15876003/custom/server/15876003/custom/scripts/prepatch.sh -dbhome /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:36:29: s_run_as_user2: Running /bin/su oracle -c ' /usr/local/patches/15876003/custom/server/15876003/custom/scripts/prepatch.sh -dbhome /opt/app/oracle/product/11.2.0/dbhome_1 '
2013-01-17 11:36:29: Removing file /tmp/filePZl0Xw
2013-01-17 11:36:29: Successfully removed file: /tmp/filePZl0Xw
2013-01-17 11:36:29: /bin/su successfully executed

2013-01-17 11:36:29: Running as user oracle: true
2013-01-17 11:36:29: s_run_as_user2: Running /bin/su oracle -c ' true '
2013-01-17 11:36:29: Removing file /tmp/fileLQZBdK
2013-01-17 11:36:29: Successfully removed file: /tmp/fileLQZBdK
2013-01-17 11:36:29: /bin/su successfully executed

2013-01-17 11:36:29: prepatch execution for DB home ... success
2013-01-17 11:36:29: Oracle user for /opt/app/oracle/product/11.2.0/dbhome_1 is oracle
2013-01-17 11:36:29: Executing command /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply /usr/local/patches/15876003/custom/server/15876003 -local -silent -ocmrf ocm.rsp -oh /opt/app/oracle/product/11.2.0/dbhome_1 as oracle
2013-01-17 11:36:29: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply /usr/local/patches/15876003/custom/server/15876003 -local -silent -ocmrf ocm.rsp -oh /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:36:29: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply /usr/local/patches/15876003/custom/server/15876003 -local -silent -ocmrf ocm.rsp -oh /opt/app/oracle/product/11.2.0/dbhome_1 '
2013-01-17 11:37:17: Removing file /tmp/file2D3exX
2013-01-17 11:37:17: Successfully removed file: /tmp/file2D3exX
2013-01-17 11:37:17: /bin/su successfully executed

2013-01-17 11:37:17: status of apply patch is 0
2013-01-17 11:37:17: The apply patch output is Oracle Interim Patch Installer version 11.2.0.3.0
 Copyright (c) 2012, Oracle Corporation.  All rights reserved.
 
 
 Oracle Home       : /opt/app/oracle/product/11.2.0/dbhome_1
 Central Inventory : /opt/app/oraInventory
    from           : /opt/app/oracle/product/11.2.0/dbhome_1/oraInst.loc
 OPatch version    : 11.2.0.3.0
 OUI version       : 11.2.0.3.0
 Log file location : /opt/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2013-01-17_11-36-30AM_1.log
 
 Verifying environment and performing prerequisite checks...
 
 Conflicts/Supersets for each patch are:
 
 Patch : 15876003
 
  Bug Superset of 14275572
  Super set bugs are:
  14275572,  13919095,  13696251,  13348650,  12659561,  13039908,  13825231,  13036424,  12794268,  13011520,  13569812,  12758736,  13000491,  13498267,  13077654,  13001901,  13550689,  13430715,  13806545,  11675721,  14082976,  12771830,  12538907,  13947200,  13066371,  13483672,  12594616,  13540563,  12897651,  12897902,  13241779,  12896850,  12726222,  12829429,  12728585,  13079948,  12876314,  13090686,  12925041,  12995950,  13251796,  12650672,  12398492,  12848480,  13582411,  13652088,  12990582,  13857364,  12975811,  12917897,  13082238,  12947871,  13037709,  13371153,  12878750,  10114953,  11772838,  13058611,  13001955,  11836951,  12965049,  13440962,  12765467,  13727853,  13425727,  12885323,  13965075,  13339443,  12784559,  13332363,  13074261,  12971251,  13811209,  12709476,  13460353,  13523527,  12857064,  13719731,  13396284,  12899169,  13111013,  13323698,  12867511,  12639013,  12959140,  13085732,  12829917,  10317921,  13843080,  12934171,  12849377,  12349553,  13924431,  13869978,  12680491,  12914824,  13789135,  12730342,  13334158,  12950823,  10418841,  13355963,  13531373,  13776758,  12720728,  13620816,  13002015,  13023609,  13024624,  12791719
 
 Patches [   14275572 ] will be rolled back.
 
 
 Do you want to proceed? [y|n]
 Y (auto-answered by -silent)
 User Responded with: Y
 OPatch continues with these patches:   15876003  
 
 Do you want to proceed? [y|n]
 Y (auto-answered by -silent)
 User Responded with: Y
 All checks passed.
 
 Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
 (Oracle Home = '/opt/app/oracle/product/11.2.0/dbhome_1')
 
 
 Is the local system ready for patching? [y|n]
 Y (auto-answered by -silent)
 User Responded with: Y
 Backing up files...
 Applying interim patch '15876003' to OH '/opt/app/oracle/product/11.2.0/dbhome_1'
 Rolling back interim patch '14275572' from OH '/opt/app/oracle/product/11.2.0/dbhome_1'
 
 Patching component oracle.rdbms, 11.2.0.3.0...
 RollbackSession removing interim patch '14275572' from inventory
 
 
 OPatch back to application of the patch '15876003' after auto-rollback.
 
 
 Patching component oracle.rdbms, 11.2.0.3.0...
 
 Verifying the update...
 Patch 15876003 successfully applied.
 OPatch Session completed with warnings.
 Log file location: /opt/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2013-01-17_11-36-30AM_1.log
 
 OPatch completed with warnings.

2013-01-17 11:37:17: patch /usr/local/patches/15876003/custom/server/15876003  apply successful for home  /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:37:17: Executing command /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply /usr/local/patches/14727310 -local -silent -ocmrf ocm.rsp -oh /opt/app/oracle/product/11.2.0/dbhome_1 as oracle
2013-01-17 11:37:17: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply /usr/local/patches/14727310 -local -silent -ocmrf ocm.rsp -oh /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:37:17: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply /usr/local/patches/14727310 -local -silent -ocmrf ocm.rsp -oh /opt/app/oracle/product/11.2.0/dbhome_1 '
2013-01-17 11:39:59: Removing file /tmp/file5tU7W9
2013-01-17 11:39:59: Successfully removed file: /tmp/file5tU7W9
2013-01-17 11:39:59: /bin/su successfully executed

2013-01-17 11:39:59: status of apply patch is 0
2013-01-17 11:39:59: The apply patch output is Oracle Interim Patch Installer version 11.2.0.3.0
 Copyright (c) 2012, Oracle Corporation.  All rights reserved.
 
 
 Oracle Home       : /opt/app/oracle/product/11.2.0/dbhome_1
 Central Inventory : /opt/app/oraInventory
    from           : /opt/app/oracle/product/11.2.0/dbhome_1/oraInst.loc
 OPatch version    : 11.2.0.3.0
 OUI version       : 11.2.0.3.0
 Log file location : /opt/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2013-01-17_11-37-18AM_1.log
 
 Verifying environment and performing prerequisite checks...
 OPatch continues with these patches:   14727310  
 
 Do you want to proceed? [y|n]
 Y (auto-answered by -silent)
 User Responded with: Y
 All checks passed.
 
 Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
 (Oracle Home = '/opt/app/oracle/product/11.2.0/dbhome_1')
 
 
 Is the local system ready for patching? [y|n]
 Y (auto-answered by -silent)
 User Responded with: Y
 Backing up files...
 Applying sub-patch '14727310' to OH '/opt/app/oracle/product/11.2.0/dbhome_1'
 
 Patching component oracle.rdbms, 11.2.0.3.0...
 
 Patching component oracle.rdbms.dbscripts, 11.2.0.3.0...
 
 Patching component oracle.rdbms.deconfig, 11.2.0.3.0...
 
 Patching component oracle.rdbms.rsf, 11.2.0.3.0...
 
 Patching component oracle.sdo.locator, 11.2.0.3.0...
 
 Patching component oracle.sysman.console.db, 11.2.0.3.0...
 
 Patching component oracle.sysman.oms.core, 10.2.0.4.4...
 
 Verifying the update...
 Composite patch 14727310 successfully applied.
 Log file location: /opt/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2013-01-17_11-37-18AM_1.log
 
 OPatch succeeded.

2013-01-17 11:39:59: patch /usr/local/patches/14727310  apply successful for home  /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:39:59: Running as user oracle: /usr/local/patches/15876003/custom/server/15876003/custom/scripts/postpatch.sh -dbhome /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:39:59: s_run_as_user2: Running /bin/su oracle -c ' /usr/local/patches/15876003/custom/server/15876003/custom/scripts/postpatch.sh -dbhome /opt/app/oracle/product/11.2.0/dbhome_1 '
2013-01-17 11:39:59: Removing file /tmp/fileeOwh17
2013-01-17 11:39:59: Successfully removed file: /tmp/fileeOwh17
2013-01-17 11:39:59: /bin/su successfully executed

2013-01-17 11:39:59: Running as user oracle: true
2013-01-17 11:39:59: s_run_as_user2: Running /bin/su oracle -c ' true '
2013-01-17 11:39:59: Removing file /tmp/file3qIa16
2013-01-17 11:39:59: Successfully removed file: /tmp/file3qIa16
2013-01-17 11:39:59: /bin/su successfully executed

2013-01-17 11:39:59: postpatch execution for DB home ... success
2013-01-17 11:39:59: Processing oracle home /opt/app/11.2.0/grid
2013-01-17 11:39:59: Opening file /etc/oracle/ocr.loc
2013-01-17 11:39:59: Value (FALSE) is set for key=local_only
2013-01-17 11:39:59: Home type of /opt/app/11.2.0/grid is CRS
2013-01-17 11:39:59: Unlock crshome...
2013-01-17 11:39:59: Exclude file used is /opt/app/11.2.0/grid/OPatch/crs/installPatch.excl
2013-01-17 11:39:59: Home location in olr.loc is 
2013-01-17 11:39:59: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stop crs -f
2013-01-17 11:41:59: Command output:
>  CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel6m1'
>  CRS-2673: Attempting to stop 'ora.crsd' on 'rhel6m1'
>  CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rhel6m1'
>  CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rhel6m1'
>  CRS-2673: Attempting to stop 'ora.CLUSTER_DG.dg' on 'rhel6m1'
>  CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rhel6m1'
>  CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rhel6m1'
>  CRS-2673: Attempting to stop 'ora.FLASH.dg' on 'rhel6m1'
>  CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rhel6m1' succeeded
>  CRS-2673: Attempting to stop 'ora.rhel6m1.vip' on 'rhel6m1'
>  CRS-2677: Stop of 'ora.rhel6m1.vip' on 'rhel6m1' succeeded
>  CRS-2672: Attempting to start 'ora.rhel6m1.vip' on 'rhel6m2'
>  CRS-2677: Stop of 'ora.DATA.dg' on 'rhel6m1' succeeded
>  CRS-2677: Stop of 'ora.registry.acfs' on 'rhel6m1' succeeded
>  CRS-2676: Start of 'ora.rhel6m1.vip' on 'rhel6m2' succeeded
>  CRS-2677: Stop of 'ora.FLASH.dg' on 'rhel6m1' succeeded
>  CRS-2677: Stop of 'ora.CLUSTER_DG.dg' on 'rhel6m1' succeeded
>  CRS-2673: Attempting to stop 'ora.asm' on 'rhel6m1'
>  CRS-2677: Stop of 'ora.asm' on 'rhel6m1' succeeded
>  CRS-2673: Attempting to stop 'ora.ons' on 'rhel6m1'
>  CRS-2677: Stop of 'ora.ons' on 'rhel6m1' succeeded
>  CRS-2673: Attempting to stop 'ora.net1.network' on 'rhel6m1'
>  CRS-2677: Stop of 'ora.net1.network' on 'rhel6m1' succeeded
>  CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rhel6m1' has completed
>  CRS-2677: Stop of 'ora.crsd' on 'rhel6m1' succeeded
>  CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rhel6m1'
>  CRS-2673: Attempting to stop 'ora.mdnsd' on 'rhel6m1'
>  CRS-2673: Attempting to stop 'ora.crf' on 'rhel6m1'
>  CRS-2673: Attempting to stop 'ora.ctssd' on 'rhel6m1'
>  CRS-2673: Attempting to stop 'ora.evmd' on 'rhel6m1'
>  CRS-2673: Attempting to stop 'ora.asm' on 'rhel6m1'
>  CRS-2677: Stop of 'ora.crf' on 'rhel6m1' succeeded
>  CRS-2677: Stop of 'ora.mdnsd' on 'rhel6m1' succeeded
>  CRS-2677: Stop of 'ora.evmd' on 'rhel6m1' succeeded
>  CRS-2677: Stop of 'ora.asm' on 'rhel6m1' succeeded
>  CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rhel6m1'
>  CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rhel6m1' succeeded
>  CRS-2677: Stop of 'ora.ctssd' on 'rhel6m1' succeeded
>  CRS-2673: Attempting to stop 'ora.cssd' on 'rhel6m1'
>  CRS-2677: Stop of 'ora.drivers.acfs' on 'rhel6m1' succeeded
>  CRS-2677: Stop of 'ora.cssd' on 'rhel6m1' succeeded
>  CRS-2673: Attempting to stop 'ora.gipcd' on 'rhel6m1'
>  CRS-2677: Stop of 'ora.gipcd' on 'rhel6m1' succeeded
>  CRS-2673: Attempting to stop 'ora.gpnpd' on 'rhel6m1'
>  CRS-2677: Stop of 'ora.gpnpd' on 'rhel6m1' succeeded
>  CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel6m1' has completed
>  CRS-4133: Oracle High Availability Services has been stopped. 
>End Command output
2013-01-17 11:41:59: /opt/app/11.2.0/grid/bin/crsctl stop crs -f
2013-01-17 11:41:59: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1
2013-01-17 11:42:00: Command output:
>  CRS-4639: Could not contact Oracle High Availability Services
>  CRS-4000: Command Check failed, or completed with errors. 
>End Command output
2013-01-17 11:42:01: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check has
2013-01-17 11:42:02: Command output:
>  CRS-4639: Could not contact Oracle High Availability Services 
>End Command output
2013-01-17 11:44:10: Waiting for complete CRS stack to stop
2013-01-17 11:44:10: Invoking removeproc to clean oracle client procs
2013-01-17 11:44:10: Executing cmd: /sbin/fuser -k /opt/app/11.2.0/grid/bin/crsctl.bin
2013-01-17 11:44:10: fuser command output for /opt/app/11.2.0/grid/bin/crsctl.bin is 
2013-01-17 11:44:10: Oracle user for /opt/app/11.2.0/grid is grid
2013-01-17 11:44:10: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid
2013-01-17 11:44:10: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/15876003 -oh /opt/app/11.2.0/grid '
2013-01-17 11:44:14: Removing file /tmp/fileCAOkny
2013-01-17 11:44:14: Successfully removed file: /tmp/fileCAOkny
2013-01-17 11:44:14: /bin/su successfully executed

2013-01-17 11:44:14: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid
2013-01-17 11:44:14: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /usr/local/patches/14727310 -oh /opt/app/11.2.0/grid '
2013-01-17 11:44:17: Removing file /tmp/filepYiTfa
2013-01-17 11:44:17: Successfully removed file: /tmp/filepYiTfa
2013-01-17 11:44:17: /bin/su successfully executed

2013-01-17 11:44:17: Status of Applicable  check  for /opt/app/11.2.0/grid is 1
2013-01-17 11:44:17: Oracle user for /opt/app/11.2.0/grid is grid
2013-01-17 11:44:17: Executing command /opt/app/11.2.0/grid/OPatch/opatch napply /usr/local/patches/15876003 -local -silent -ocmrf ocm.rsp -oh /opt/app/11.2.0/grid as grid
2013-01-17 11:44:17: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch napply /usr/local/patches/15876003 -local -silent -ocmrf ocm.rsp -oh /opt/app/11.2.0/grid
2013-01-17 11:44:17: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch napply /usr/local/patches/15876003 -local -silent -ocmrf ocm.rsp -oh /opt/app/11.2.0/grid '
2013-01-17 11:48:18: Removing file /tmp/fileftFTxU
2013-01-17 11:48:18: Successfully removed file: /tmp/fileftFTxU
2013-01-17 11:48:18: /bin/su successfully executed

2013-01-17 11:48:18: status of apply patch is 0
2013-01-17 11:48:18: The apply patch output is Oracle Interim Patch Installer version 11.2.0.3.0
 Copyright (c) 2012, Oracle Corporation.  All rights reserved.
 
 
 Oracle Home       : /opt/app/11.2.0/grid
 Central Inventory : /opt/app/oraInventory
    from           : /opt/app/11.2.0/grid/oraInst.loc
 OPatch version    : 11.2.0.3.0
 OUI version       : 11.2.0.3.0
 Log file location : /opt/app/11.2.0/grid/cfgtoollogs/opatch/opatch2013-01-17_11-44-18AM_1.log
 
 Verifying environment and performing prerequisite checks...
 
 Conflicts/Supersets for each patch are:
 
 Patch : 15876003
 
  Bug Superset of 14275572
  Super set bugs are:
  14275572,  13919095,  13696251,  13348650,  12659561,  13039908,  13825231,  13036424,  12794268,  13011520,  13569812,  12758736,  13000491,  13498267,  13077654,  13001901,  13550689,  13430715,  13806545,  11675721,  14082976,  12771830,  12538907,  13947200,  13066371,  13483672,  12594616,  13540563,  12897651,  12897902,  13241779,  12896850,  12726222,  12829429,  12728585,  13079948,  12876314,  13090686,  12925041,  12995950,  13251796,  12650672,  12398492,  12848480,  13582411,  13652088,  12990582,  13857364,  12975811,  12917897,  13082238,  12947871,  13037709,  13371153,  12878750,  10114953,  11772838,  13058611,  13001955,  11836951,  12965049,  13440962,  12765467,  13727853,  13425727,  12885323,  13965075,  13339443,  12784559,  13332363,  13074261,  12971251,  13811209,  12709476,  13460353,  13523527,  12857064,  13719731,  13396284,  12899169,  13111013,  13323698,  12867511,  12639013,  12959140,  13085732,  12829917,  10317921,  13843080,  12934171,  12849377,  12349553,  13924431,  13869978,  12680491,  12914824,  13789135,  12730342,  13334158,  12950823,  10418841,  13355963,  13531373,  13776758,  12720728,  13620816,  13002015,  13023609,  13024624,  12791719,  13886023,  13255295,  13821454,  12782756,  14625969,  14152875,  14186070,  12873909,  14214257,  12914722,  13243172,  12842804,  13045518,  12765868,  12772345,  12663376,  13345868,  14059576,  13683090,  12932852,  13889047,  12695029,  13146560,  13038806,  14251904,  14070200,  13820621,  14304758,  13396356,  13697828,  13258062,  12834777,  12996572,  13941934,  13657366,  13019958,  12810890,  13888719,  13502441,  13726162,  13880925,  14153867,  13506114,  12820045,  13604057,  12823838,  13877508,  12823042,  14494305,  13582706,  13617861,  12825835,  13263435,  13025879,  13853089,  14009845,  13410987,  13570879,  13637590,  12827493,  13247273,  13068077
 
 Patches [   14275572 ] will be rolled back.
 
 
 Do you want to proceed? [y|n]
 Y (auto-answered by -silent)
 User Responded with: Y
 OPatch continues with these patches:   15876003  
 
 Do you want to proceed? [y|n]
 Y (auto-answered by -silent)
 User Responded with: Y
 All checks passed.
 
 Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
 (Oracle Home = '/opt/app/11.2.0/grid')
 
 
 Is the local system ready for patching? [y|n]
 Y (auto-answered by -silent)
 User Responded with: Y
 Backing up files...
 Applying interim patch '15876003' to OH '/opt/app/11.2.0/grid'
 Rolling back interim patch '14275572' from OH '/opt/app/11.2.0/grid'
 
 Patching component oracle.crs, 11.2.0.3.0...
 
 Patching component oracle.usm, 11.2.0.3.0...
 RollbackSession removing interim patch '14275572' from inventory
 
 
 OPatch back to application of the patch '15876003' after auto-rollback.
 
 
 Patching component oracle.crs, 11.2.0.3.0...
 
 Patching component oracle.usm, 11.2.0.3.0...
 
 Verifying the update...
 Patch 15876003 successfully applied.
 OPatch Session completed with warnings.
 Log file location: /opt/app/11.2.0/grid/cfgtoollogs/opatch/opatch2013-01-17_11-44-18AM_1.log
 
 OPatch completed with warnings.

2013-01-17 11:48:18: patch /usr/local/patches/15876003  apply successful for home  /opt/app/11.2.0/grid
2013-01-17 11:48:18: Executing command /opt/app/11.2.0/grid/OPatch/opatch napply /usr/local/patches/14727310 -local -silent -ocmrf ocm.rsp -oh /opt/app/11.2.0/grid as grid
2013-01-17 11:48:18: Running as user grid: /opt/app/11.2.0/grid/OPatch/opatch napply /usr/local/patches/14727310 -local -silent -ocmrf ocm.rsp -oh /opt/app/11.2.0/grid
2013-01-17 11:48:18: s_run_as_user2: Running /bin/su grid -c ' /opt/app/11.2.0/grid/OPatch/opatch napply /usr/local/patches/14727310 -local -silent -ocmrf ocm.rsp -oh /opt/app/11.2.0/grid '
2013-01-17 11:49:39: Removing file /tmp/file5fgSjJ
2013-01-17 11:49:39: Successfully removed file: /tmp/file5fgSjJ
2013-01-17 11:49:39: /bin/su successfully executed

2013-01-17 11:49:39: status of apply patch is 0
2013-01-17 11:49:39: The apply patch output is Oracle Interim Patch Installer version 11.2.0.3.0
 Copyright (c) 2012, Oracle Corporation.  All rights reserved.
 
 
 Oracle Home       : /opt/app/11.2.0/grid
 Central Inventory : /opt/app/oraInventory
    from           : /opt/app/11.2.0/grid/oraInst.loc
 OPatch version    : 11.2.0.3.0
 OUI version       : 11.2.0.3.0
 Log file location : /opt/app/11.2.0/grid/cfgtoollogs/opatch/opatch2013-01-17_11-48-19AM_1.log
 
 Verifying environment and performing prerequisite checks...
 OPatch continues with these patches:   14727310  
 
 Do you want to proceed? [y|n]
 Y (auto-answered by -silent)
 User Responded with: Y
 All checks passed.
 
 Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
 (Oracle Home = '/opt/app/11.2.0/grid')
 
 
 Is the local system ready for patching? [y|n]
 Y (auto-answered by -silent)
 User Responded with: Y
 Backing up files...
 Applying sub-patch '14727310' to OH '/opt/app/11.2.0/grid'
 ApplySession: Optional component(s) [ oracle.sysman.console.db, 11.2.0.3.0 ] , [ oracle.sysman.oms.core, 10.2.0.4.4 ]  not present in the Oracle Home or a higher version is found.
 
 Patching component oracle.rdbms, 11.2.0.3.0...
 
 Patching component oracle.rdbms.dbscripts, 11.2.0.3.0...
 
 Patching component oracle.rdbms.deconfig, 11.2.0.3.0...
 
 Patching component oracle.rdbms.rsf, 11.2.0.3.0...
 
 Patching component oracle.sdo.locator, 11.2.0.3.0...
 
 Verifying the update...
 find: `./crf/admin/run/crfmond': Permission denied
 find: `./crf/admin/run/crflogd': Permission denied
 Composite patch 14727310 successfully applied.
 Log file location: /opt/app/11.2.0/grid/cfgtoollogs/opatch/opatch2013-01-17_11-48-19AM_1.log
 
 OPatch succeeded.

2013-01-17 11:49:39: patch /usr/local/patches/14727310  apply successful for home  /opt/app/11.2.0/grid
2013-01-17 11:49:39: Performing Post patch actions
2013-01-17 11:49:39: norestart flag is set to 
2013-01-17 11:49:39: Opening file /etc/oracle/ocr.loc
2013-01-17 11:49:39: Value (FALSE) is set for key=local_only
2013-01-17 11:49:39: Performing Post patch actions for Grid Home /opt/app/11.2.0/grid
2013-01-17 11:49:39: Executing cmd: /opt/app/11.2.0/grid/rdbms/install/rootadd_rdbms.sh
2013-01-17 11:49:39: setrdbmsfileperms succeeded
2013-01-17 11:49:39: Patching Oracle Clusterware
2013-01-17 11:49:39: norestart flag is set to 
2013-01-17 11:49:39: Executing cmd: /bin/rpm -q sles-release
2013-01-17 11:49:39: Command output:
>  package sles-release is not installed 
>End Command output
2013-01-17 11:49:39: init file = /opt/app/11.2.0/grid/crs/init/init.ohasd
2013-01-17 11:49:39: Copying file /opt/app/11.2.0/grid/crs/init/init.ohasd to /etc/init.d directory
2013-01-17 11:49:39: Setting init.ohasd permission in /etc/init.d directory
2013-01-17 11:49:39: init file = /opt/app/11.2.0/grid/crs/init/ohasd
2013-01-17 11:49:39: Copying file /opt/app/11.2.0/grid/crs/init/ohasd to /etc/init.d directory
2013-01-17 11:49:39: Setting ohasd permission in /etc/init.d directory
2013-01-17 11:49:39: Executing cmd: /bin/rpm -q sles-release
2013-01-17 11:49:39: Command output:
>  package sles-release is not installed 
>End Command output
2013-01-17 11:49:39: Removing "/etc/rc.d/rc3.d/S96ohasd"
2013-01-17 11:49:39: Removing file /etc/rc.d/rc3.d/S96ohasd
2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc3.d/S96ohasd
2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc3.d/S96ohasd" pointing to /etc/init.d/ohasd
2013-01-17 11:49:39: Removing "/etc/rc.d/rc5.d/S96ohasd"
2013-01-17 11:49:39: Removing file /etc/rc.d/rc5.d/S96ohasd
2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc5.d/S96ohasd
2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc5.d/S96ohasd" pointing to /etc/init.d/ohasd
2013-01-17 11:49:39: Removing "/etc/rc.d/rc0.d/K15ohasd"
2013-01-17 11:49:39: Removing file /etc/rc.d/rc0.d/K15ohasd
2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc0.d/K15ohasd
2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc0.d/K15ohasd" pointing to /etc/init.d/ohasd
2013-01-17 11:49:39: Removing "/etc/rc.d/rc1.d/K15ohasd"
2013-01-17 11:49:39: Removing file /etc/rc.d/rc1.d/K15ohasd
2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc1.d/K15ohasd
2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc1.d/K15ohasd" pointing to /etc/init.d/ohasd
2013-01-17 11:49:39: Removing "/etc/rc.d/rc2.d/K15ohasd"
2013-01-17 11:49:39: Removing file /etc/rc.d/rc2.d/K15ohasd
2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc2.d/K15ohasd
2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc2.d/K15ohasd" pointing to /etc/init.d/ohasd
2013-01-17 11:49:39: Removing "/etc/rc.d/rc3.d/K15ohasd"
2013-01-17 11:49:39: Removing file /etc/rc.d/rc3.d/K15ohasd
2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc3.d/K15ohasd
2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc3.d/K15ohasd" pointing to /etc/init.d/ohasd
2013-01-17 11:49:39: Removing "/etc/rc.d/rc4.d/K15ohasd"
2013-01-17 11:49:39: Removing file /etc/rc.d/rc4.d/K15ohasd
2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc4.d/K15ohasd
2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc4.d/K15ohasd" pointing to /etc/init.d/ohasd
2013-01-17 11:49:39: Removing "/etc/rc.d/rc6.d/K15ohasd"
2013-01-17 11:49:39: Removing file /etc/rc.d/rc6.d/K15ohasd
2013-01-17 11:49:39: Successfully removed file: /etc/rc.d/rc6.d/K15ohasd
2013-01-17 11:49:39: Creating a link "/etc/rc.d/rc6.d/K15ohasd" pointing to /etc/init.d/ohasd
2013-01-17 11:49:39: The file ohasd has been successfully linked to the RC directories
2013-01-17 11:49:39: Executing /opt/app/11.2.0/grid/bin/acfsroot install
2013-01-17 11:49:39: Executing cmd: /opt/app/11.2.0/grid/bin/acfsroot install
2013-01-17 11:51:21: Command output:
>  ACFS-9300: ADVM/ACFS distribution files found.
>  ACFS-9312: Existing ADVM/ACFS installation detected.
>  ACFS-9314: Removing previous ADVM/ACFS installation.
>  ACFS-9315: Previous ADVM/ACFS components successfully removed.
>  ACFS-9307: Installing requested ADVM/ACFS software.
>  ACFS-9308: Loading installed ADVM/ACFS drivers.
>  ACFS-9321: Creating udev for ADVM/ACFS.
>  ACFS-9323: Creating module dependencies - this may take some time.
>  ACFS-9154: Loading 'oracleoks.ko' driver.
>  ACFS-9154: Loading 'oracleadvm.ko' driver.
>  ACFS-9154: Loading 'oracleacfs.ko' driver.
>  ACFS-9327: Verifying ADVM/ACFS devices.
>  ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
>  ACFS-9156: Detecting control device '/dev/ofsctl'.
>  ACFS-9309: ADVM/ACFS installation correctness verified. 
>End Command output
2013-01-17 11:51:21: /opt/app/11.2.0/grid/bin/acfsroot install ... success
2013-01-17 11:51:21: USM driver install status is 1
2013-01-17 11:51:21: Validate crsctl command
2013-01-17 11:51:21: Validating /opt/app/11.2.0/grid/bin/crsctl
2013-01-17 11:51:21: Starting Oracle Clusterware
2013-01-17 11:51:21: Executing /opt/app/11.2.0/grid/bin/crsctl start crs
2013-01-17 11:51:21: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl start crs
2013-01-17 11:51:28: Command output:
>  CRS-4123: Oracle High Availability Services has been started. 
>End Command output
2013-01-17 11:51:28: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource
2013-01-17 11:51:30: Command output:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4000: Command Status failed, or completed with errors. 
>End Command output
2013-01-17 11:51:30: Waiting for Oracle CRSD and EVMD to start
2013-01-17 11:51:35: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource
2013-01-17 11:51:36: Command output:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4000: Command Status failed, or completed with errors. 
>End Command output
2013-01-17 11:51:36: Waiting for Oracle CRSD and EVMD to start
2013-01-17 11:51:41: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource
2013-01-17 11:51:43: Command output:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4000: Command Status failed, or completed with errors. 
>End Command output
2013-01-17 11:51:43: Waiting for Oracle CRSD and EVMD to start
2013-01-17 11:51:48: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource
2013-01-17 11:51:50: Command output:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4000: Command Status failed, or completed with errors. 
>End Command output
2013-01-17 11:51:50: Waiting for Oracle CRSD and EVMD to start
2013-01-17 11:51:55: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource
2013-01-17 11:51:57: Command output:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4000: Command Status failed, or completed with errors. 
>End Command output
2013-01-17 11:51:57: Waiting for Oracle CRSD and EVMD to start
2013-01-17 11:52:02: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource
2013-01-17 11:52:04: Command output:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4000: Command Status failed, or completed with errors. 
>End Command output
2013-01-17 11:52:04: Waiting for Oracle CRSD and EVMD to start
2013-01-17 11:52:09: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource
2013-01-17 11:52:10: Command output:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4000: Command Status failed, or completed with errors. 
>End Command output
2013-01-17 11:52:10: Waiting for Oracle CRSD and EVMD to start
2013-01-17 11:52:15: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource
2013-01-17 11:52:17: Command output:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4000: Command Status failed, or completed with errors. 
>End Command output
2013-01-17 11:52:17: Waiting for Oracle CRSD and EVMD to start
2013-01-17 11:52:22: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource
2013-01-17 11:52:24: Command output:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4000: Command Status failed, or completed with errors. 
>End Command output
2013-01-17 11:52:24: Waiting for Oracle CRSD and EVMD to start
2013-01-17 11:52:29: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource
2013-01-17 11:52:29: Command output:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4000: Command Status failed, or completed with errors. 
>End Command output
2013-01-17 11:52:29: Waiting for Oracle CRSD and EVMD to start
2013-01-17 11:52:34: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource
2013-01-17 11:52:34: Command output:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4000: Command Status failed, or completed with errors. 
>End Command output
2013-01-17 11:52:34: Waiting for Oracle CRSD and EVMD to start
2013-01-17 11:52:39: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource
2013-01-17 11:52:40: Command output:
>  NAME=ora.CLUSTER_DG.dg
>  TYPE=ora.diskgroup.type
>  TARGET=ONLINE           , ONLINE
>  STATE=ONLINE on rhel6m1, ONLINE on rhel6m2
>  
>  NAME=ora.DATA.dg
>  TYPE=ora.diskgroup.type
>  TARGET=ONLINE           , ONLINE
>  STATE=ONLINE on rhel6m1, ONLINE on rhel6m2
>  
>  NAME=ora.FLASH.dg
>  TYPE=ora.diskgroup.type
>  TARGET=ONLINE           , ONLINE
>  STATE=ONLINE on rhel6m1, ONLINE on rhel6m2
>  
>  NAME=ora.LISTENER.lsnr
>  TYPE=ora.listener.type
>  TARGET=ONLINE , ONLINE
>  STATE=OFFLINE, ONLINE on rhel6m2
>  
>  NAME=ora.LISTENER_SCAN1.lsnr
>  TYPE=ora.scan_listener.type
>  TARGET=ONLINE
>  STATE=ONLINE on rhel6m2
>  
>  NAME=ora.asm
>  TYPE=ora.asm.type
>  TARGET=ONLINE           , ONLINE
>  STATE=ONLINE on rhel6m1, ONLINE on rhel6m2
>  
>  NAME=ora.cvu
>  TYPE=ora.cvu.type
>  TARGET=ONLINE
>  STATE=ONLINE on rhel6m2
>  
>  NAME=ora.gsd
>  TYPE=ora.gsd.type
>  TARGET=OFFLINE, OFFLINE
>  STATE=OFFLINE, OFFLINE
>  
>  NAME=ora.net1.network
>  TYPE=ora.network.type
>  TARGET=ONLINE           , ONLINE
>  STATE=ONLINE on rhel6m1, ONLINE on rhel6m2
>  
>  NAME=ora.oc4j
>  TYPE=ora.oc4j.type
>  TARGET=ONLINE
>  STATE=ONLINE on rhel6m2
>  
>  NAME=ora.ons
>  TYPE=ora.ons.type
>  TARGET=ONLINE           , ONLINE
>  STATE=ONLINE on rhel6m1, ONLINE on rhel6m2
>  
>  NAME=ora.registry.acfs
>  TYPE=ora.registry.acfs.type
>  TARGET=ONLINE           , ONLINE
>  STATE=ONLINE on rhel6m1, ONLINE on rhel6m2
>  
>  NAME=ora.rhel6m1.vip
>  TYPE=ora.cluster_vip_net1.type
>  TARGET=ONLINE
>  STATE=ONLINE on rhel6m1
>  
>  NAME=ora.rhel6m2.vip
>  TYPE=ora.cluster_vip_net1.type
>  TARGET=ONLINE
>  STATE=ONLINE on rhel6m2
>  
>  NAME=ora.scan1.vip
>  TYPE=ora.scan_vip.type
>  TARGET=ONLINE
>  STATE=ONLINE on rhel6m2
>  
>  NAME=ora.std11g2.db
>  TYPE=ora.database.type
>  TARGET=OFFLINE, ONLINE
>  STATE=OFFLINE, ONLINE on rhel6m2
>  
>  NAME=ora.std11g2.myservice.svc
>  TYPE=ora.service.type
>  TARGET=OFFLINE, ONLINE
>  STATE=OFFLINE, ONLINE on rhel6m2
>   
>End Command output
2013-01-17 11:52:40: Running /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1
2013-01-17 11:52:40: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1
2013-01-17 11:52:40: Command output:
>  **************************************************************
>  rhel6m1:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4529: Cluster Synchronization Services is online
>  CRS-4533: Event Manager is online
>  ************************************************************** 
>End Command output
2013-01-17 11:52:40: Checking the status of cluster
2013-01-17 11:52:45: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1
2013-01-17 11:52:45: Command output:
>  **************************************************************
>  rhel6m1:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4529: Cluster Synchronization Services is online
>  CRS-4533: Event Manager is online
>  ************************************************************** 
>End Command output
2013-01-17 11:52:45: Checking the status of cluster
2013-01-17 11:52:50: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1
2013-01-17 11:52:50: Command output:
>  **************************************************************
>  rhel6m1:
>  CRS-4535: Cannot communicate with Cluster Ready Services
>  CRS-4529: Cluster Synchronization Services is online
>  CRS-4533: Event Manager is online
>  ************************************************************** 
>End Command output
2013-01-17 11:52:50: Checking the status of cluster
2013-01-17 11:52:55: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl check cluster -n rhel6m1
2013-01-17 11:52:55: Command output:
>  **************************************************************
>  rhel6m1:
>  CRS-4537: Cluster Ready Services is online
>  CRS-4529: Cluster Synchronization Services is online
>  CRS-4533: Event Manager is online
>  ************************************************************** 
>End Command output
2013-01-17 11:52:55: Oracle CRS stack installed and running
2013-01-17 11:52:55: Performing Post patch actions
2013-01-17 11:52:55: norestart flag is set to 
2013-01-17 11:52:55: Opening file /etc/oracle/ocr.loc
2013-01-17 11:52:55: Value (FALSE) is set for key=local_only
2013-01-17 11:52:55: Performing Post Patch start  action for DB Home /opt/app/oracle/product/11.2.0/dbhome_1
2013-01-17 11:52:55: Executing cmd: /opt/app/11.2.0/grid/bin/crsctl stat resource -c rhel6m1
2013-01-17 11:52:56: Command output:
>  NAME=ora.CLUSTER_DG.dg
>  TYPE=ora.diskgroup.type
>  TARGET=ONLINE
>  STATE=ONLINE
>  
>  NAME=ora.DATA.dg
>  TYPE=ora.diskgroup.type
>  TARGET=ONLINE
>  STATE=ONLINE
>  
>  NAME=ora.FLASH.dg
>  TYPE=ora.diskgroup.type
>  TARGET=ONLINE
>  STATE=ONLINE
>  
>  NAME=ora.LISTENER.lsnr
>  TYPE=ora.listener.type
>  TARGET=ONLINE
>  STATE=ONLINE
>  
>  NAME=ora.asm
>  TYPE=ora.asm.type
>  TARGET=ONLINE
>  STATE=ONLINE
>  
>  NAME=ora.gsd
>  TYPE=ora.gsd.type
>  TARGET=OFFLINE
>  STATE=OFFLINE
>  
>  NAME=ora.net1.network
>  TYPE=ora.network.type
>  TARGET=ONLINE
>  STATE=ONLINE
>  
>  NAME=ora.ons
>  TYPE=ora.ons.type
>  TARGET=ONLINE
>  STATE=ONLINE
>  
>  NAME=ora.registry.acfs
>  TYPE=ora.registry.acfs.type
>  TARGET=ONLINE
>  STATE=ONLINE
>  
>  NAME=ora.rhel6m1.vip
>  TYPE=ora.cluster_vip_net1.type
>  CARDINALITY_ID=1
>  TARGET=ONLINE
>  STATE=ONLINE
>   
>End Command output
2013-01-17 11:52:56: Server assignments completed. Ready to start databases
2013-01-17 11:52:56: Running as user oracle: /opt/app/oracle/product/11.2.0/dbhome_1/bin/srvctl start home -o /opt/app/oracle/product/11.2.0/dbhome_1 -s /opt/app/oracle/product/11.2.0/dbhome_1/srvm/admin/stophome.txt -n rhel6m1
2013-01-17 11:52:56: s_run_as_user2: Running /bin/su oracle -c ' /opt/app/oracle/product/11.2.0/dbhome_1/bin/srvctl start home -o /opt/app/oracle/product/11.2.0/dbhome_1 -s /opt/app/oracle/product/11.2.0/dbhome_1/srvm/admin/stophome.txt -n rhel6m1 '
2013-01-17 11:53:16: Removing file /tmp/filePCaf67
2013-01-17 11:53:16: Successfully removed file: /tmp/filePCaf67
2013-01-17 11:53:16: /bin/su successfully executed

2013-01-17 11:53:16: /opt/app/oracle/product/11.2.0/dbhome_1/bin/srvctl start home -o /opt/app/oracle/product/11.2.0/dbhome_1 -s /opt/app/oracle/product/11.2.0/dbhome_1/srvm/admin/stophome.txt -n rhel6m1 output is 
2013-01-17 11:53:16: Started resources from datbase home /opt/app/oracle/product/11.2.0/dbhome_1
Posted by at
Labels: , ,

Tuesday, November 6, 2012

warning: ignoring old commands for target `pcscfg.cfg'

Following could be seen when applying PSU 11.2.0.3.4 on a DB home
OPatch found the word "warning" in the stderr of the make command.
Please look at this stderr. You can re-run this make command.
Stderr output:
ins_precomp.mk:19: warning: overriding commands for target `pcscfg.cfg'
/opt/app/oracle/product/11.2.0/ent11.2.0.3/precomp/lib/env_precomp.mk:2160: warning: ignoring old commands for target `pcscfg.cfg'
/opt/app/oracle/product/11.2.0/ent11.2.0.3/precomp/lib/ins_precomp.mk:19: warning: overriding commands for target `pcscfg.cfg'
/opt/app/oracle/product/11.2.0/ent11.2.0.3/precomp/lib/env_precomp.mk:2160: warning: ignoring old commands for target `pcscfg.cfg'
This is only a warning and patch has been successfully applied.

Useful metalink notes
Opatch warning: overriding commands for target xxxx [ID 1448337.1]

Related Post
ins_srvm.mk: warning: overriding commands for target libsrvm11.so
Posted by at
Labels: , , , ,
Subscribe to: Posts (Atom)