The blog post list the steps for upgrading 10gR2 system (
used earlier for 10.2.0.4 to 10.2.0.5 upgrade) to 11gR2 (11.2.0.3). The steps are similar to
11.1.0.7 to 11.2.0.3 upgrade but with some minor differences.
Relevant sections from 11gR2 Upgrade Guide.
Direct upgrade to the new Oracle Database 11g release is supported from 9.2.0.8 or higher, 10.1.0.5 or higher, 10.2.0.2 or higher, and 11.1.0.6 or higher. Note that Oracle Clusterware release 10.2.0.n must be at release 10.2.0.3 (or higher), before you attempt to upgrade it to Oracle Clusterware 11g.
You may need to keep the old Oracle Home for a while after a successful upgrade. However, after you have tested the newly upgraded Oracle database, you will need to remove the old Oracle Home. As the clusterware user on all nodes, enter the following SQL*Plus commands.
Detach OLD_HOME (where OLD_HOME is your pre-upgrade Oracle Home:
$OLD_HOME/oui/bin/runInstaller -detachHome -silent -local
Confirm OLD_HOME is removed from central inventory:
$OLD_HOME/OPatch/opatch lsinventory -all
Remove files in OLD_HOME manually:
/bin/rm -rf $OLD_HOME
From 11gR2 GI Install Guide
Before you start an Oracle RAC or Oracle Database installation on an Oracle Clusterware 11g release 11.2 installation, if you are upgrading from releases 11.1.0.7, 11.1.0.6, and 10.2.0.4, then Oracle recommends that you check for the latest recommended patches for the release you are upgrading from, and install those patches as needed on your existing database installations before upgrading.
For more information on recommended patches, refer to "Oracle Upgrade Companion," which is available through Note
785351.1.
The Cluster runs on RHEL 4
uname -rmi
2.6.9-89.ELsmp x86_64 x86_64
With vote and ocr files in raw devices
crsctl query css votedisk
0. 0 /dev/raw/raw2
$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 107612
Used space (kbytes) : 4604
Available space (kbytes) : 103008
ID : 972426829
Device/File Name : /dev/raw/raw1
Device/File integrity check succeeded
Device/File not configured
Cluster registry integrity check succeededThe pre-req checks are also similar to that on
11.1.0.7 to 11.2.0.3 checks, ntp check will fail if the ntp.conf file is present and ip_local_port_range will fail as the minimum value is different from 10gR2 to 11gR2. Noteably doesn't require any patches to be applied before the upgrade.
But unlike the ocr disk sharedness check that passed in 11gR1 the raw device sharedness check will fail.
Checking OCR integrity...
Check for compatible storage device for OCR location "/dev/raw/raw1"...
ERROR:
PRVF-10037 : Failed to retrieve storage type for "/dev/raw/raw1" on node "rac2"
Could not get the type of storage
ERROR:
PRVF-10037 : Failed to retrieve storage type for "/dev/raw/raw1" on node "rac1"
Could not get the type of storage
OCR integrity check failed
But this error could be ignored as per
Checklist for PRVF-10037 : Failed to retrieve storage type for xx on node xx [ID 1233505.1] by running the ocrcheck manually.
The full output from the cluvfy check is given below.
./runcluvfy.sh stage -pre crsinst -upgrade -n rac1,rac2 -rolling -src_crshome /opt/crs/oracle/product/10.2.0/crs -dest_crshome /opt/app/11.2.0/grid -dest_version 11.2.0.3.0 -fixup -fixupdir /home/oracle/fixupscript -verbose
Performing pre-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "rac1"
Destination Node Reachable?
------------------------------------ ------------------------
rac2 yes
rac1 yes
Result: Node reachability check passed from node "rac1"
Checking user equivalence...
Check: User equivalence for user "oracle"
Node Name Status
------------------------------------ ------------------------
rac2 passed
rac1 passed
Result: User equivalence check passed for user "oracle"
Checking CRS user consistency
Result: CRS user consistency check successful
Checking node connectivity...
Checking hosts config file...
Node Name Status
------------------------------------ ------------------------
rac2 passed
rac1 passed
Verification of the hosts config file successful
Interface information for node "rac2"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
eth0 192.168.0.86 192.168.0.0 0.0.0.0 192.168.0.100 08:00:27:90:0E:D3 1500
eth0 192.168.0.90 192.168.0.0 0.0.0.0 192.168.0.100 08:00:27:90:0E:D3 1500
eth1 192.168.0.88 192.168.0.0 0.0.0.0 192.168.0.100 08:00:27:C4:DE:3F 1500
Interface information for node "rac1"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
eth0 192.168.0.85 192.168.0.0 0.0.0.0 192.168.0.100 08:00:27:81:60:F7 1500
eth0 192.168.0.89 192.168.0.0 0.0.0.0 192.168.0.100 08:00:27:81:60:F7 1500
eth1 192.168.0.87 192.168.0.0 0.0.0.0 192.168.0.100 08:00:27:C9:FD:D3 1500
Check: Node connectivity for interface "eth0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
rac2[192.168.0.86] rac2[192.168.0.90] yes
rac2[192.168.0.86] rac2[192.168.0.88] yes
rac2[192.168.0.86] rac1[192.168.0.85] yes
rac2[192.168.0.86] rac1[192.168.0.89] yes
rac2[192.168.0.86] rac1[192.168.0.87] yes
rac2[192.168.0.90] rac2[192.168.0.88] yes
rac2[192.168.0.90] rac1[192.168.0.85] yes
rac2[192.168.0.90] rac1[192.168.0.89] yes
rac2[192.168.0.90] rac1[192.168.0.87] yes
rac2[192.168.0.88] rac1[192.168.0.85] yes
rac2[192.168.0.88] rac1[192.168.0.89] yes
rac2[192.168.0.88] rac1[192.168.0.87] yes
rac1[192.168.0.85] rac1[192.168.0.89] yes
rac1[192.168.0.85] rac1[192.168.0.87] yes
rac1[192.168.0.89] rac1[192.168.0.87] yes
Result: Node connectivity passed for interface "eth0"
Check: TCP connectivity of subnet "192.168.0.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
rac1:192.168.0.85 rac2:192.168.0.86 passed
rac1:192.168.0.85 rac2:192.168.0.90 passed
rac1:192.168.0.85 rac2:192.168.0.88 passed
rac1:192.168.0.85 rac1:192.168.0.89 passed
rac1:192.168.0.85 rac1:192.168.0.87 passed
Result: TCP connectivity check passed for subnet "192.168.0.0"
Check: Node connectivity for interface "eth1"
Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.0.0".
Subnet mask consistency check passed.
Result: Node connectivity check passed
Checking multicast communication...
Checking subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0"...
Check of subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0" passed.
Check of multicast communication passed.
Checking OCR integrity...
Check for compatible storage device for OCR location "/dev/raw/raw1"...
ERROR:
PRVF-10037 : Failed to retrieve storage type for "/dev/raw/raw1" on node "rac2"
Could not get the type of storage
ERROR:
PRVF-10037 : Failed to retrieve storage type for "/dev/raw/raw1" on node "rac1"
Could not get the type of storage
OCR integrity check failed
Checking ASMLib configuration.
Node Name Status
------------------------------------ ------------------------
rac2 passed
rac1 passed
Result: Check for ASMLib configuration passed.
Check: Total memory
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 1.9609GB (2056180.0KB) 1.5GB (1572864.0KB) passed
rac1 1.9609GB (2056180.0KB) 1.5GB (1572864.0KB) passed
Result: Total memory check passed
Check: Available memory
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 1.326GB (1390364.0KB) 50MB (51200.0KB) passed
rac1 1.1387GB (1194000.0KB) 50MB (51200.0KB) passed
Result: Available memory check passed
Check: Swap space
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 3.9987GB (4192956.0KB) 2.9414GB (3084270.0KB) passed
rac1 3.9987GB (4192956.0KB) 2.9414GB (3084270.0KB) passed
Result: Swap space check passed
Check: Free disk space for "rac2:/opt/app/11.2.0/grid,rac2:/tmp"
Path Node Name Mount point Available Required Status
---------------- ------------ ------------ ------------ ------------ ------------
/opt/app/11.2.0/grid rac2 / 13.3779GB 7.5GB passed
/tmp rac2 / 13.3779GB 7.5GB passed
Result: Free disk space check passed for "rac2:/opt/app/11.2.0/grid,rac2:/tmp"
Check: Free disk space for "rac1:/opt/app/11.2.0/grid,rac1:/tmp"
Path Node Name Mount point Available Required Status
---------------- ------------ ------------ ------------ ------------ ------------
/opt/app/11.2.0/grid rac1 / 11.3978GB 7.5GB passed
/tmp rac1 / 11.3978GB 7.5GB passed
Result: Free disk space check passed for "rac1:/opt/app/11.2.0/grid,rac1:/tmp"
Check: User existence for "oracle"
Node Name Status Comment
------------ ------------------------ ------------------------
rac2 passed exists(500)
rac1 passed exists(500)
Checking for multiple users with UID value 500
Result: Check for multiple users with UID value 500 passed
Result: User existence check passed for "oracle"
Check: Group existence for "oinstall"
Node Name Status Comment
------------ ------------------------ ------------------------
rac2 passed exists
rac1 passed exists
Result: Group existence check passed for "oinstall"
Check: Membership of user "oracle" in group "oinstall" [as Primary]
Node Name User Exists Group Exists User in Group Primary Status
---------------- ------------ ------------ ------------ ------------ ------------
rac2 yes yes yes yes passed
rac1 yes yes yes yes passed
Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed
Check: Run level
Node Name run level Required Status
------------ ------------------------ ------------------------ ----------
rac2 3 3,5 passed
rac1 3 3,5 passed
Result: Run level check passed
Check: Hard limits for "maximum open file descriptors"
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
rac2 hard 65536 65536 passed
rac1 hard 65536 65536 passed
Result: Hard limits check passed for "maximum open file descriptors"
Check: Soft limits for "maximum open file descriptors"
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
rac2 soft 1024 1024 passed
rac1 soft 1024 1024 passed
Result: Soft limits check passed for "maximum open file descriptors"
Check: Hard limits for "maximum user processes"
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
rac2 hard 16384 16384 passed
rac1 hard 16384 16384 passed
Result: Hard limits check passed for "maximum user processes"
Check: Soft limits for "maximum user processes"
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
rac2 soft 2047 2047 passed
rac1 soft 2047 2047 passed
Result: Soft limits check passed for "maximum user processes"
There are no oracle patches required for home "/opt/crs/oracle/product/10.2.0/crs".
There are no oracle patches required for home "/opt/app/11.2.0/grid".
Check: System architecture
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 x86_64 x86_64 passed
rac1 x86_64 x86_64 passed
Result: System architecture check passed
Check: Kernel version
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 2.6.9-89.ELsmp 2.6.9 passed
rac1 2.6.9-89.ELsmp 2.6.9 passed
Result: Kernel version check passed
Check: Kernel parameter for "semmsl"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 250 250 250 passed
rac1 250 250 250 passed
Result: Kernel parameter check passed for "semmsl"
Check: Kernel parameter for "semmns"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 32000 32000 32000 passed
rac1 32000 32000 32000 passed
Result: Kernel parameter check passed for "semmns"
Check: Kernel parameter for "semopm"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 100 100 100 passed
rac1 100 100 100 passed
Result: Kernel parameter check passed for "semopm"
Check: Kernel parameter for "semmni"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 128 128 128 passed
rac1 128 128 128 passed
Result: Kernel parameter check passed for "semmni"
Check: Kernel parameter for "shmmax"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 1052764160 1052764160 1052764160 passed
rac1 1052764160 1052764160 1052764160 passed
Result: Kernel parameter check passed for "shmmax"
Check: Kernel parameter for "shmmni"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 4096 4096 4096 passed
rac1 4096 4096 4096 passed
Result: Kernel parameter check passed for "shmmni"
Check: Kernel parameter for "shmall"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 2097152 2097152 2097152 passed
rac1 2097152 2097152 2097152 passed
Result: Kernel parameter check passed for "shmall"
Check: Kernel parameter for "file-max"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 6815744 6815744 6815744 passed
rac1 6815744 6815744 6815744 passed
Result: Kernel parameter check passed for "file-max"
Check: Kernel parameter for "ip_local_port_range"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed
rac1 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed
Result: Kernel parameter check passed for "ip_local_port_range"
Check: Kernel parameter for "rmem_default"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 1048576 1048576 262144 passed
rac1 1048576 1048576 262144 passed
Result: Kernel parameter check passed for "rmem_default"
Check: Kernel parameter for "rmem_max"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 4194304 4194304 4194304 passed
rac1 4194304 4194304 4194304 passed
Result: Kernel parameter check passed for "rmem_max"
Check: Kernel parameter for "wmem_default"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 262144 262144 262144 passed
rac1 262144 262144 262144 passed
Result: Kernel parameter check passed for "wmem_default"
Check: Kernel parameter for "wmem_max"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 1048576 1048576 1048576 passed
rac1 1048576 1048576 1048576 passed
Result: Kernel parameter check passed for "wmem_max"
Check: Kernel parameter for "aio-max-nr"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rac2 1048576 1048576 1048576 passed
rac1 1048576 1048576 1048576 passed
Result: Kernel parameter check passed for "aio-max-nr"
Check: Package existence for "make"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 make-3.80-7.EL4 make-3.80 passed
rac1 make-3.80-7.EL4 make-3.80 passed
Result: Package existence check passed for "make"
Check: Package existence for "binutils"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 binutils-2.15.92.0.2-25 binutils-2.15.92.0.2 passed
rac1 binutils-2.15.92.0.2-25 binutils-2.15.92.0.2 passed
Result: Package existence check passed for "binutils"
Check: Package existence for "gcc(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 gcc(x86_64)-3.4.6-11 gcc(x86_64)-3.4.6 passed
rac1 gcc(x86_64)-3.4.6-11 gcc(x86_64)-3.4.6 passed
Result: Package existence check passed for "gcc(x86_64)"
Check: Package existence for "libaio(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 libaio(x86_64)-0.3.105-2 libaio(x86_64)-0.3.105 passed
rac1 libaio(x86_64)-0.3.105-2 libaio(x86_64)-0.3.105 passed
Result: Package existence check passed for "libaio(x86_64)"
Check: Package existence for "glibc(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 glibc(x86_64)-2.3.4-2.43 glibc(x86_64)-2.3.4-2.41 passed
rac1 glibc(x86_64)-2.3.4-2.43 glibc(x86_64)-2.3.4-2.41 passed
Result: Package existence check passed for "glibc(x86_64)"
Check: Package existence for "compat-libstdc++-33(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 compat-libstdc++-33(x86_64)-3.2.3-47.3 compat-libstdc++-33(x86_64)-3.2.3 passed
rac1 compat-libstdc++-33(x86_64)-3.2.3-47.3 compat-libstdc++-33(x86_64)-3.2.3 passed
Result: Package existence check passed for "compat-libstdc++-33(x86_64)"
Check: Package existence for "elfutils-libelf(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 elfutils-libelf(x86_64)-0.97.1-5 elfutils-libelf(x86_64)-0.97 passed
rac1 elfutils-libelf(x86_64)-0.97.1-5 elfutils-libelf(x86_64)-0.97 passed
Result: Package existence check passed for "elfutils-libelf(x86_64)"
Check: Package existence for "elfutils-libelf-devel"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 elfutils-libelf-devel-0.97.1-5 elfutils-libelf-devel-0.97 passed
rac1 elfutils-libelf-devel-0.97.1-5 elfutils-libelf-devel-0.97 passed
Result: Package existence check passed for "elfutils-libelf-devel"
Check: Package existence for "glibc-common"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 glibc-common-2.3.4-2.43 glibc-common-2.3.4 passed
rac1 glibc-common-2.3.4-2.43 glibc-common-2.3.4 passed
Result: Package existence check passed for "glibc-common"
Check: Package existence for "glibc-devel(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 glibc-devel(x86_64)-2.3.4-2.43 glibc-devel(x86_64)-2.3.4 passed
rac1 glibc-devel(x86_64)-2.3.4-2.43 glibc-devel(x86_64)-2.3.4 passed
Result: Package existence check passed for "glibc-devel(x86_64)"
Check: Package existence for "glibc-headers"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 glibc-headers-2.3.4-2.43 glibc-headers-2.3.4 passed
rac1 glibc-headers-2.3.4-2.43 glibc-headers-2.3.4 passed
Result: Package existence check passed for "glibc-headers"
Check: Package existence for "gcc-c++(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 gcc-c++(x86_64)-3.4.6-11 gcc-c++(x86_64)-3.4.6 passed
rac1 gcc-c++(x86_64)-3.4.6-11 gcc-c++(x86_64)-3.4.6 passed
Result: Package existence check passed for "gcc-c++(x86_64)"
Check: Package existence for "libaio-devel(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 libaio-devel(x86_64)-0.3.105-2 libaio-devel(x86_64)-0.3.105 passed
rac1 libaio-devel(x86_64)-0.3.105-2 libaio-devel(x86_64)-0.3.105 passed
Result: Package existence check passed for "libaio-devel(x86_64)"
Check: Package existence for "libgcc(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 libgcc(x86_64)-3.4.6-11 libgcc(x86_64)-3.4.6 passed
rac1 libgcc(x86_64)-3.4.6-11 libgcc(x86_64)-3.4.6 passed
Result: Package existence check passed for "libgcc(x86_64)"
Check: Package existence for "libstdc++(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 libstdc++(x86_64)-3.4.6-11 libstdc++(x86_64)-3.4.6 passed
rac1 libstdc++(x86_64)-3.4.6-11 libstdc++(x86_64)-3.4.6 passed
Result: Package existence check passed for "libstdc++(x86_64)"
Check: Package existence for "libstdc++-devel(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 libstdc++-devel(x86_64)-3.4.6-11 libstdc++-devel(x86_64)-3.4.6 passed
rac1 libstdc++-devel(x86_64)-3.4.6-11 libstdc++-devel(x86_64)-3.4.6 passed
Result: Package existence check passed for "libstdc++-devel(x86_64)"
Check: Package existence for "sysstat"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 sysstat-5.0.5-25.el4 sysstat-5.0.5 passed
rac1 sysstat-5.0.5-25.el4 sysstat-5.0.5 passed
Result: Package existence check passed for "sysstat"
Check: Package existence for "pdksh"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 pdksh-5.2.14-30.6 pdksh-5.2.14 passed
rac1 pdksh-5.2.14-30.6 pdksh-5.2.14 passed
Result: Package existence check passed for "pdksh"
Check: Package existence for "expat(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rac2 expat(x86_64)-1.95.7-4 expat(x86_64)-1.95.7 passed
rac1 expat(x86_64)-1.95.7-4 expat(x86_64)-1.95.7 passed
Result: Package existence check passed for "expat(x86_64)"
Checking for multiple users with UID value 0
Result: Check for multiple users with UID value 0 passed
Check: Current group ID
Result: Current group ID check passed
Starting check for consistency of primary group of root user
Node Name Status
------------------------------------ ------------------------
rac2 passed
rac1 passed
Check for consistency of root user's primary group passed
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes
No NTP Daemons or Services were found to be running
Result: Clock synchronization check using Network Time Protocol(NTP) passed
Checking Core file name pattern consistency...
Core file name pattern consistency check passed.
Checking to make sure user "oracle" is not in "root" group
Node Name Status Comment
------------ ------------------------ ------------------------
rac2 passed does not exist
rac1 passed does not exist
Result: User "oracle" is not part of "root" group. Check passed
Check default user file creation mask
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
rac2 0022 0022 passed
rac1 0022 0022 passed
Result: Default user file creation mask check passed
Checking consistency of file "/etc/resolv.conf" across nodes
Checking the file "/etc/resolv.conf" to make sure only one of domain and search entries is defined
File "/etc/resolv.conf" does not have both domain and search entries defined
Checking if domain entry in file "/etc/resolv.conf" is consistent across the nodes...
domain entry in file "/etc/resolv.conf" is consistent across nodes
Checking if search entry in file "/etc/resolv.conf" is consistent across the nodes...
search entry in file "/etc/resolv.conf" is consistent across nodes
Checking DNS response time for an unreachable node
Node Name Status
------------------------------------ ------------------------
rac2 passed
rac1 passed
The DNS response time for an unreachable node is within acceptable limit on all nodes
File "/etc/resolv.conf" is consistent across nodes
UDev attributes check for OCR locations started...
Result: UDev attributes check passed for OCR locations
UDev attributes check for Voting Disk locations started...
Result: UDev attributes check passed for Voting Disk locations
Check: Time zone consistency
Result: Time zone consistency check passed
Checking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.
Checking Oracle Cluster Voting Disk configuration...
Oracle Cluster Voting Disk configuration check passed
Clusterware version consistency passed
Pre-check for cluster services setup was unsuccessful on all the nodes.
Pre-check is unsuccessful because of the above mentioned check.
Configure a SCAN IP to be used with 11gR2 GI upgrade.
Create the new asm admin groups.
groupadd asmadmin
groupadd asmdba
groupadd asmoper
Modify the Oracle user from
id oracle
uid=501(oracle) gid=501(oinstall) groups=501(oinstall),502(dba),503(oper)
To
usermod -g oinstall -G dba,oper,asmdba,asmoper,asmadmin oracle
id oracle
uid=501(oracle) gid=501(oinstall) groups=501(oinstall),502(dba),503(oper),504(asmadmin),505(asmdba),506(asmoper)
Start the clusterware upgrade while the cluster stack is up
crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application ONLINE ONLINE rac1
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora.rac10g2.db application ONLINE ONLINE rac1
ora....21.inst application ONLINE ONLINE rac1
ora....22.inst application ONLINE ONLINE rac2
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application ONLINE ONLINE rac2
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip application ONLINE ONLINE rac2
Execute runInstaller
Select all nodes and ASM upgrade.
It is not possible to do a rolling ASM upgrade with 10gR2. When it is time to do a ASM upgrade all database instance will be shutdown. But the GI install and configuration will happen in a rolling manner.
Specify the new scan ip
Password for less privilege ASMSNMP user. Used to monitor ASM.
Specify new ASM admin groups
New location for GI used in the out-of-place upgrade.
Manually verify the failed check by checking the ignoreall check box.
Upgrade Summary
Execute rootupgrade.sh script one node at a time. Database and ASM will be brought down and back up again once the script has finished executing.
Running rootupgrade.sh on first node (rac1)
# /opt/app/11.2.0/grid/rootupgrade.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /opt/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /opt/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
OLR initialization - successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Replacing Clusterware entries in inittab
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Configure Oracle Grid Infrastructure for a Cluster ... succeededSoftware version is upgrade but active version remain 10.2.0.5 until all nodes are upgraded
crsctl query crs activeversion
CRS active version on the cluster is [10.2.0.5.0]
crsctl query crs softwareversion
CRS software version on node [rac1] is [11.2.0.3.0]
Running rootupgrade.sh on second node (rac2)
# /opt/app/11.2.0/grid/rootupgrade.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /opt/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /opt/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
OLR initialization - successful
Replacing Clusterware entries in inittab
clscfg: EXISTING configuration version 5 detected.
clscfg: version 5 is 11g Release 2.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Start upgrade invoked..
Started to upgrade the Oracle Clusterware. This operation may take a few minutes.
Started to upgrade the OCR.
Started to upgrade the CSS.
Started to upgrade the CRS.
The CRS was successfully upgraded.
Oracle Clusterware operating version was successfully set to 11.2.0.3.0
Preparing packages for installation...
cvuqdisk-1.0.9-1
Configure Oracle Grid Infrastructure for a Cluster ... succeededActive version is upgraded to 11.2.0.3
crsctl query crs activeversion
CRS active version on the cluster is [11.2.0.3.0]
When the OK button is clicked on Execute Configuration Script dialog other configuration tasks starts. During ASM upgrade all database instances will be brought down automatically and once ASM is upgraded will be brought up.
This conclude the clusterware upgrade to grid infrastructure and GI will be using the raw devices for ocr and vote disk.
crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE a3b3d5071afb7fc0ffb0f7bc4a90fd43 (/dev/raw/raw2) []
Located 1 voting disk(s).
ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 103596
Used space (kbytes) : 6032
Available space (kbytes) : 97564
ID : 972426829
Device/File Name : /dev/raw/raw1
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeededSome resource types will have auto start option set never. These could be changed as required
Resource Name Type Auto Start State
----------- ------ ----------------
ora.DATA.dg ora.diskgroup.type never
ora.FLASH.dg ora.diskgroup.type never
ora.LISTENER.lsnr ora.listener.type restore
ora.LISTENER_SCAN1.lsnr ora.scan_listener.type restore
ora.asm ora.asm.type never
ora.cvu ora.cvu.type restore
ora.gsd ora.gsd.type always
ora.net1.network ora.network.type restore
ora.oc4j ora.oc4j.type restore
ora.ons ora.ons.type always
ora.rac1.vip ora.cluster_vip_net1.type restore
ora.rac10g2.db application restore
ora.rac10g2.rac10g21.inst application 1
ora.rac10g2.rac10g22.inst application 1
ora.rac2.vip ora.cluster_vip_net1.type restore
ora.scan1.vip ora.scan_vip.type restore
Script to get the above output is available in
11.1.0.7 to 11.2.0.3 post and how to modify the option available
here.
Next step is to upgrade the RAC software and upgrade the database. As per 1358166.1 there's no pre-patch required if the timezone is 4 on 10gR2 but it's advised to upgrade the timezone to 14 after the upgrade. This could be done while upgrading the database. Current timezone is 4
SQL> SELECT version FROM v$timezone_file;
VERSION
----------
4Execute runInstaller from database software location.
Select all nodes part of the RAC
New location for out-of-place upgrade.
OS groups for database administration.
Summary
Execute root.sh on all nodes
End of root.sh and clicking OK button will result in start of DBUA. All but one instance will be brought down, upgrade will continue on the instance that is up and once completed all instance will be brought up.
Upgrade timezone during the database upgrade
Pre-upgrade summary
Upgrade result
Conclude the RAC software and database upgrade
Similar to 11.1.0.7 to 11.2.0.3 remote listener entry will have both tnsnames.ora entries from 10gR2 and scan ip
remote_listener string LISTENERS_RAC10G2, rac-scan:1521
Even though 10gR2 RAC had Jan 2012 PSU applied and _external_scn_rejection_threshold_hours set it was commented during upgrade and was not present after the upgrade.
Create an
application service for FCF.
Useful metalink notes
Checklist for PRVF-10037 : Failed to retrieve storage type for xx on node xx
[ID 1233505.1]
ASM upgrade failure from 10.2.0.4 to 11.2.0.2
[ID 1336812.1]
Alternative Way To Upgrade An ASM RAC Configuration From Release 10.2 Or 11.1 To Release 11.2
[ID 1422415.1]
Actions For DST Updates When Upgrading To Or Applying The 11.2.0.3 Patchset
[ID 1358166.1]
Things to Consider Before Upgrading to 11.2.0.3 to Avoid Poor Performance or Wrong Results
[ID 1392633.1]
Oracle 11gR2 Upgrade Companion
[ID 785351.1]
