Friday, April 24, 2015

Upgrading RAC from to - Database

After upgrading grid infrastructure from to the next step is the upgrade of RAC software and the database itself. There are earlier post which upgraded to and also upgraded in data guard configuration and with PDBs. These posts have useful metalink notes that could be beneficial in RAC environment as well. The database software upgrade is an out of place upgrade with role separation. Use orachk or cluvfy to check the pre upgrade status.
cluvfy stage  -pre dbinst -upgrade -src_dbhome /opt/app/oracle/product/11.2.0/dbhome_1 -dest_dbhome /opt/app/oracle/product/12.1.0/dbhome_2 -dest_version
Create additional user groups (backupdba,kmdba,dgdba) and install the using the software only installation option out of place to current oracle home. One difference that was noted is that with there's no need to manually correct the oracle binary's ownership as before. After the database installation the oracle binaries will have ownership as oracel:oinstall even though the correct ownership should be (for role separated installation) oracle:asmadmin. However this get auto rectified when the DBUA is run.
Before running the DBUA check oracle user has the write permission on the audit folder under $ORACLE_BASE if not grant write permission for oinstall ggroup.
chmod 770 $ORACLE_BASE/audit
The default listener name has been changed on this cluster setup. This missing default listener causes a problem similar to earlier upgrade (from to
To fix this add default listener and start it before DBUA is run.
$ srvctl add listener -listener listener -p 1521
$ srvctl start listener -l listener
To avoid the bug 19518079 apply the patch to 12c oracle home before DBUA is run. However this is not a must and could use other workarounds as well. For more information refere MOS 19518079.8 and 1941383.1
Run preupgrade script to identify any pre-reqs tasks that must be done on the database before the upgrade.
cd /opt/app/oracle/product/12.1.0/dbhome_2/rdbms/admin
SQL> @preupgrd.sql

Run the DBUA to start the database upgrade.
Timezone is also be upgraded at the same time the database.
Upgrade tasks completion
Upgrade results
Check time timezone information is same on both timezone file and database registry as there was a mismatch between these values in after upgrading from
SQL> select * from v$timezone_file;

FILENAME                VERSION     CON_ID
-------------------- ---------- ----------
timezlrg_18.dat              18          0

SQL> select TZ_VERSION from registry$database;

If ASM configuration is referring the default listener modify it before removing the default listener entry added during the upgrade.
[grid@rhel6m2 ~]$ srvctl config asm
ASM home: <CRS home>
Password file:
ASM listener: LISTENER

$ srvctl modify asm -l MYLISTENER
$ srvctl config asm
ASM home: <CRS home>
Password file:

$ srvctl stop listener -l listener
$ srvctl remove listener -l listener
If database patches are being applied make sure sqlpatch directory has write permission for oracle user.
chmod 775 /opt/app/oracle/cfgtoollogs/sqlpatch
If the database upgrade is satisfactory upgrade the compatible parameter to new version.
alter system set compatible='' scope=spfile sid='*';
ASM diskgroup compatible parameters could also be upgrade at this time. There are dependencies between the DB compatibility value and ASM's rdbms compatibility value as such consider in which order to update these values.
alter diskgroup data set attribute 'compatible.asm'='';
alter diskgroup flash set attribute 'compatible.asm'='';
alter diskgroup flash set attribute 'compatible.rdbms'='';
alter diskgroup data set attribute 'compatible.rdbms'='';
alter diskgroup cluster_dg set attribute 'compatible.rdbms'='';
alter diskgroup cluster_dg set attribute 'compatible.asm'='';
alter diskgroup flash set attribute 'compatible.advm'='';
Verify that upgrade entry is added to patch history.
ACTION_TIME                    ACTION           VERSION            COMMENTS
------------------------------ ---------------- ------------------ --------------------------
11-JUN-12 PM   APPLY             Patchset
12-JUN-12 PM   APPLY             PSU
19-JUL-12 PM   APPLY             PSU
15-NOV-12 PM   APPLY             PSU
22-JAN-13 PM   APPLY             PSU
24-APR-13 AM   APPLY             PSU
31-JUL-13 PM   APPLY             PSU
09-SEP-13 PM   VIEW INVALIDATE                     view invalidation
09-SEP-13 PM   UPGRADE         Upgraded from
09-SEP-13 PM   APPLY             Patchset
08-MAY-14 PM   APPLY             PSU
25-JUL-14 PM   APPLY             PSU
17-NOV-14 PM   APPLY             PSU
22-JAN-15 PM   jvmpsu.sql   RAN jvmpsu.sql
22-JAN-15 PM   APPLY                               Patch 19877440 applied
22-JAN-15 PM   APPLY     OJVM PSU post-install
22-JAN-15 PM   APPLY             PSU
22-APR-15 PM   VIEW INVALIDATE                     view invalidation
22-APR-15 PM   UPGRADE         Upgraded from
22-APR-15 PM   jvmpsu.sql   RAN jvmpsu.sql
This conclude the upgrade of RAC DB from to

Useful metalink notes
Oracle Database Upgrade Path Reference List [ID 730365.1]
Oracle 12cR1 Upgrade Companion [ID 1462240.1]
Memory Guard Offlines Database Service Due to Low Free Memory [ID 1929994.1]
DBA_REGISTRY_HISTORY Does Not Get Updated on Database Upgraded to [ID 1941383.1]
Bug 19518079 - DBA_REGISTRY_HISTORY is not updated during upgrade to [ID 19518079.8]
ORA-600 [kwqitnmphe:ltbagi] or ORA-600 [kwqdlprochstentry:ltbagi] on the SYS$SERVICE_METRICS_TAB, Orphan Subscribers [ID 1986078.1]
Master Note For Oracle Database 12c Release 1 (12.1) Database/Client Installation/Upgrade/Migration Standalone Environment (Non-RAC) [ID 1520299.1]
Actions For DST Updates When Upgrading To Or Applying The Patchset [ID 1665676.1]
Things to Consider Before Upgrading to to Avoid Poor Performance or Wrong Results [ID 2034610.1]

Related Posts
Upgrading (11gR2) Database to (12c) Using DBUA
Upgrading from to RAC
Upgrade Oracle Database 12c1 from to
Upgrading 12c CDB and PDB from to