Tuesday, March 14, 2023

Unplug-Plug-Upgrade from 19c to 21c Using Autoupgrade

This post shows the steps for upgrading a 19c PDB with autoupgrade tool using the unplug-plug-uprade method. The 21c CDB is called cdb21c and 19c CDB is called testcdb. The PDB that will be unpluged from 19c and plugged into 21c and upgraded is called testpdb2. Both 19c and 21c CDBs reside in the same Oracle restart configuration and the PDB has service dbxrw associated with it.
Resource Name             Type                      Target             State              Host
-------------             ------                    -------            --------           ----------
ora.cdb21c.db             ora.database.type         ONLINE             ONLINE             ip-172-31-10-91
ora.testcdb.db            ora.database.type         ONLINE             ONLINE             ip-172-31-10-91
ora.testcdb.dbxrw.svc     ora.service.type          ONLINE             ONLINE             ip-172-31-10-91

If any privileges were reovked from public due to security reason (CIS standard recommendations) then grant those privileges back before starting the ugprade process. Once the upgrade is completed then those privilges could be revoked again.
Generate a sample upgrade config file with
java -jar autoupgrade.jar -create_sample_file config sample_config.cfg
and customize the "Unplug/Plug upgrade" section.
global.autoupg_log_dir=/home/oracle/upgradelogs
global.keystore=/home/oracle/upgtde

#
# Database number 1 - Unplug/Plug upgrade
#
upg1.log_dir=/home/oracle/autoupgrade/cdb21c
upg1.sid=testcdb
upg1.source_home=/opt/app/oracle/product/19.x.0/dbhome_2
upg1.target_cdb=cdb21c
upg1.target_home=/opt/app/oracle/product/21.x.0/dbhome_1
upg1.pdbs=testpdb2                    # Comma delimited list of pdb names that will be upgraded and moved to the target CDB
upg1.run_utlrp=yes                   # Optional. Whether or not to run utlrp after upgrade
upg1.timezone_upg=yes                # Optional. Whether or not to run the timezone upgrade
upg1.target_version=21       # Oracle version of the target ORACLE_HOME.  Only required when the target Oracle database version is 12.2


# copies the pdb to inside target cdb directory structure. default is nocopy where datafiles remains in 19c cdb directory strcuture in ASM. 
upg1.target_pdb_copy_option=file_name_convert=NONE
global.keystore is used to give the location for the autoupgrade key store. Since both target and source CDBs use TDE, it helps to create a autoupgrade keystore and save the respective CDBs key store passowrd in it.
Secondly the "upg1.target_pdb_copy_option=file_name_convert=NONE" copies the data files inside the target CDB datafile directory structure. If not the PDB datafile will remain where they are (under the source CDB's directory structure).
To create autoupgrade keystore run the autoupgrade with load_password option.
java -jar autoupgrade.jar -config unplug_plug.cfg -load_password
Processing config file ...

Starting AutoUpgrade Password Loader - Type help for available options
Creating new AutoUpgrade keystore - Password required
Enter password:
Enter password again:
AutoUpgrade keystore was successfully created

TDE> list
+----------+----------------+------------------+-----------+------------------+
|ORACLE_SID| Action Required|      TDE Password|SEPS Status|Active Wallet Type|
+----------+----------------+------------------+-----------+------------------+
|    cdb21c|Add TDE password|No password loaded|   Inactive|               Any|
|   testcdb|Add TDE password|No password loaded|   Inactive|               Any|
+----------+----------------+------------------+-----------+------------------+
It detects the databases that are up and running in the host and could be listed with list command. Next add the each database's keystore password to autoupgrade keystore.
TDE> add testcdb
Enter your secret/Password:
Re-enter your secret/Password:
TDE> add cdb21c
Enter your secret/Password:
Re-enter your secret/Password:
TDE> list
+----------+---------------+------------+-----------+------------------+
|ORACLE_SID|Action Required|TDE Password|SEPS Status|Active Wallet Type|
+----------+---------------+------------+-----------+------------------+
|    cdb21c|               |    Verified|   Inactive|               Any|
|   testcdb|               |    Verified|   Inactive|               Any|
+----------+---------------+------------+-----------+------------------+
TDE> save
Convert the AutoUpgrade keystore to auto-login [YES|NO] ? YES
It helps to have a auto-login keystore so the upgrade could run smoothly without waiting for password to be entered.
Next run the preuprage checks. This is done using the analyze option.
java -jar autoupgrade.jar -config unplug_plug.cfg -mode analyze
AutoUpgrade 22.4.220712 launched with default internal options
Processing config file ...
Loading AutoUpgrade keystore
AutoUpgrade keystore was successfully loaded
+--------------------------------+
| Starting AutoUpgrade execution |
+--------------------------------+
1 PDB(s) will be analyzed
Type 'help' to list console commands
upg> lsj -a 10
upg> +----+-------+---------+---------+-------+----------+-------+----------------------------+
|Job#|DB_NAME|    STAGE|OPERATION| STATUS|START_TIME|UPDATED|                     MESSAGE|
+----+-------+---------+---------+-------+----------+-------+----------------------------+
| 100|testcdb|PRECHECKS|EXECUTING|RUNNING|  13:58:20| 7s ago|Loading database information|
+----+-------+---------+---------+-------+----------+-------+----------------------------+
Total jobs 1

The command lsj is running every 10 seconds. PRESS ENTER TO EXIT
Job 100 completed
------------------- Final Summary --------------------
Number of databases            [ 1 ]

Jobs finished                  [1]
Jobs failed                    [0]

Please check the summary report at:
/home/oracle/upgradelogs/cfgtoollogs/upgrade/auto/status/status.html
/home/oracle/upgradelogs/cfgtoollogs/upgrade/auto/status/status.log
Check the status html for any of the precheck failed. In this case there was no failures.

If any fixup are needed then run the autoupgrade with fixup option.
java -jar autoupgrade.jar -config unplug_plug.cfg -mode fixups
This will generate a fixup report.



Finally run the autoupgrade with deploy option to begin the upgrade.
java -jar autoupgrade.jar -config unplug_plug.cfg -mode deploy
When this is run on the source CDB alert log the PDB unplug could noticed.
2023-03-09T14:13:16.820993+00:00
Completed: alter pluggable database TESTPDB2 unplug into '/home/oracle/upgtde/testcdb-TESTPDB2.xml' encrypt using *
drop pluggable database TESTPDB2 keep datafiles
2023-03-09T14:13:17.607648+00:00
Deleted Oracle managed file +DATA/TESTCDB/9CBA2DF91A8C7012E053F4071FAC36E9/TEMPFILE/temp.329.1131016147
2023-03-09T14:13:17.635197+00:00
Stopped service testpdb2
Completed: drop pluggable database TESTPDB2 keep datafiles
On the target CDB's alert log the PDB plug-ing couldbe noticed.
2023-03-09T14:13:17.933191+00:00
create pluggable database "TESTPDB2"   using '/home/oracle/upgtde/testcdb-TESTPDB2.xml' COPY file_name_convert=NONE tempfile reuse keystore identified by * decrypt using *
Monitor the upgrade process and once completed check the status.html that all tasks ran without any issue.

The service associated with the PDB is not migrated as part of the upgrade. This will remain in the offline status.
Resource Name             Type                      Target             State              Host
-------------             ------                    -------            --------           ----------
ora.cdb21c.testpdb2.pdb   ora.pdb.type              ONLINE             ONLINE             ip-172-31-10-91
ora.testcdb.db            ora.database.type         ONLINE             ONLINE             ip-172-31-10-91
ora.testcdb.dbxrw.svc     ora.service.type          OFFLINE            OFFLINE
Oracle confirmed via SR that migration of service is not available with autoupgrade. So this has to be done manually. Remove the old service using the 19c Oracle Home's srvctl
srvctl remove service -db $ORACLE_SID -service dbxrw
Add the service again using 21c Oracle Home's srvctl.
srvctl add service -db $ORACLE_SID -pdb testpdb2 -service dbxrw -role PRIMARY -notification TRUE -failovertype NONE -failovermethod NONE -failoverdelay 0 -failoverretry 0 -drain_timeout 5 -stopoption IMMEDIATE
With this both PDB and service are now part of the target CDB.
Resource Name             Type                      Target             State              Host
-------------             ------                    -------            --------           ----------
ora.cdb21c.db             ora.database.type         ONLINE             ONLINE             ip-172-31-10-91
ora.cdb21c.dbxrw.svc      ora.service.type          ONLINE             ONLINE             ip-172-31-10-91
ora.cdb21c.testpdb2.pdb   ora.pdb.type              ONLINE             ONLINE             ip-172-31-10-91
Check the PDB data files reside inside the target CDB directory path.
    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 TESTPDB2                       READ WRITE NO

SQL>  select name from v$datafile where con_id=3;

NAME
----------------------------------------------------------------------------------------------------
+DATA/CDB21C/9CBA2DF91A8C7012E053F4071FAC36E9/DATAFILE/system.332.1131027199
+DATA/CDB21C/9CBA2DF91A8C7012E053F4071FAC36E9/DATAFILE/sysaux.326.1131027199
+DATA/CDB21C/9CBA2DF91A8C7012E053F4071FAC36E9/DATAFILE/undotbs1.280.1131027199
This conclude the upgrade of PDB using unplug-plug-upgrade method.