Patching OS in DBCS VM DB Using DBCLI

There's an earlier post which shows updating and patching the OS. This method used yum to update the OS. It is still valid and available in documentation as well.
Recently Oracle introduced a new way to patch the OS, that is using the dbcli. The dbcli is the tool to use if patching the DB is done using command line. With this new enhancement same tool dbcli could be used to patching and updating the OS as well.
1. To check if any OS updates are available run the get-availableospatches command. If patches are not available updateAvailable will show false.

#  dbcli get-availableospatches -j
{
  "updateAvailable" : false,
  "rebootIsRequired" : false,
  "updateableRpms" : null,
  "installedRpms" : null
}

2. If updates are available then it will show updateAvailable true and also if a reboot is required after update.

dbcli get-availableospatches
Update Available     Reboot Required
-------------------- --------------------
Yes                  Yes

To get a list of rpms update use -j option.

bcli get-availableospatches -j
{
  "updateAvailable" : true,
  "rebootIsRequired" : true,
  "updateableRpms" : [ "bind.x86_64::32:9.11.4-26.P2.el7_9.5", "bind-export-libs.x86_64::32:9.11.4-26.P2.el7_9.5", "bind-libs.x86_64::32:9.11.4-26.P2.el7_9.5", "bind-libs-lite.x86_64::32:9.11.4-26.P2.el7_9.5", "bind-license.noarch::32:  9.11.4-26.P2.el7_9.5", "bind-utils.x86_64::32:9.11.4-26.P2.el7_9.5", "curl.x86_64::7.29.0-59.0.3.el7_9.1", "device-mapper.x86_64::7:1.02.170-6.0.5.el7_9.5", "device-mapper-event.x86_64::7:1.02.170-6.0.5.el7_9.5", "device-mapper-event-l  ibs.x86_64::7:1.02.170-6.0.5.el7_9.5", "device-mapper-libs.x86_64::7:1.02.170-6.0.5.el7_9.5", "device-mapper-multipath.x86_64::0.4.9-134.0.3.el7_9", "device-mapper-multipath-libs.x86_64::0.4.9-134.0.3.el7_9", "dhclient.x86_64::12:4.2.5  -83.0.1.el7_9.1", "dhcp.x86_64::12:4.2.5-83.0.1.el7_9.1", "dhcp-common.x86_64::12:4.2.5-83.0.1.el7_9.1", "dhcp-libs.x86_64::12:4.2.5-83.0.1.el7_9.1", "dmidecode.x86_64::1:3.2-5.0.1.el7_9.1", "glib2.x86_64::2.56.1-9.el7_9", "glibc.i686:  :2.17-324.0.1.el7_9", "glibc.x86_64::2.17-324.0.1.el7_9", "glibc-common.x86_64::2.17-324.0.1.el7_9", "glibc-devel.x86_64::2.17-324.0.1.el7_9", "glibc-headers.x86_64::2.17-324.0.1.el7_9", "grub2.x86_64::1:2.02-0.87.0.9.el7_9.6", "grub2-  common.noarch::1:2.02-0.87.0.9.el7_9.6", "grub2-efi-x64.x86_64::1:2.02-0.87.0.9.el7_9.6", "grub2-pc.x86_64::1:2.02-0.87.0.9.el7_9.6", "grub2-pc-modules.noarch::1:2.02-0.87.0.9.el7_9.6", "grub2-tools.x86_64::1:2.02-0.87.0.9.el7_9.6", "g  rub2-tools-extra.x86_64::1:2.02-0.87.0.9.el7_9.6", "grub2-tools-minimal.x86_64::1:2.02-0.87.0.9.el7_9.6", "gssproxy.x86_64::0.7.0-30.el7_9", "hwdata.x86_64::0.252-9.7.0.1.el7", "iscsi-initiator-utils.x86_64::6.2.0.874-20.0.5.el7_9", "i  scsi-initiator-utils-iscsiuio.x86_64::6.2.0.874-20.0.5.el7_9", "kernel-devel.x86_64::3.10.0-1160.36.2.el7", "kernel-headers.x86_64::3.10.0-1160.36.2.el7", "kernel-uek.x86_64::4.1.12-124.53.5.el7uek", "kernel-uek-firmware.noarch::4.1.12  -124.53.5.el7uek", "kexec-tools.x86_64::2.0.15-51.0.3.el7_9.3", "kpartx.x86_64::0.4.9-134.0.3.el7_9", "krb5-libs.x86_64::1.15.1-50.0.1.el7", "libcurl.x86_64::7.29.0-59.0.3.el7_9.1", "libgudev1.x86_64::219-78.0.5.el7_9.3", "libipa_hbac.  x86_64::1.16.5-10.0.1.el7_9.8", "libldb.x86_64::1.5.4-2.el7_9", "libsmbclient.x86_64::4.10.16-15.el7_9", "libsss_autofs.x86_64::1.16.5-10.0.1.el7_9.8", "libsss_certmap.x86_64::1.16.5-10.0.1.el7_9.8", "libsss_idmap.x86_64::1.16.5-10.0.1  .el7_9.8", "libsss_nss_idmap.x86_64::1.16.5-10.0.1.el7_9.8", "libsss_sudo.x86_64::1.16.5-10.0.1.el7_9.8", "libwbclient.x86_64::4.10.16-15.el7_9", "libxml2.i686::2.9.1-6.0.3.el7.5", "libxml2.x86_64::2.9.1-6.0.3.el7.5", "libxml2-python.x  86_64::2.9.1-6.0.3.el7.5", "linux-firmware.noarch::999:20210617-999.8.git0f66b74b.el7", "lvm2.x86_64::7:2.02.187-6.0.5.el7_9.5", "lvm2-libs.x86_64::7:2.02.187-6.0.5.el7_9.5", "mokutil.x86_64::15.3-1.0.5.el7", "nettle.x86_64::2.7.1-9.el  7_9", "nfs-utils.x86_64::1:1.3.0-0.68.0.1.el7.1", "nss.x86_64::3.53.1-7.el7_9", "nss-sysinit.x86_64::3.53.1-7.el7_9", "nss-tools.x86_64::3.53.1-7.el7_9", "openldap.x86_64::2.4.44-23.el7_9", "openssl.x86_64::1:1.0.2k-21.0.3.el7_9", "ope  nssl-libs.x86_64::1:1.0.2k-21.0.3.el7_9", "oracle-database-preinstall-19c.x86_64::1.0-3.el7", "pciutils.x86_64::3.5.1-3.0.1.el7", "pciutils-libs.x86_64::3.5.1-3.0.1.el7", "python.x86_64::2.7.5-90.0.3.el7", "python-libs.x86_64::2.7.5-90  .0.3.el7", "python-sssdconfig.noarch::1.16.5-10.0.1.el7_9.8", "redhat-release-server.x86_64::1:7.9-6.0.1.el7_9", "rsyslog.x86_64::8.24.0-57.0.1.el7_9.1", "samba-client-libs.x86_64::4.10.16-15.el7_9", "samba-common.noarch::4.10.16-15.el  7_9", "samba-common-libs.x86_64::4.10.16-15.el7_9", "screen.x86_64::4.1.0-0.27.20120314git3c2946.el7_9", "selinux-policy.noarch::3.13.1-268.0.13.el7_9.2", "selinux-policy-targeted.noarch::3.13.1-268.0.13.el7_9.2", "shim-x64.x86_64::15.  3-1.0.5.el7", "sos.noarch::3.9-5.0.11.el7_9.6", "sssd.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-ad.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-client.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-common.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-common-pac.x  86_64::1.16.5-10.0.1.el7_9.8", "sssd-ipa.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-krb5.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-krb5-common.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-ldap.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-proxy.x86_64::1.16.5  -10.0.1.el7_9.8", "systemd.x86_64::219-78.0.5.el7_9.3", "systemd-libs.x86_64::219-78.0.5.el7_9.3", "systemd-sysv.x86_64::219-78.0.5.el7_9.3", "unzip.x86_64::6.0-22.el7_9" ],
  "installedRpms" : [ "bind.x86_64::32:9.11.4-26.P2.el7_9.3", "bind-export-libs.x86_64::32:9.11.4-26.P2.el7_9.3", "bind-libs.x86_64::32:9.11.4-26.P2.el7_9.3", "bind-libs-lite.x86_64::32:9.11.4-26.P2.el7_9.3", "bind-license.noarch::32:9  .11.4-26.P2.el7_9.3", "bind-utils.x86_64::32:9.11.4-26.P2.el7_9.3", "curl.x86_64::7.29.0-59.0.1.el7_9.1", "device-mapper.x86_64::7:1.02.170-6.0.3.el7_9.3", "device-mapper-event.x86_64::7:1.02.170-6.0.3.el7_9.3", "7:1.02.170-6.0.3.el7_9  .3::@pdit_ol7_latest/$releasever", "device-mapper-libs.x86_64::7:1.02.170-6.0.3.el7_9.3", "0.4.9-134.0.1.el7_9::@pdit_ol7_latest/$releasever", "0.4.9-134.0.1.el7_9::@pdit_ol7_latest/$releasever", "dhclient.x86_64::12:4.2.5-82.0.1.el7",   "dhcp.x86_64::12:4.2.5-82.0.1.el7", "dhcp-common.x86_64::12:4.2.5-82.0.1.el7", "dhcp-libs.x86_64::12:4.2.5-82.0.1.el7", "dmidecode.x86_64::1:3.2-5.el7", "glib2.x86_64::2.56.1-8.el7", "glibc.i686::2.17-322.0.1.el7_9", "glibc.x86_64::2.  17-322.0.1.el7_9", "glibc-common.x86_64::2.17-322.0.1.el7_9", "glibc-devel.x86_64::2.17-322.0.1.el7_9", "glibc-headers.x86_64::2.17-322.0.1.el7_9", "grub2.x86_64::1:2.02-0.87.0.5.el7", "grub2-common.noarch::1:2.02-0.87.0.5.el7", "grub2  -efi-x64.x86_64::1:2.02-0.87.0.5.el7", "grub2-pc.x86_64::1:2.02-0.87.0.5.el7", "grub2-pc-modules.noarch::1:2.02-0.87.0.5.el7", "grub2-tools.x86_64::1:2.02-0.87.0.5.el7", "grub2-tools-extra.x86_64::1:2.02-0.87.0.5.el7", "grub2-tools-min  imal.x86_64::1:2.02-0.87.0.5.el7", "gssproxy.x86_64::0.7.0-29.el7", "hwdata.x86_64::0.252-9.7.el7", "6.2.0.874-20.0.1.el7_9::@pdit_ol7_latest/$releasever", "6.2.0.874-20.0.1.el7_9::@pdit_ol7_latest/$releasever", "kernel-devel.x86_64::3  .10.0-1160.15.2.el7", "kernel-headers.x86_64::3.10.0-1160.15.2.el7", "kernel-uek.x86_64::4.1.12-124.47.3.el7uek", "kernel-uek-firmware.noarch::4.1.12-124.47.3.el7uek", "kexec-tools.x86_64::2.0.15-51.0.3.el7_9.1", "kpartx.x86_64::0.4.9-  134.0.1.el7_9", "krb5-libs.x86_64::1.15.1-50.el7", "libcurl.x86_64::7.29.0-59.0.1.el7_9.1", "libgudev1.x86_64::219-78.0.1.el7_9.3", "libipa_hbac.x86_64::1.16.5-10.0.1.el7_9.7", "libldb.x86_64::1.5.4-1.el7", "libsmbclient.x86_64::4.10.1  6-9.el7_9", "libsss_autofs.x86_64::1.16.5-10.0.1.el7_9.7", "libsss_certmap.x86_64::1.16.5-10.0.1.el7_9.7", "libsss_idmap.x86_64::1.16.5-10.0.1.el7_9.7", "libsss_nss_idmap.x86_64::1.16.5-10.0.1.el7_9.7", "libsss_sudo.x86_64::1.16.5-10.0  .1.el7_9.7", "libwbclient.x86_64::4.10.16-9.el7_9", "libxml2.i686::2.9.1-6.0.1.el7.5", "libxml2.x86_64::2.9.1-6.0.1.el7.5", "libxml2-python.x86_64::2.9.1-6.0.1.el7.5", "linux-firmware.noarch::999:20200902-999.5.gitd5f9eea5.el7", "lvm2.  x86_64::7:2.02.187-6.0.3.el7_9.3", "lvm2-libs.x86_64::7:2.02.187-6.0.3.el7_9.3", "mokutil.x86_64::15-2.0.9.el7", "nettle.x86_64::2.7.1-8.el7", "nfs-utils.x86_64::1:1.3.0-0.68.0.1.el7", "nss.x86_64::3.53.1-3.el7_9", "nss-sysinit.x86_64:  :3.53.1-3.el7_9", "nss-tools.x86_64::3.53.1-3.el7_9", "openldap.x86_64::2.4.44-22.el7", "openssl.x86_64::1:1.0.2k-21.el7_9", "openssl-libs.x86_64::1:1.0.2k-21.el7_9", "1.0-2.el7::@pdit_ol7_latest/$releasever", "pciutils.x86_64::3.5.1-3  .el7", "pciutils-libs.x86_64::3.5.1-3.el7", "python.x86_64::2.7.5-90.0.1.el7", "python-libs.x86_64::2.7.5-90.0.1.el7", "python-sssdconfig.noarch::1.16.5-10.0.1.el7_9.7", "1:7.9-3.0.1.el7::@pdit_ol7_latest/$releasever", "rsyslog.x86_64:  :8.24.0-57.el7_9", "samba-client-libs.x86_64::4.10.16-9.el7_9", "samba-common.noarch::4.10.16-9.el7_9", "samba-common-libs.x86_64::4.10.16-9.el7_9", "screen.x86_64::4.1.0-0.26.20120314git3c2946.el7", "selinux-policy.noarch::3.13.1-268.  0.1.el7_9.2", "3.13.1-268.0.1.el7_9.2::@pdit_ol7_latest/$releasever", "shim-x64.x86_64::15-2.0.9.el7", "sos.noarch::3.9-5.0.3.el7_9.2", "sssd.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-ad.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-client.x86_64:  :1.16.5-10.0.1.el7_9.7", "sssd-common.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-common-pac.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-ipa.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-krb5.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-krb5-common.x86_64::1.16.  5-10.0.1.el7_9.7", "sssd-ldap.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-proxy.x86_64::1.16.5-10.0.1.el7_9.7", "systemd.x86_64::219-78.0.1.el7_9.3", "systemd-libs.x86_64::219-78.0.1.el7_9.3", "systemd-sysv.x86_64::219-78.0.1.el7_9.3", "unzi  p.x86_64::6.0-21.el7" ]
}

3. Note the kernel before the update.

uname -r
4.1.12-124.47.3.el7uek.x86_64

4. Run the pre-update check for the os.

dbcli update-server -c os -p
{
  "jobId" : "9b02cf04-92a2-4299-8cd0-4ce649b962ec",
  "status" : "Created",
  "message" : null,
  "reports" : [ ],
  "createTimestamp" : "September 24, 2021 14:18:46 PM BST",
  "resourceList" : [ ],
  "description" : "OS Patching Prechecks",
  "updatedTime" : "September 24, 2021 14:18:47 PM BST",
  "percentageProgress" : "0%",
  "cause" : null,
  "action" : null
}

Check if pre-udpate check is successful.

dbcli describe-job -i "9b02cf04-92a2-4299-8cd0-4ce649b962ec"

Job details
----------------------------------------------------------------
                     ID:  9b02cf04-92a2-4299-8cd0-4ce649b962ec
            Description:  OS Patching Prechecks
                 Status:  Success
                Created:  September 24, 2021 2:18:46 PM BST
               Progress:  100%
                Message:

Task Name                                                                Start Time                          End Time                            Status
------------------------------------------------------------------------ ----------------------------------- ----------------------------------- ----------

5. Run the os update.

dbcli update-server -c os
{
  "jobId" : "a7097060-6234-401d-bfd7-1c1fcbb9de85",
  "status" : "Created",
  "message" : null,
  "reports" : [ ],
  "createTimestamp" : "September 24, 2021 15:41:34 PM BST",
  "resourceList" : [ ],
  "description" : "OS Patching",
  "updatedTime" : "September 24, 2021 15:41:34 PM BST",
  "percentageProgress" : "0%",
  "cause" : null,
  "action" : null
}

The first run of the OS update failed with below output..

dbcli describe-job -i "a7097060-6234-401d-bfd7-1c1fcbb9de85"

Job details
----------------------------------------------------------------
                     ID:  a7097060-6234-401d-bfd7-1c1fcbb9de85
            Description:  OS Patching
                 Status:  Failure
                Created:  September 24, 2021 3:41:34 PM BST
               Progress:  78%
                Message:  DCS-10206:Patch bundle conflicts with: Yum Failed to resolve conflict
                  Cause:  Conflict check failed.
                 Action:  Rollback the conflicting patches and retry the operation.

Task Name                                                                Start Time                          End Time                            Status
------------------------------------------------------------------------ ----------------------------------- ----------------------------------- ----------
OS Component Patching                                                    September 24, 2021 3:41:34 PM BST   September 24, 2021 4:22:04 PM BST   Failure
task:TaskSequential_919                                                  September 24, 2021 3:56:45 PM BST   September 24, 2021 4:22:04 PM BST   Failure
Validating OS Patches                                                    September 24, 2021 3:56:45 PM BST   September 24, 2021 4:22:04 PM BST   Failure

After a SR Oracle support confimed that updating is hitting a bug (internal 33403308). After the bug was fixed by Oracle development team ran the command again and OS update was successful.

dbcli update-server -c os
{
  "jobId" : "b1ad604f-7961-4854-a05d-9e43958312b7",
  "status" : "Created",
  "message" : null,
  "reports" : [ ],
  "createTimestamp" : "September 29, 2021 09:32:50 AM BST",
  "resourceList" : [ ],
  "description" : "OS Patching",
  "updatedTime" : "September 29, 2021 09:32:51 AM BST",
  "percentageProgress" : "0%",
  "cause" : null,
  "action" : null
}

dbcli describe-job -i "b1ad604f-7961-4854-a05d-9e43958312b7"

Job details
----------------------------------------------------------------
                     ID:  b1ad604f-7961-4854-a05d-9e43958312b7
            Description:  OS Patching
                 Status:  Success
                Created:  September 29, 2021 9:32:50 AM BST
               Progress:  100%
                Message:

Task Name                                                                Start Time                          End Time                            Status
------------------------------------------------------------------------ ----------------------------------- ----------------------------------- ----------

The issue seems to have been fixed permanently as subsequent OS patching of other DB systems didn't have any issue.

6. Reboot the node and verify the kernel is updated.

uname -r
4.1.12-124.53.5.el7uek.x86_

Related Post
Updating the OS and Patching OCI PaaS DB Using DBCLI

Updating the DCS Agent to the Expected Version

Following error was given while patching DBCS VM.

 dbcli update-server
{
  "jobId" : "a81b17c5-3c22-44a7-ae9e-a6c4b62fe5d6",
  "status" : "Created",
  "message" : null,
  "reports" : [ ],
  "createTimestamp" : "August 24, 2021 12:06:07 PM BST",
  "resourceList" : [ ],
  "description" : "Server Patching",
  "updatedTime" : "August 24, 2021 12:06:07 PM BST",
  "percentageProgress" : "0%",
  "cause" : null,
  "action" : null
}

dbcli describe-job -i a81b17c5-3c22-44a7-ae9e-a6c4b62fe5d6

Job details
----------------------------------------------------------------
                     ID:  a81b17c5-3c22-44a7-ae9e-a6c4b62fe5d6
            Description:  Server Patching
                 Status:  Failure
                Created:  August 24, 2021 12:06:07 PM BST
               Progress:  0%
                Message:  DCS-10205:Operation: Patch update failed. Found unsupported DCS Agent version: 21.2.2.2.0. Expected version: 21.2.3.0.0
                  Cause:  Attempted operation required expected version of DCS Agent.
                 Action:  Update the DCS Agent to the expected version, use the command 'odacli update-dcsagent' to update DCS Agent.

Running the odacli as suggested by the error message also failed.

odacli update-dcsagent
dbcli: 'update-dcsagent' is not an dbcli command.
usage: dbcli    [-h/--help]
                <category> [-h/--help]
                <operation> [-h/--help]
                <command> [-h/--help]
                <command> [<args>]

Installed DCS* verions were

rpm -qa|grep dcs
dcs-cli-21.2.2.2.0_210608.0448-1.x86_64
dcs-agent-21.3.1.0.0_210624.0711-16.x86_64
dcs-admin-20.4.2.1.0_210114.1437-1.x86_64

Running cliadm update-dbcli which updates the dbcli didn't resolve it.



Solution was to restart the dcs admin and agent (refer 2473667.1).

systemctl stop initdcsagent
systemctl stop initdcsadmin

systemctl start initdcsagent
systemctl start initdcsadmin

rpm -qa | grep dcs
dcs-agent-21.3.1.0.0_210624.0711-16.x86_64
dcs-cli-21.3.1.0.0_210624.0711-1.x86_64
dcs-admin-20.4.2.1.0_210114.1437-1.x86_64

Afterwards patching continued without error.

Backup and Recover Using DBCLI on OCI PaaS DB

OCI Paas DBs could be configured with automatic backup in which case OCI would take care of the backup. The database gets backed up to an object storage. But the backup files or the object storage bucket they are stored in is not visible from the console. This post shows the steps for backing up a PaaS DB to a user created object storage bucket using DBCLI. It is assumed a object store bucket is already created. For this post an object store bucket called "orabaks" was created. This will be used as the backup storage.

Backup using DBCLI
1. In order to use this object storage bucket it must be associated with an object store swift end point. Create object store swift end point as shown below. This uses tenancy and username used to login to the tenancy and the authentication token as the password for the "object store swift password". The end point URL must be changed to reflect the region where the DB resides. Refer previous post on dbcli command to find the region.

# dbcli create-objectstoreswift -n orabackupstore -t tenancyName -u asanga@email.com -e https://swiftobjectstorage.eu-frankfurt-1.oraclecloud.com/v1 -p
object store swift password:
{
  "jobId" : "3fa9e4fd-9da1-413d-9bca-e2434afb9229",
  "status" : "Created",
  "message" : "Create object store swift",
  "reports" : [ ],
  "createTimestamp" : "March 13, 2019 11:12:26 AM UTC",
  "resourceList" : [ {
    "resourceId" : "cb78ef29-a447-4b2d-af17-5b110a6717cb",
    "resourceType" : "ObjectStoreSwift",
    "jobId" : "3fa9e4fd-9da1-413d-9bca-e2434afb9229",
    "updatedTime" : "March 13, 2019 11:12:26 AM UTC"
  } ],
  "description" : "create object store:orabackupstore",
  "updatedTime" : "March 13, 2019 11:12:26 AM UTC"
}
# dbcli list-objectstoreswifts

ID                                       Name                 UserName                       TenantName                Url
---------------------------------------- -------------------- ------------------------------ ------------------------- ---------------------------------------------------------------------------
c9ffa569-7f15-4220-8524-0e6f294871a4     bnCGRmStQJ1DnEMR1p4a bnCGRmStQJ1DnEMR1p4a           dbbackupfra               https://swiftobjectstorage.eu-frankfurt-1.oraclecloud.com/v1
cb78ef29-a447-4b2d-af17-5b110a6717cb     orabackupstore       asanga@email.com               tenancyName               https://swiftobjectstorage.eu-frankfurt-1.oraclecloud.com/v1

2. Create a backup configuration using the object store swift ID created in the previous step and the object store name (orabaks). The backup destination is listed as object store and this could be changed to disk for disk backups, in which case backups will be created in the FRA.

# dbcli create-backupconfig -d OBJECTSTORE -c orabaks -o cb78ef29-a447-4b2d-af17-5b110a6717cb -w 1 -n mybakconfig1
{
  "jobId" : "0ecd1ed3-658a-4374-ad42-105f80e5480d",
  "status" : "Created",
  "message" : "backup config creation",
  "reports" : [ ],
  "createTimestamp" : "March 13, 2019 11:21:05 AM UTC",
  "resourceList" : [ {
    "resourceId" : "09c01611-6a58-49d1-b9de-553594b73cfc",
    "resourceType" : "BackupConfig",
    "jobId" : "0ecd1ed3-658a-4374-ad42-105f80e5480d",
    "updatedTime" : "March 13, 2019 11:21:05 AM UTC"
  } ],
  "description" : "create backup config:mybakconfig1",
  "updatedTime" : "March 13, 2019 11:21:05 AM UTC"
}

dbcli list-backupconfigs

ID                                       Name                 RecoveryWindow   CrosscheckEnabled   BackupDestination
---------------------------------------- -------------------- ---------------- ------------------- --------------------
2e6e46e9-9dca-4906-a2e1-27b8df727152     bnCGRmStQJ1DnEMR1p4a_BC 30               true                ObjectStore
09c01611-6a58-49d1-b9de-553594b73cfc     mybakconfig1         1                   true                ObjectStore


dbcli describe-backupconfig -i 09c01611-6a58-49d1-b9de-553594b73cfc
Backup Config details
----------------------------------------------------------------
                     ID: 09c01611-6a58-49d1-b9de-553594b73cfc
                   Name: mybakconfig1
      CrosscheckEnabled: true
         RecoveryWindow: 1
      BackupDestination: ObjectStore
         BackupLocation: orabaks
          ObjectStoreId: cb78ef29-a447-4b2d-af17-5b110a6717cb
            CreatedTime: March 13, 2019 11:21:05 AM UTC
            UpdatedTime: March 13, 2019 11:21:34 AM UTC

3. Next step is to associate the newly created database configuration with the database. For this the database id is needed. Use the following command to find the DB id and the ID of the current backup configuration associated with the DB.

dbcli list-databases

ID                                       DB Name    DB Type  DB Version           CDB        Class    Shape    Storage    Status        DbHomeID
---------------------------------------- ---------- -------- -------------------- ---------- -------- -------- ---------- ------------ ----------------------------------------
690f6c09-7c48-4332-ab65-b86f932c1a33     apcdb      Si       18.5.0.0.190115      true       Oltp              ASM        Configured   0f59f70a-167b-4d8a-84e3-6e7d7529b093

dbcli describe-database -i 690f6c09-7c48-4332-ab65-b86f932c1a33

Database details
----------------------------------------------------------------
                     ID: 690f6c09-7c48-4332-ab65-b86f932c1a33
            Description: apcdb
                DB Name: apcdb
             DB Version: 18.5.0.0.190115
                DB Type: Si
             DB Edition: EE
                   DBID: 1068299060
 Instance Only Database: false
                    CDB: true
               PDB Name: cgpdb
    PDB Admin User Name: pdbuser
                  Class: Oltp
                  Shape:
                Storage: ASM
          DB Redundancy:
           CharacterSet: AL32UTF8
  National CharacterSet: AL16UTF16
               Language: AMERICAN
              Territory: AMERICA
                Home ID: 0f59f70a-167b-4d8a-84e3-6e7d7529b093
        Console Enabled: false
     Level 0 Backup Day: Sunday
     AutoBackup Enabled: false
         BackupConfigId: 2e6e46e9-9dca-4906-a2e1-27b8df727152
                Created: March 11, 2019 9:06:44 AM UTC
         DB Domain Name: dbsubnet.fravcn.oraclevcn.com

4. To associate the new backup configuration with the database use the update database command.

dbcli update-database -i 690f6c09-7c48-4332-ab65-b86f932c1a33 -bi 09c01611-6a58-49d1-b9de-553594b73cfc
{
  "jobId" : "597671db-4db5-4a1e-91af-325d8c9ae39e",
  "status" : "Created",
  "message" : "update database",
  "reports" : [ ],
  "createTimestamp" : "March 13, 2019 11:26:50 AM UTC",
  "resourceList" : [ {
    "resourceId" : "690f6c09-7c48-4332-ab65-b86f932c1a33",
    "resourceType" : "DB",
    "jobId" : "597671db-4db5-4a1e-91af-325d8c9ae39e",
    "updatedTime" : "March 13, 2019 11:26:50 AM UTC"
  } ],
  "description" : "update database : apcdb",
  "updatedTime" : "March 13, 2019 11:26:50 AM UTC"
}

 dbcli describe-database -i 690f6c09-7c48-4332-ab65-b86f932c1a33

Database details
----------------------------------------------------------------
                     ID: 690f6c09-7c48-4332-ab65-b86f932c1a33
            Description: apcdb
                DB Name: apcdb
             DB Version: 18.5.0.0.190115
                DB Type: Si
             DB Edition: EE
                   DBID: 1068299060
 Instance Only Database: false
                    CDB: true
               PDB Name: cgpdb
    PDB Admin User Name: pdbuser
                  Class: Oltp
                  Shape:
                Storage: ASM
          DB Redundancy:
           CharacterSet: AL32UTF8
  National CharacterSet: AL16UTF16
               Language: AMERICAN
              Territory: AMERICA
                Home ID: 0f59f70a-167b-4d8a-84e3-6e7d7529b093
        Console Enabled: false
     Level 0 Backup Day: Sunday
     AutoBackup Enabled: false
         BackupConfigId: 09c01611-6a58-49d1-b9de-553594b73cfc
                Created: March 11, 2019 9:06:44 AM UTC
         DB Domain Name: dbsubnet.fravcn.oraclevcn.com

5. Create a database backup by specifying the DB id. The backups files will be stored in the user created object store created (orabaks). The following command shows creating an archive log backup.

dbcli create-backup -bt ArchiveLog -i 690f6c09-7c48-4332-ab65-b86f932c1a33
{
  "jobId" : "047bbf1e-2ef2-4efd-a0dd-663d0b2c4456",
  "status" : "Created",
  "message" : null,
  "reports" : [ ],
  "createTimestamp" : "March 13, 2019 11:34:01 AM UTC",
  "resourceList" : [ ],
  "description" : "Create ArchiveLog Backup with TAG-auto for Db:apcdb in OSS:orabaks",
  "updatedTime" : "March 13, 2019 11:34:01 AM UTC"
}

dbcli describe-job -i 047bbf1e-2ef2-4efd-a0dd-663d0b2c4456

Job details
----------------------------------------------------------------
                     ID:  047bbf1e-2ef2-4efd-a0dd-663d0b2c4456
            Description:  Create ArchiveLog Backup with TAG-auto for Db:apcdb in OSS:orabaks
                 Status:  Success
                Created:  March 13, 2019 11:34:01 AM UTC
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Validate backup config                   March 13, 2019 11:34:04 AM UTC      March 13, 2019 11:34:04 AM UTC      Success
Database container validation            March 13, 2019 11:34:04 AM UTC      March 13, 2019 11:34:05 AM UTC      Success
libopc existence check                   March 13, 2019 11:34:05 AM UTC      March 13, 2019 11:34:05 AM UTC      Success
Backup Validations                       March 13, 2019 11:34:05 AM UTC      March 13, 2019 11:34:54 AM UTC      Success
Recovery Window validation               March 13, 2019 11:34:54 AM UTC      March 13, 2019 11:35:17 AM UTC      Success
Archivelog deletion policy configuration March 13, 2019 11:35:17 AM UTC      March 13, 2019 11:35:37 AM UTC      Success
Archive log backup                       March 13, 2019 11:35:37 AM UTC      March 13, 2019 11:38:41 AM UTC      Success
Backup TDE Wallet files to ObjectStore   March 13, 2019 11:38:41 AM UTC      March 13, 2019 11:38:42 AM UTC      Success

The created backup files are visible inside the object storage bucket.

6. A RMAN backup report could be created using the DB id. The rman report will list the log files which would have finer details of the backup.

dbcli create-rmanbackupreport -i 690f6c09-7c48-4332-ab65-b86f932c1a33 -w detailed -rn archlogreport
{
  "jobId" : "20f74edf-dcbf-4bd7-89cd-51358d2ab7d6",
  "status" : "Created",
  "message" : "Rman BackupReport creation.",
  "reports" : [ ],
  "createTimestamp" : "March 13, 2019 11:44:45 AM UTC",
  "resourceList" : [ {
    "resourceId" : "34dbed6a-4182-4fb5-a1b8-8918a799485f",
    "resourceType" : "Report",
    "jobId" : "20f74edf-dcbf-4bd7-89cd-51358d2ab7d6",
    "updatedTime" : "March 13, 2019 11:44:45 AM UTC"
  } ],
  "description" : "Create detailed Backup Report ",
  "updatedTime" : "March 13, 2019 11:44:45 AM UTC"
}


dbcli list-rmanbackupreports

ID                                       Name                 ReportType DbId                                     createTime                          updatedTime
---------------------------------------- -------------------- ---------- ---------------------------------------- ----------------------------------- -----------------------------------
34dbed6a-4182-4fb5-a1b8-8918a799485f     archlogreport        detailed   690f6c09-7c48-4332-ab65-b86f932c1a33     March 13, 2019 11:44:45 AM UTC      March 13, 2019 11:45:36 AM UTC

describe-rmanbackupreport -i 34dbed6a-4182-4fb5-a1b8-8918a799485f
Backup Report details
----------------------------------------------------------------
                     ID: 34dbed6a-4182-4fb5-a1b8-8918a799485f
            Report Type: detailed
               Location: Node apdb: /opt/oracle/dcs/log/apdb/rman/bkup/apcdb_fra2g6/rman_list_backup_detail/2019-03-13/rman_list_backup_detail_2019-03-13_11-45-20.0508.log
            Database ID: 690f6c09-7c48-4332-ab65-b86f932c1a33
            CreatedTime: March 13, 2019 11:44:45 AM UTC

ID                                       Description                                                                 Created                             Status
---------------------------------------- --------------------------------------------------------------------------- ----------------------------------- ----------
b5d96197-91ec-4319-abf7-09b39d70bafb     Create Regular-L0 Backup with TAG-auto for Db:apcdb in OSS:orabaks          March 13, 2019 12:01:36 PM UTC      Success
8bc91a59-2bee-4b9e-aa59-b73048adca38     Create Archivelog Backup with TAG-auto for Db:apcdb in OSS:orabaks          March 13, 2019 12:53:59 PM UTC      Success

Backup report could be deleted using the following command.

dbcli delete-rmanbackupreport -i 34dbed6a-4182-4fb5-a1b8-8918a799485f

To remove a backup configuration, if its assign to the DB then update the DB with a different backup configuration. Use the following command to delete the backup configuration by specifying backup configuration id.

dbcli delete-backupconfig -i 09c01611-6a58-49d1-b9de-553594b73cfc

To delete a object store swift use the following command, specifying object store swift id.

dbcli delete-objectstoreswift -i cb78ef29-a447-4b2d-af17-5b110a6717cb

Recover using DBCLI
Following steps shows point in time recovery using DBCLI to simulate a recovery from accidental drop of a table. DBCLI also provides other recovery types as well (Latest|SCN).
1. The table is droped using purge option.

SQL> conn asanga/asanga#1234$ABC@pdbsrvx
Connected.
SQL> ! date
Wed Mar 13 12:58:37 UTC 2019

SQL> drop table x purge;
Table dropped.

2. Use recover database command to execute a PITR. The recovery time stamp (specified by -r) is just before the table was dropped.

dbcli recover-database -i 690f6c09-7c48-4332-ab65-b86f932c1a33 -r 03/13/2019 12:58:35 -t PITR
{
  "jobId" : "037b6bfd-1bc3-4c2e-af5e-1a22dbc05189",
  "status" : "Created",
  "message" : null,
  "reports" : [ ],
  "createTimestamp" : "March 13, 2019 13:01:57 PM UTC",
  "resourceList" : [ ],
  "description" : "Create recovery-pitr : time '03/13/2019 12:58:35' for db : apcdb",
  "updatedTime" : "March 13, 2019 13:01:57 PM UTC"
}

Monitor the recovery using the recovery job id given above.

dbcli describe-job -i 037b6bfd-1bc3-4c2e-af5e-1a22dbc05189

Job details
----------------------------------------------------------------
                     ID:  037b6bfd-1bc3-4c2e-af5e-1a22dbc05189
            Description:  Create recovery-pitr : time '03/13/2019 12:58:35' for db : apcdb
                 Status:  Success
                Created:  March 13, 2019 1:01:57 PM UTC
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Database recovery validation             March 13, 2019 1:02:08 PM UTC       March 13, 2019 1:03:05 PM UTC       Success
Database recovery                        March 13, 2019 1:03:05 PM UTC       March 13, 2019 1:05:54 PM UTC       Success
Enable block change tracking             March 13, 2019 1:05:54 PM UTC       March 13, 2019 1:05:56 PM UTC       Success
Database opening                         March 13, 2019 1:05:56 PM UTC       March 13, 2019 1:06:53 PM UTC       Success
Database restart                         March 13, 2019 1:06:53 PM UTC       March 13, 2019 1:08:55 PM UTC       Success
Recovery metadata persistance            March 13, 2019 1:08:55 PM UTC       March 13, 2019 1:08:55 PM UTC       Success

Verify table is recovered

SQL> desc x
 Name     Null?    Type
 -------- ------- ----------------------------
 A                NUMBER

Updating the OS and Patching OCI PaaS DB Using DBCLI

OCI provide a command line interface for managing the PaaS DB (DBCLI) beside the console. This post shows the steps for updating the OS and patching the DB System and the DB using DBCLI.

Updating the OS
To update the OS the database server needs access to the YUM repository. If the DB system is in a private subnet then configure a NAT gateway to access the YUM repository. It is assumed that NAT gateway is configured and DB server can access the internet or egress traffic is allowed only to YUM repository location.
1. Check the region where the DB server resides.

# curl -s http://169.254.169.254/opc/v1/instance/ |grep region
  "region" : "eu-frankfurt-1",

2. In this case the region is eu-frankfurt-1. Replace the region name in below URL with actual region and get the yum repos file.

https://swiftobjectstorage.region_name.oraclecloud.com/v1/dbaaspatchstore/DBaaSOSPatches/oci_dbaas_ol6repo

# wget https://swiftobjectstorage.eu-frankfurt-1.oraclecloud.com/v1/dbaaspatchstore/DBaaSOSPatches/oci_dbaas_ol6repo -O /tmp/oci_dbaas_ol6repo
--2019-03-12 10:14:05--  https://swiftobjectstorage.eu-frankfurt-1.oraclecloud.com/v1/dbaaspatchstore/DBaaSOSPatches/oci_dbaas_ol6repo
...
Saving to: `/tmp/oci_dbaas_ol6repo'

3. Get the version lock file. Version lock file is needed to control the update. An update will occur only if a versionlock file has a valid update available to apply to the DB system.

# wget https://swiftobjectstorage.eu-frankfurt-1.oraclecloud.com/v1/dbaaspatchstore/DBaaSOSPatches/versionlock.list -O /tmp/versionlock.list
--2019-03-12 10:14:37--  https://swiftobjectstorage.eu-frankfurt-1.oraclecloud.com/v1/dbaaspatchstore/DBaaSOSPatches/versionlock.list
...
Saving to: `/tmp/versionlock.list'

4. Copy the downloaded files to their respective locations in the DB server

cp /tmp/oci_dbaas_ol6repo /etc/yum.repos.d/ol6.repo

cp /tmp/versionlock.list /etc/yum/pluginconf.d/versionlock.list

5. Open the yum repository file and enable the repository for the region. During this testing it was found that Frankfurt region was enabled by default.

6. Note the current kernel version and run the update command.

uname -r
4.1.12-124.20.3.el6uek.x86_64

yum update

Any warning similar to following could be ignored.

Warning: RPMDB altered outside of yum.
** Found 3 pre-existing rpmdb problem(s), 'yum check' output follows:
oda-hw-mgmt-18.3.1.0.0_LINUX.X64_181212-1.x86_64 has missing requires of libnfsodm18.so()(64bit)
oda-hw-mgmt-18.3.1.0.0_LINUX.X64_181212-1.x86_64 has missing requires of perl(GridDefParams)
oda-hw-mgmt-18.3.1.0.0_LINUX.X64_181212-1.x86_64 has missing requires of perl(s_GridSteps)

Also towards the end of the update following error.

ol6_UEKR4/filelists_db                                                                                                                                                                                                | 1.6 MB     00:00
Uploading /boot/vmlinuz-4.1.12-124.25.1.el6uek.x86_64 to http://169.254.0.3/kernel

Error uploading /kernel: 502 - <html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
</body>
</html>

In spite of this error the kernel gets updated as shown below

6. Once the update is complete reboot the server and check the kernel version.

# reboot
# uname -r
4.1.12-124.25.1.el6uek.x86_64

Patching the PaaS DB
Patching the DB is of two parts similar to patching using console. First is to patch the DB system (Grid Infrastructure) and then the DB.

Updating the DBCLI
Before the patching could begin check if there's a newer version of DBCLI and update DBCLI.

cliadm update-dbcli

Job details
----------------------------------------------------------------
                     ID:  d0a0e701-3c5a-4f72-a97e-455efd02cefd
            Description:  DcsCli patching
                 Status:  Created
                Created:  March 12, 2019 10:59:31 AM UTC
                Message:  Dcs cli will be updated

Wait couple of minutes and check the outcome of the update, (use the job id given)

dbcli describe-job -i d0a0e701-3c5a-4f72-a97e-455efd02cefd

Job details
----------------------------------------------------------------
                     ID:  d0a0e701-3c5a-4f72-a97e-455efd02cefd
            Description:  DcsCli patching
                 Status:  Success
                Created:  March 12, 2019 10:59:31 AM UTC
                Message:

Task Name           Start Time                          End Time                            Status
------------------- -------------------------------     ----------------------------------- ----------
dcs-cli upgrade     March 12, 2019 10:59:32 AM UTC      March 12, 2019 10:59:34 AM UTC      Success

Check the currently available patches. This command list available patches for all versions.

# dbcli describe-latestpatch
componentType   availableVersion
--------------- --------------------
gi              12.2.0.1.190115
gi              12.1.0.2.190115
gi              18.5.0.0.190115
db              11.2.0.4.190115
db              12.2.0.1.190115
db              12.1.0.2.190115
db              18.5.0.0.190115

To list the patching available for installed GI and DB run the following command.

# dbcli describe-component
System Version
---------------
18.3.3.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
GI                                        18.3.0.0.180717       18.5.0.0.190115
DB                                        18.3.0.0.180717       18.5.0.0.190115

Patching the DB System
To patch the DB system run update server command.

# dbcli update-server
{
  "jobId" : "058707c8-a9ec-48f0-ba47-47e5aa4f65c6",
  "status" : "Created",
  "message" : null,
  "reports" : [ ],
  "createTimestamp" : "March 12, 2019 11:05:00 AM UTC",
  "resourceList" : [ ],
  "description" : "Server Patching",
  "updatedTime" : "March 12, 2019 11:05:01 AM UTC"
}

The dcs agent log could be used to monitor the patch apply in progress

cd /opt/oracle/dcs/log
tail -100f dcs-agent.log

Another method is to use the describe job option with the job id.

dbcli describe-job -i "058707c8-a9ec-48f0-ba47-47e5aa4f65c6"

Job details
----------------------------------------------------------------
                     ID:  058707c8-a9ec-48f0-ba47-47e5aa4f65c6
            Description:  Server Patching
                 Status:  Running
                Created:  March 12, 2019 11:05:00 AM UTC
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Create Patching Repository Directories   March 12, 2019 11:05:01 AM UTC      March 12, 2019 11:05:01 AM UTC      Success
Download latest patch metadata           March 12, 2019 11:05:01 AM UTC      March 12, 2019 11:05:01 AM UTC      Success
Update Patching Repository               March 12, 2019 11:05:02 AM UTC      March 12, 2019 11:05:02 AM UTC      Running

Once the patching is complete the status will change to success.

dbcli describe-job -i "058707c8-a9ec-48f0-ba47-47e5aa4f65c6"

Job details
----------------------------------------------------------------
                     ID:  058707c8-a9ec-48f0-ba47-47e5aa4f65c6
            Description:  Server Patching
                 Status:  Success
                Created:  March 12, 2019 11:05:00 AM UTC
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Create Patching Repository Directories   March 12, 2019 11:05:01 AM UTC      March 12, 2019 11:05:01 AM UTC      Success
Download latest patch metadata           March 12, 2019 11:05:01 AM UTC      March 12, 2019 11:05:01 AM UTC      Success
Update Patching Repository               March 12, 2019 11:05:02 AM UTC      March 12, 2019 11:08:24 AM UTC      Success
Opatch updation                          March 12, 2019 11:08:25 AM UTC      March 12, 2019 11:08:30 AM UTC      Success
Patch conflict check                     March 12, 2019 11:08:31 AM UTC      March 12, 2019 11:10:31 AM UTC      Success
apply clusterware patch                  March 12, 2019 11:10:31 AM UTC      March 12, 2019 11:32:08 AM UTC      Success
Updating GiHome version                  March 12, 2019 11:32:08 AM UTC      March 12, 2019 11:32:35 AM UTC      Success
Update System version                    March 12, 2019 11:32:52 AM UTC      March 12, 2019 11:32:52 AM UTC      Success

Use the component describe command again to see the patched version. Since the latest patch was applied the available column will state "up-to-date"

dbcli describe-component
System Version
---------------
18.3.3.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
GI                                        18.5.0.0.190115       up-to-date
DB                                         18.3.0.0.180717       18.5.0.0.190115

Patching the DB
Patching of the DB happens by specifying the DB home to patch. For this the DB home ID is needed. To find the DB home ID use the list home command.

dbcli list-dbhomes

ID                                       Name                 DB Version             Home Location                                 Status
---------------------------------------- -------------------- ---------------------- --------------------------------------------- ----------
0f59f70a-167b-4d8a-84e3-6e7d7529b093     OraDB18000_home1     18.3.0.0.180717        /u01/app/oracle/product/18.0.0.0/dbhome_1     Configured

Run the home patching command specifying the DB Home ID.

# dbcli update-dbhome -i 0f59f70a-167b-4d8a-84e3-6e7d7529b093
{
  "jobId" : "dae78453-e3a4-4b6e-ba6b-baaeb8902f2f",
  "status" : "Created",
  "message" : null,
  "reports" : [ ],
  "createTimestamp" : "March 12, 2019 11:37:09 AM UTC",
  "resourceList" : [ ],
  "description" : "DB Home Patching: Home Id is 0f59f70a-167b-4d8a-84e3-6e7d7529b093",
  "updatedTime" : "March 12, 2019 11:37:09 AM UTC"
}

Use the job describe command to find the status of the job.

# dbcli describe-job -i "dae78453-e3a4-4b6e-ba6b-baaeb8902f2f"

Job details
----------------------------------------------------------------
                     ID:  dae78453-e3a4-4b6e-ba6b-baaeb8902f2f
            Description:  DB Home Patching: Home Id is 0f59f70a-167b-4d8a-84e3-6e7d7529b093
                 Status:  Success
                Created:  March 12, 2019 11:37:09 AM UTC
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Create Patching Repository Directories   March 12, 2019 11:37:09 AM UTC      March 12, 2019 11:37:09 AM UTC      Success
Download latest patch metadata           March 12, 2019 11:37:09 AM UTC      March 12, 2019 11:37:09 AM UTC      Success
checking GiHome version                  March 12, 2019 11:37:09 AM UTC      March 12, 2019 11:37:09 AM UTC      Success
Update System version                    March 12, 2019 11:37:09 AM UTC      March 12, 2019 11:37:09 AM UTC      Success
Update Patching Repository               March 12, 2019 11:37:09 AM UTC      March 12, 2019 11:37:55 AM UTC      Success
Validating dbHome available space        March 12, 2019 11:37:55 AM UTC      March 12, 2019 11:37:55 AM UTC      Success
Opatch updation                          March 12, 2019 11:37:56 AM UTC      March 12, 2019 11:38:00 AM UTC      Success
Patch conflict check                     March 12, 2019 11:38:00 AM UTC      March 12, 2019 11:41:00 AM UTC      Success
db upgrade                               March 12, 2019 11:41:00 AM UTC      March 12, 2019 11:59:06 AM UTC      Success
SqlPatch upgrade                         March 12, 2019 11:59:06 AM UTC      March 12, 2019 12:02:14 PM UTC      Success
updating the Database version            March 12, 2019 12:02:29 PM UTC      March 12, 2019 12:02:43 PM UTC      Success

The list component will show the DB is also up-to-date.

# dbcli describe-component
System Version
---------------
18.3.3.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
GI                                        18.5.0.0.190115       up-to-date
DB                                        18.5.0.0.190115       up-to-date

Querying the DB will show that sql patch registry is updated.

select PATCH_ID,PATCH_TYPE,ACTION,STATUS,SOURCE_VERSION,TARGET_VERSION,ACTION_TIME from dba_registry_sqlpatch;

  PATCH_ID PATCH_TYPE ACTION          STATUS                    SOURCE_VERSION  TARGET_VERSION  ACTION_TIME
---------- ---------- --------------- ------------------------- --------------- --------------- ------------------------------
  28090523 RU         APPLY           SUCCESS                   18.1.0.0.0      18.3.0.0.0      06-AUG-18 07.10.59.424957 PM
  27923415 INTERIM    APPLY           SUCCESS                   18.1.0.0.0      18.1.0.0.0      06-AUG-18 07.10.59.428991 PM
  28822489 RU         APPLY           SUCCESS                   18.3.0.0.0      18.5.0.0.0      12-MAR-19 11.55.48.034138 AM
  29143230 INTERIM    APPLY           SUCCESS                   18.5.0.0.0      18.5.0.0.0      12-MAR-19 12.00.42.612755 PM

The patched version of the DB System and DB was not updated on the console view. The console showed the pre-patch version for the DB system and DB. One way to rectify this was to run the patch from the console. This will not reapply the patch. But during the pre-patch check the applied patch will be detected and console will update itself with the current patch information.