Monday, October 4, 2021

Patching OS in DBCS VM DB Using DBCLI

There's an earlier post which shows updating and patching the OS. This method used yum to update the OS. It is still valid and available in documentation as well.
Recently Oracle introduced a new way to patch the OS, that is using the dbcli. The dbcli is the tool to use if patching the DB is done using command line. With this new enhancement same tool dbcli could be used to patching and updating the OS as well.
1. To check if any OS updates are available run the get-availableospatches command. If patches are not available updateAvailable will show false.
#  dbcli get-availableospatches -j
{
  "updateAvailable" : false,
  "rebootIsRequired" : false,
  "updateableRpms" : null,
  "installedRpms" : null
}
2. If updates are available then it will show updateAvailable true and also if a reboot is required after update.
dbcli get-availableospatches
Update Available     Reboot Required
-------------------- --------------------
Yes                  Yes
To get a list of rpms update use -j option.
bcli get-availableospatches -j
{
  "updateAvailable" : true,
  "rebootIsRequired" : true,
  "updateableRpms" : [ "bind.x86_64::32:9.11.4-26.P2.el7_9.5", "bind-export-libs.x86_64::32:9.11.4-26.P2.el7_9.5", "bind-libs.x86_64::32:9.11.4-26.P2.el7_9.5", "bind-libs-lite.x86_64::32:9.11.4-26.P2.el7_9.5", "bind-license.noarch::32:  9.11.4-26.P2.el7_9.5", "bind-utils.x86_64::32:9.11.4-26.P2.el7_9.5", "curl.x86_64::7.29.0-59.0.3.el7_9.1", "device-mapper.x86_64::7:1.02.170-6.0.5.el7_9.5", "device-mapper-event.x86_64::7:1.02.170-6.0.5.el7_9.5", "device-mapper-event-l  ibs.x86_64::7:1.02.170-6.0.5.el7_9.5", "device-mapper-libs.x86_64::7:1.02.170-6.0.5.el7_9.5", "device-mapper-multipath.x86_64::0.4.9-134.0.3.el7_9", "device-mapper-multipath-libs.x86_64::0.4.9-134.0.3.el7_9", "dhclient.x86_64::12:4.2.5  -83.0.1.el7_9.1", "dhcp.x86_64::12:4.2.5-83.0.1.el7_9.1", "dhcp-common.x86_64::12:4.2.5-83.0.1.el7_9.1", "dhcp-libs.x86_64::12:4.2.5-83.0.1.el7_9.1", "dmidecode.x86_64::1:3.2-5.0.1.el7_9.1", "glib2.x86_64::2.56.1-9.el7_9", "glibc.i686:  :2.17-324.0.1.el7_9", "glibc.x86_64::2.17-324.0.1.el7_9", "glibc-common.x86_64::2.17-324.0.1.el7_9", "glibc-devel.x86_64::2.17-324.0.1.el7_9", "glibc-headers.x86_64::2.17-324.0.1.el7_9", "grub2.x86_64::1:2.02-0.87.0.9.el7_9.6", "grub2-  common.noarch::1:2.02-0.87.0.9.el7_9.6", "grub2-efi-x64.x86_64::1:2.02-0.87.0.9.el7_9.6", "grub2-pc.x86_64::1:2.02-0.87.0.9.el7_9.6", "grub2-pc-modules.noarch::1:2.02-0.87.0.9.el7_9.6", "grub2-tools.x86_64::1:2.02-0.87.0.9.el7_9.6", "g  rub2-tools-extra.x86_64::1:2.02-0.87.0.9.el7_9.6", "grub2-tools-minimal.x86_64::1:2.02-0.87.0.9.el7_9.6", "gssproxy.x86_64::0.7.0-30.el7_9", "hwdata.x86_64::0.252-9.7.0.1.el7", "iscsi-initiator-utils.x86_64::6.2.0.874-20.0.5.el7_9", "i  scsi-initiator-utils-iscsiuio.x86_64::6.2.0.874-20.0.5.el7_9", "kernel-devel.x86_64::3.10.0-1160.36.2.el7", "kernel-headers.x86_64::3.10.0-1160.36.2.el7", "kernel-uek.x86_64::4.1.12-124.53.5.el7uek", "kernel-uek-firmware.noarch::4.1.12  -124.53.5.el7uek", "kexec-tools.x86_64::2.0.15-51.0.3.el7_9.3", "kpartx.x86_64::0.4.9-134.0.3.el7_9", "krb5-libs.x86_64::1.15.1-50.0.1.el7", "libcurl.x86_64::7.29.0-59.0.3.el7_9.1", "libgudev1.x86_64::219-78.0.5.el7_9.3", "libipa_hbac.  x86_64::1.16.5-10.0.1.el7_9.8", "libldb.x86_64::1.5.4-2.el7_9", "libsmbclient.x86_64::4.10.16-15.el7_9", "libsss_autofs.x86_64::1.16.5-10.0.1.el7_9.8", "libsss_certmap.x86_64::1.16.5-10.0.1.el7_9.8", "libsss_idmap.x86_64::1.16.5-10.0.1  .el7_9.8", "libsss_nss_idmap.x86_64::1.16.5-10.0.1.el7_9.8", "libsss_sudo.x86_64::1.16.5-10.0.1.el7_9.8", "libwbclient.x86_64::4.10.16-15.el7_9", "libxml2.i686::2.9.1-6.0.3.el7.5", "libxml2.x86_64::2.9.1-6.0.3.el7.5", "libxml2-python.x  86_64::2.9.1-6.0.3.el7.5", "linux-firmware.noarch::999:20210617-999.8.git0f66b74b.el7", "lvm2.x86_64::7:2.02.187-6.0.5.el7_9.5", "lvm2-libs.x86_64::7:2.02.187-6.0.5.el7_9.5", "mokutil.x86_64::15.3-1.0.5.el7", "nettle.x86_64::2.7.1-9.el  7_9", "nfs-utils.x86_64::1:1.3.0-0.68.0.1.el7.1", "nss.x86_64::3.53.1-7.el7_9", "nss-sysinit.x86_64::3.53.1-7.el7_9", "nss-tools.x86_64::3.53.1-7.el7_9", "openldap.x86_64::2.4.44-23.el7_9", "openssl.x86_64::1:1.0.2k-21.0.3.el7_9", "ope  nssl-libs.x86_64::1:1.0.2k-21.0.3.el7_9", "oracle-database-preinstall-19c.x86_64::1.0-3.el7", "pciutils.x86_64::3.5.1-3.0.1.el7", "pciutils-libs.x86_64::3.5.1-3.0.1.el7", "python.x86_64::2.7.5-90.0.3.el7", "python-libs.x86_64::2.7.5-90  .0.3.el7", "python-sssdconfig.noarch::1.16.5-10.0.1.el7_9.8", "redhat-release-server.x86_64::1:7.9-6.0.1.el7_9", "rsyslog.x86_64::8.24.0-57.0.1.el7_9.1", "samba-client-libs.x86_64::4.10.16-15.el7_9", "samba-common.noarch::4.10.16-15.el  7_9", "samba-common-libs.x86_64::4.10.16-15.el7_9", "screen.x86_64::4.1.0-0.27.20120314git3c2946.el7_9", "selinux-policy.noarch::3.13.1-268.0.13.el7_9.2", "selinux-policy-targeted.noarch::3.13.1-268.0.13.el7_9.2", "shim-x64.x86_64::15.  3-1.0.5.el7", "sos.noarch::3.9-5.0.11.el7_9.6", "sssd.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-ad.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-client.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-common.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-common-pac.x  86_64::1.16.5-10.0.1.el7_9.8", "sssd-ipa.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-krb5.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-krb5-common.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-ldap.x86_64::1.16.5-10.0.1.el7_9.8", "sssd-proxy.x86_64::1.16.5  -10.0.1.el7_9.8", "systemd.x86_64::219-78.0.5.el7_9.3", "systemd-libs.x86_64::219-78.0.5.el7_9.3", "systemd-sysv.x86_64::219-78.0.5.el7_9.3", "unzip.x86_64::6.0-22.el7_9" ],
  "installedRpms" : [ "bind.x86_64::32:9.11.4-26.P2.el7_9.3", "bind-export-libs.x86_64::32:9.11.4-26.P2.el7_9.3", "bind-libs.x86_64::32:9.11.4-26.P2.el7_9.3", "bind-libs-lite.x86_64::32:9.11.4-26.P2.el7_9.3", "bind-license.noarch::32:9  .11.4-26.P2.el7_9.3", "bind-utils.x86_64::32:9.11.4-26.P2.el7_9.3", "curl.x86_64::7.29.0-59.0.1.el7_9.1", "device-mapper.x86_64::7:1.02.170-6.0.3.el7_9.3", "device-mapper-event.x86_64::7:1.02.170-6.0.3.el7_9.3", "7:1.02.170-6.0.3.el7_9  .3::@pdit_ol7_latest/$releasever", "device-mapper-libs.x86_64::7:1.02.170-6.0.3.el7_9.3", "0.4.9-134.0.1.el7_9::@pdit_ol7_latest/$releasever", "0.4.9-134.0.1.el7_9::@pdit_ol7_latest/$releasever", "dhclient.x86_64::12:4.2.5-82.0.1.el7",   "dhcp.x86_64::12:4.2.5-82.0.1.el7", "dhcp-common.x86_64::12:4.2.5-82.0.1.el7", "dhcp-libs.x86_64::12:4.2.5-82.0.1.el7", "dmidecode.x86_64::1:3.2-5.el7", "glib2.x86_64::2.56.1-8.el7", "glibc.i686::2.17-322.0.1.el7_9", "glibc.x86_64::2.  17-322.0.1.el7_9", "glibc-common.x86_64::2.17-322.0.1.el7_9", "glibc-devel.x86_64::2.17-322.0.1.el7_9", "glibc-headers.x86_64::2.17-322.0.1.el7_9", "grub2.x86_64::1:2.02-0.87.0.5.el7", "grub2-common.noarch::1:2.02-0.87.0.5.el7", "grub2  -efi-x64.x86_64::1:2.02-0.87.0.5.el7", "grub2-pc.x86_64::1:2.02-0.87.0.5.el7", "grub2-pc-modules.noarch::1:2.02-0.87.0.5.el7", "grub2-tools.x86_64::1:2.02-0.87.0.5.el7", "grub2-tools-extra.x86_64::1:2.02-0.87.0.5.el7", "grub2-tools-min  imal.x86_64::1:2.02-0.87.0.5.el7", "gssproxy.x86_64::0.7.0-29.el7", "hwdata.x86_64::0.252-9.7.el7", "6.2.0.874-20.0.1.el7_9::@pdit_ol7_latest/$releasever", "6.2.0.874-20.0.1.el7_9::@pdit_ol7_latest/$releasever", "kernel-devel.x86_64::3  .10.0-1160.15.2.el7", "kernel-headers.x86_64::3.10.0-1160.15.2.el7", "kernel-uek.x86_64::4.1.12-124.47.3.el7uek", "kernel-uek-firmware.noarch::4.1.12-124.47.3.el7uek", "kexec-tools.x86_64::2.0.15-51.0.3.el7_9.1", "kpartx.x86_64::0.4.9-  134.0.1.el7_9", "krb5-libs.x86_64::1.15.1-50.el7", "libcurl.x86_64::7.29.0-59.0.1.el7_9.1", "libgudev1.x86_64::219-78.0.1.el7_9.3", "libipa_hbac.x86_64::1.16.5-10.0.1.el7_9.7", "libldb.x86_64::1.5.4-1.el7", "libsmbclient.x86_64::4.10.1  6-9.el7_9", "libsss_autofs.x86_64::1.16.5-10.0.1.el7_9.7", "libsss_certmap.x86_64::1.16.5-10.0.1.el7_9.7", "libsss_idmap.x86_64::1.16.5-10.0.1.el7_9.7", "libsss_nss_idmap.x86_64::1.16.5-10.0.1.el7_9.7", "libsss_sudo.x86_64::1.16.5-10.0  .1.el7_9.7", "libwbclient.x86_64::4.10.16-9.el7_9", "libxml2.i686::2.9.1-6.0.1.el7.5", "libxml2.x86_64::2.9.1-6.0.1.el7.5", "libxml2-python.x86_64::2.9.1-6.0.1.el7.5", "linux-firmware.noarch::999:20200902-999.5.gitd5f9eea5.el7", "lvm2.  x86_64::7:2.02.187-6.0.3.el7_9.3", "lvm2-libs.x86_64::7:2.02.187-6.0.3.el7_9.3", "mokutil.x86_64::15-2.0.9.el7", "nettle.x86_64::2.7.1-8.el7", "nfs-utils.x86_64::1:1.3.0-0.68.0.1.el7", "nss.x86_64::3.53.1-3.el7_9", "nss-sysinit.x86_64:  :3.53.1-3.el7_9", "nss-tools.x86_64::3.53.1-3.el7_9", "openldap.x86_64::2.4.44-22.el7", "openssl.x86_64::1:1.0.2k-21.el7_9", "openssl-libs.x86_64::1:1.0.2k-21.el7_9", "1.0-2.el7::@pdit_ol7_latest/$releasever", "pciutils.x86_64::3.5.1-3  .el7", "pciutils-libs.x86_64::3.5.1-3.el7", "python.x86_64::2.7.5-90.0.1.el7", "python-libs.x86_64::2.7.5-90.0.1.el7", "python-sssdconfig.noarch::1.16.5-10.0.1.el7_9.7", "1:7.9-3.0.1.el7::@pdit_ol7_latest/$releasever", "rsyslog.x86_64:  :8.24.0-57.el7_9", "samba-client-libs.x86_64::4.10.16-9.el7_9", "samba-common.noarch::4.10.16-9.el7_9", "samba-common-libs.x86_64::4.10.16-9.el7_9", "screen.x86_64::4.1.0-0.26.20120314git3c2946.el7", "selinux-policy.noarch::3.13.1-268.  0.1.el7_9.2", "3.13.1-268.0.1.el7_9.2::@pdit_ol7_latest/$releasever", "shim-x64.x86_64::15-2.0.9.el7", "sos.noarch::3.9-5.0.3.el7_9.2", "sssd.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-ad.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-client.x86_64:  :1.16.5-10.0.1.el7_9.7", "sssd-common.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-common-pac.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-ipa.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-krb5.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-krb5-common.x86_64::1.16.  5-10.0.1.el7_9.7", "sssd-ldap.x86_64::1.16.5-10.0.1.el7_9.7", "sssd-proxy.x86_64::1.16.5-10.0.1.el7_9.7", "systemd.x86_64::219-78.0.1.el7_9.3", "systemd-libs.x86_64::219-78.0.1.el7_9.3", "systemd-sysv.x86_64::219-78.0.1.el7_9.3", "unzi  p.x86_64::6.0-21.el7" ]
}
3. Note the kernel before the update.
uname -r
4.1.12-124.47.3.el7uek.x86_64
4. Run the pre-update check for the os.
dbcli update-server -c os -p
{
  "jobId" : "9b02cf04-92a2-4299-8cd0-4ce649b962ec",
  "status" : "Created",
  "message" : null,
  "reports" : [ ],
  "createTimestamp" : "September 24, 2021 14:18:46 PM BST",
  "resourceList" : [ ],
  "description" : "OS Patching Prechecks",
  "updatedTime" : "September 24, 2021 14:18:47 PM BST",
  "percentageProgress" : "0%",
  "cause" : null,
  "action" : null
}
Check if pre-udpate check is successful.
dbcli describe-job -i "9b02cf04-92a2-4299-8cd0-4ce649b962ec"

Job details
----------------------------------------------------------------
                     ID:  9b02cf04-92a2-4299-8cd0-4ce649b962ec
            Description:  OS Patching Prechecks
                 Status:  Success
                Created:  September 24, 2021 2:18:46 PM BST
               Progress:  100%
                Message:

Task Name                                                                Start Time                          End Time                            Status
------------------------------------------------------------------------ ----------------------------------- ----------------------------------- ----------


5. Run the os update.
dbcli update-server -c os
{
  "jobId" : "a7097060-6234-401d-bfd7-1c1fcbb9de85",
  "status" : "Created",
  "message" : null,
  "reports" : [ ],
  "createTimestamp" : "September 24, 2021 15:41:34 PM BST",
  "resourceList" : [ ],
  "description" : "OS Patching",
  "updatedTime" : "September 24, 2021 15:41:34 PM BST",
  "percentageProgress" : "0%",
  "cause" : null,
  "action" : null
}
The first run of the OS update failed with below output..
dbcli describe-job -i "a7097060-6234-401d-bfd7-1c1fcbb9de85"

Job details
----------------------------------------------------------------
                     ID:  a7097060-6234-401d-bfd7-1c1fcbb9de85
            Description:  OS Patching
                 Status:  Failure
                Created:  September 24, 2021 3:41:34 PM BST
               Progress:  78%
                Message:  DCS-10206:Patch bundle conflicts with: Yum Failed to resolve conflict
                  Cause:  Conflict check failed.
                 Action:  Rollback the conflicting patches and retry the operation.

Task Name                                                                Start Time                          End Time                            Status
------------------------------------------------------------------------ ----------------------------------- ----------------------------------- ----------
OS Component Patching                                                    September 24, 2021 3:41:34 PM BST   September 24, 2021 4:22:04 PM BST   Failure
task:TaskSequential_919                                                  September 24, 2021 3:56:45 PM BST   September 24, 2021 4:22:04 PM BST   Failure
Validating OS Patches                                                    September 24, 2021 3:56:45 PM BST   September 24, 2021 4:22:04 PM BST   Failure

After a SR Oracle support confimed that updating is hitting a bug (internal 33403308). After the bug was fixed by Oracle development team ran the command again and OS update was successful.
dbcli update-server -c os
{
  "jobId" : "b1ad604f-7961-4854-a05d-9e43958312b7",
  "status" : "Created",
  "message" : null,
  "reports" : [ ],
  "createTimestamp" : "September 29, 2021 09:32:50 AM BST",
  "resourceList" : [ ],
  "description" : "OS Patching",
  "updatedTime" : "September 29, 2021 09:32:51 AM BST",
  "percentageProgress" : "0%",
  "cause" : null,
  "action" : null
}

dbcli describe-job -i "b1ad604f-7961-4854-a05d-9e43958312b7"

Job details
----------------------------------------------------------------
                     ID:  b1ad604f-7961-4854-a05d-9e43958312b7
            Description:  OS Patching
                 Status:  Success
                Created:  September 29, 2021 9:32:50 AM BST
               Progress:  100%
                Message:

Task Name                                                                Start Time                          End Time                            Status
------------------------------------------------------------------------ ----------------------------------- ----------------------------------- ----------
The issue seems to have been fixed permanently as subsequent OS patching of other DB systems didn't have any issue.

6. Reboot the node and verify the kernel is updated.
uname -r
4.1.12-124.53.5.el7uek.x86_

Related Post
Updating the OS and Patching OCI PaaS DB Using DBCLI