This post is not a step by step guide to installing 18c RAC but list the main differences in the installation process compared to previous versions and things to look out for. Similar to
12cR2 clusterware the 18c cluster installation is no longer done through runInstaller. The downloaded Oracle Grid Infrastructure image files are unzipped into the folder path where GI is to be installed. This only need to be done on the local node only. During installation, the software is copied and installed on all other nodes in the cluster.
Unlike
12cR1, it is possible to specify a separate disk group for GIMR at the installation time. Oracle documentation says "You cannot migrate the GIMR from one disk group to another later". 12cR1 also mentioned the same but there were ways to
move GIMR to different disk group after the installation. It has been not verified whether the same techniques could be used for 18c as well.
1. The OS version and kernel versions used for this RAC setup is shown below. Unlike the previous post on 12cR2 this post for 18c uses RHEL7
cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.4 (Maipo)
uname -r
3.10.0-693.el7.x86_64
2. Create the user groups and users similar to previous 12.1 installation. As said in an
earlier post, 12.2 introduce a new user group for RAC management. Following from install guide "You must designate a group as the OSRACDBA group during database installation. Members of this group are granted the SYSRAC privileges to perform day–to–day administration of Oracle databases on an Oracle RAC cluster".
groupadd dba
groupadd oinstall
groupadd oper
groupadd asmoper
groupadd asmdba
groupadd asmadmin
groupadd backupdba
groupadd dgdba
groupadd kmdba
groupadd racdba
useradd -g oinstall -G dba,oper,asmdba,backupdba,dgdba,kmdba,racdba oracle
useradd -g oinstall -G asmadmin,asmdba,asmoper,dba grid
For user equivelancy ensure that user id and group id are same across the cluster nodes. The option -g could be used to ensure that same user/group id is used for creating users across all nodes. The nodes in this case are named rhel71 and rhel72.
[oracle@rhel71 ~]$ id oracle
uid=501(oracle) gid=501(oinstall) groups=501(oinstall),1000(dba),1002(oper),1004(asmdba),1006(backupdba),1007(dgdba),1008(kmdba),1009(racdba)
[oracle@rhel71 ~]$ ssh rhel72 id oracle
uid=501(oracle) gid=501(oinstall) groups=501(oinstall),1000(dba),1002(oper),1004(asmdba),1006(backupdba),1007(dgdba),1008(kmdba),1009(racdba)
[oracle@rhel71 ~]$ id grid
uid=502(grid) gid=501(oinstall) groups=501(oinstall),1000(dba),1003(asmoper),1004(asmdba),1005(asmadmin)
[oracle@rhel71 ~]$ ssh rhel72 id grid
uid=502(grid) gid=501(oinstall) groups=501(oinstall),1000(dba),1003(asmoper),1004(asmdba),1005(asmadmin)
3. If a separate disk group is used for GIMR then it needs to be at least 35GB in size. Following from installation doc "
during installation of an Oracle Standalone Cluster, if you create the MGMT disk group for Grid Infrastructure Management Repository (GIMR), then the installer requires that you use a disk group with at least 35 GB of available space". A 40GB disk was created to store the GIMR and udev rules were created for granting necessary ownership and permission.
cat /etc/udev/rules.d/99-sd.rules
KERNEL=="sd*1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",RESULT=="1ATA_VBOX_HARDDISK_VB1a5c8787-e082e7a7",SYMLINK+="oracleasm/ocr1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",RESULT=="1ATA_VBOX_HARDDISK_VB1a72679c-ab23e15f",SYMLINK+="oracleasm/ocr2",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",RESULT=="1ATA_VBOX_HARDDISK_VB8f43e75c-3e0bec6d",SYMLINK+="oracleasm/ocr3",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",RESULT=="1ATA_VBOX_HARDDISK_VBf8475ab7-d401ec8b",SYMLINK+="oracleasm/gimr",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",RESULT=="1ATA_VBOX_HARDDISK_VB34650f33-5ecdc029",SYMLINK+="oracleasm/data1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent",RESULT=="1ATA_VBOX_HARDDISK_VBdb057b08-1733da4b",SYMLINK+="oracleasm/fra1",OWNER="grid",GROUP="asmadmin",MODE="0660"
4. Create the GI installation directory and unzip the downloaded install media into the directory.
mkdir -p /opt/app/18.x.0/grid
chown grid:oinstall /opt/app/18.x.0/grid
unzip ./LINUX.X64_180000_grid_home.zip -d /opt/app/18.x.0/grid
5. Run cluvfy to check the pre-reqs checks. The failure related to shm shown at the end could be ignored as per
2065603.1 and
25907259.8[grid@rhel71 grid]$ ./runcluvfy.sh stage -pre crsinst -n rhel71,rhel72 -r 18 -osdba asmdba -osoper asmoper -asm -asmgrp asmadmin -presence local -asmdev /dev/oracleasm/data1,/dev/oracleasm/fra1,/dev/oracleasm/ocr1,/dev/oracleasm/ocr2,/dev/oracleasm/ocr3,/dev/oracleasm/gimr -crshome /opt/app/18.x.0/grid -fixup -verbose
Verifying Physical Memory ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 2.7806GB (2915620.0KB) 8GB (8388608.0KB) failed
rhel71 2.7806GB (2915620.0KB) 8GB (8388608.0KB) failed
Verifying Physical Memory ...FAILED (PRVF-7530)
Verifying Available Physical Memory ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 2.6736GB (2803492.0KB) 50MB (51200.0KB) passed
rhel71 2.4872GB (2608032.0KB) 50MB (51200.0KB) passed
Verifying Available Physical Memory ...PASSED
Verifying Swap Size ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 3.7246GB (3905532.0KB) 2.7806GB (2915620.0KB) passed
rhel71 3.7246GB (3905532.0KB) 2.7806GB (2915620.0KB) passed
Verifying Swap Size ...PASSED
Verifying Free Space: rhel72:/usr,rhel72:/var,rhel72:/etc,rhel72:/opt/app/18.x.0/grid,rhel72:/sbin,rhel72:/tmp ...
Path Node Name Mount point Available Required Status
---------------- ------------ ------------ ------------ ------------ ------------
/usr rhel72 / 28.9385GB 25MB passed
/var rhel72 / 28.9385GB 5MB passed
/etc rhel72 / 28.9385GB 25MB passed
/opt/app/18.x.0/grid rhel72 / 28.9385GB 6.9GB passed
/sbin rhel72 / 28.9385GB 10MB passed
/tmp rhel72 / 28.9385GB 1GB passed
Verifying Free Space: rhel72:/usr,rhel72:/var,rhel72:/etc,rhel72:/opt/app/18.x.0/grid,rhel72:/sbin,rhel72:/tmp ...PASSED
Verifying Free Space: rhel71:/usr,rhel71:/var,rhel71:/etc,rhel71:/opt/app/18.x.0/grid,rhel71:/sbin,rhel71:/tmp ...
Path Node Name Mount point Available Required Status
---------------- ------------ ------------ ------------ ------------ ------------
/usr rhel71 / 18.9439GB 25MB passed
/var rhel71 / 18.9439GB 5MB passed
/etc rhel71 / 18.9439GB 25MB passed
/opt/app/18.x.0/grid rhel71 / 18.9439GB 6.9GB passed
/sbin rhel71 / 18.9439GB 10MB passed
/tmp rhel71 / 18.9439GB 1GB passed
Verifying Free Space: rhel71:/usr,rhel71:/var,rhel71:/etc,rhel71:/opt/app/18.x.0/grid,rhel71:/sbin,rhel71:/tmp ...PASSED
Verifying User Existence: grid ...
Node Name Status Comment
------------ ------------------------ ------------------------
rhel72 passed exists(502)
rhel71 passed exists(502)
Verifying Users With Same UID: 502 ...PASSED
Verifying User Existence: grid ...PASSED
Verifying Group Existence: asmadmin ...
Node Name Status Comment
------------ ------------------------ ------------------------
rhel72 passed exists
rhel71 passed exists
Verifying Group Existence: asmadmin ...PASSED
Verifying Group Existence: asmoper ...
Node Name Status Comment
------------ ------------------------ ------------------------
rhel72 passed exists
rhel71 passed exists
Verifying Group Existence: asmoper ...PASSED
Verifying Group Existence: asmdba ...
Node Name Status Comment
------------ ------------------------ ------------------------
rhel72 passed exists
rhel71 passed exists
Verifying Group Existence: asmdba ...PASSED
Verifying Group Existence: oinstall ...
Node Name Status Comment
------------ ------------------------ ------------------------
rhel72 passed exists
rhel71 passed exists
Verifying Group Existence: oinstall ...PASSED
Verifying Group Membership: asmdba ...
Node Name User Exists Group Exists User in Group Status
---------------- ------------ ------------ ------------ ----------------
rhel72 yes yes yes passed
rhel71 yes yes yes passed
Verifying Group Membership: asmdba ...PASSED
Verifying Group Membership: asmadmin ...
Node Name User Exists Group Exists User in Group Status
---------------- ------------ ------------ ------------ ----------------
rhel72 yes yes yes passed
rhel71 yes yes yes passed
Verifying Group Membership: asmadmin ...PASSED
Verifying Group Membership: oinstall(Primary) ...
Node Name User Exists Group Exists User in Group Primary Status
---------------- ------------ ------------ ------------ ------------ ------------
rhel72 yes yes yes yes passed
rhel71 yes yes yes yes passed
Verifying Group Membership: oinstall(Primary) ...PASSED
Verifying Group Membership: asmoper ...
Node Name User Exists Group Exists User in Group Status
---------------- ------------ ------------ ------------ ----------------
rhel72 yes yes yes passed
rhel71 yes yes yes passed
Verifying Group Membership: asmoper ...PASSED
Verifying Run Level ...
Node Name run level Required Status
------------ ------------------------ ------------------------ ----------
rhel72 3 3,5 passed
rhel71 3 3,5 passed
Verifying Run Level ...PASSED
Verifying Hard Limit: maximum open file descriptors ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
rhel72 hard 65536 65536 passed
rhel71 hard 65536 65536 passed
Verifying Hard Limit: maximum open file descriptors ...PASSED
Verifying Soft Limit: maximum open file descriptors ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
rhel72 soft 65536 1024 passed
rhel71 soft 65536 1024 passed
Verifying Soft Limit: maximum open file descriptors ...PASSED
Verifying Hard Limit: maximum user processes ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
rhel72 hard 16384 16384 passed
rhel71 hard 16384 16384 passed
Verifying Hard Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum user processes ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
rhel72 soft 16384 2047 passed
rhel71 soft 16384 2047 passed
Verifying Soft Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum stack size ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
rhel72 soft 10240 10240 passed
rhel71 soft 10240 10240 passed
Verifying Soft Limit: maximum stack size ...PASSED
Verifying Architecture ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 x86_64 x86_64 passed
rhel71 x86_64 x86_64 passed
Verifying Architecture ...PASSED
Verifying OS Kernel Version ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 3.10.0-693.el7.x86_64 3.10.0 passed
rhel71 3.10.0-693.el7.x86_64 3.10.0 passed
Verifying OS Kernel Version ...PASSED
Verifying OS Kernel Parameter: semmsl ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 3010 3010 250 passed
rhel72 3010 3010 250 passed
Verifying OS Kernel Parameter: semmsl ...PASSED
Verifying OS Kernel Parameter: semmns ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 385280 385280 32000 passed
rhel72 385280 385280 32000 passed
Verifying OS Kernel Parameter: semmns ...PASSED
Verifying OS Kernel Parameter: semopm ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 3010 3010 100 passed
rhel72 3010 3010 100 passed
Verifying OS Kernel Parameter: semopm ...PASSED
Verifying OS Kernel Parameter: semmni ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 128 128 128 passed
rhel72 128 128 128 passed
Verifying OS Kernel Parameter: semmni ...PASSED
Verifying OS Kernel Parameter: shmmax ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 4398046511104 4398046511104 1492797440 passed
rhel72 4398046511104 4398046511104 1492797440 passed
Verifying OS Kernel Parameter: shmmax ...PASSED
Verifying OS Kernel Parameter: shmmni ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 4096 4096 4096 passed
rhel72 4096 4096 4096 passed
Verifying OS Kernel Parameter: shmmni ...PASSED
Verifying OS Kernel Parameter: shmall ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 1073741824 1073741824 1073741824 passed
rhel72 1073741824 1073741824 1073741824 passed
Verifying OS Kernel Parameter: shmall ...PASSED
Verifying OS Kernel Parameter: file-max ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 6815744 6815744 6815744 passed
rhel72 6815744 6815744 6815744 passed
Verifying OS Kernel Parameter: file-max ...PASSED
Verifying OS Kernel Parameter: ip_local_port_range ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 between 9000 & 65500 between 9000 & 65500 between 9000 & 65535 passed
rhel72 between 9000 & 65500 between 9000 & 65500 between 9000 & 65535 passed
Verifying OS Kernel Parameter: ip_local_port_range ...PASSED
Verifying OS Kernel Parameter: rmem_default ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 4194304 4194304 262144 passed
rhel72 4194304 4194304 262144 passed
Verifying OS Kernel Parameter: rmem_default ...PASSED
Verifying OS Kernel Parameter: rmem_max ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 4194304 4194304 4194304 passed
rhel72 4194304 4194304 4194304 passed
Verifying OS Kernel Parameter: rmem_max ...PASSED
Verifying OS Kernel Parameter: wmem_default ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 1048576 1048576 262144 passed
rhel72 1048576 1048576 262144 passed
Verifying OS Kernel Parameter: wmem_default ...PASSED
Verifying OS Kernel Parameter: wmem_max ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 2097152 2097152 1048576 passed
rhel72 2097152 2097152 1048576 passed
Verifying OS Kernel Parameter: wmem_max ...PASSED
Verifying OS Kernel Parameter: aio-max-nr ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
rhel71 3145728 3145728 1048576 passed
rhel72 3145728 3145728 1048576 passed
Verifying OS Kernel Parameter: aio-max-nr ...PASSED
Verifying Package: binutils-2.23.52.0.1 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 binutils-2.25.1-31.base.el7 binutils-2.23.52.0.1 passed
rhel71 binutils-2.25.1-31.base.el7 binutils-2.23.52.0.1 passed
Verifying Package: binutils-2.23.52.0.1 ...PASSED
Verifying Package: compat-libcap1-1.10 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 compat-libcap1-1.10-7.el7 compat-libcap1-1.10 passed
rhel71 compat-libcap1-1.10-7.el7 compat-libcap1-1.10 passed
Verifying Package: compat-libcap1-1.10 ...PASSED
Verifying Package: libgcc-4.8.2 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 libgcc(x86_64)-4.8.5-16.el7 libgcc(x86_64)-4.8.2 passed
rhel71 libgcc(x86_64)-4.8.5-16.el7 libgcc(x86_64)-4.8.2 passed
Verifying Package: libgcc-4.8.2 (x86_64) ...PASSED
Verifying Package: libstdc++-4.8.2 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 libstdc++(x86_64)-4.8.5-16.el7 libstdc++(x86_64)-4.8.2 passed
rhel71 libstdc++(x86_64)-4.8.5-16.el7 libstdc++(x86_64)-4.8.2 passed
Verifying Package: libstdc++-4.8.2 (x86_64) ...PASSED
Verifying Package: libstdc++-devel-4.8.2 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 libstdc++-devel(x86_64)-4.8.5-16.el7 libstdc++-devel(x86_64)-4.8.2 passed
rhel71 libstdc++-devel(x86_64)-4.8.5-16.el7 libstdc++-devel(x86_64)-4.8.2 passed
Verifying Package: libstdc++-devel-4.8.2 (x86_64) ...PASSED
Verifying Package: sysstat-10.1.5 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 sysstat-10.1.5-12.el7 sysstat-10.1.5 passed
rhel71 sysstat-10.1.5-12.el7 sysstat-10.1.5 passed
Verifying Package: sysstat-10.1.5 ...PASSED
Verifying Package: gcc-c++-4.8.2 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 gcc-c++-4.8.5-16.el7 gcc-c++-4.8.2 passed
rhel71 gcc-c++-4.8.5-16.el7 gcc-c++-4.8.2 passed
Verifying Package: gcc-c++-4.8.2 ...PASSED
Verifying Package: ksh ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 ksh ksh passed
rhel71 ksh ksh passed
Verifying Package: ksh ...PASSED
Verifying Package: make-3.82 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 make-3.82-23.el7 make-3.82 passed
rhel71 make-3.82-23.el7 make-3.82 passed
Verifying Package: make-3.82 ...PASSED
Verifying Package: glibc-2.17 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 glibc(x86_64)-2.17-196.el7 glibc(x86_64)-2.17 passed
rhel71 glibc(x86_64)-2.17-196.el7 glibc(x86_64)-2.17 passed
Verifying Package: glibc-2.17 (x86_64) ...PASSED
Verifying Package: glibc-devel-2.17 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 glibc-devel(x86_64)-2.17-196.el7 glibc-devel(x86_64)-2.17 passed
rhel71 glibc-devel(x86_64)-2.17-196.el7 glibc-devel(x86_64)-2.17 passed
Verifying Package: glibc-devel-2.17 (x86_64) ...PASSED
Verifying Package: libaio-0.3.109 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 libaio(x86_64)-0.3.109-13.el7 libaio(x86_64)-0.3.109 passed
rhel71 libaio(x86_64)-0.3.109-13.el7 libaio(x86_64)-0.3.109 passed
Verifying Package: libaio-0.3.109 (x86_64) ...PASSED
Verifying Package: libaio-devel-0.3.109 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 libaio-devel(x86_64)-0.3.109-13.el7 libaio-devel(x86_64)-0.3.109 passed
rhel71 libaio-devel(x86_64)-0.3.109-13.el7 libaio-devel(x86_64)-0.3.109 passed
Verifying Package: libaio-devel-0.3.109 (x86_64) ...PASSED
Verifying Package: nfs-utils-1.2.3-15 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 nfs-utils-1.3.0-0.48.el7 nfs-utils-1.2.3-15 passed
rhel71 nfs-utils-1.3.0-0.48.el7 nfs-utils-1.2.3-15 passed
Verifying Package: nfs-utils-1.2.3-15 ...PASSED
Verifying Package: smartmontools-6.2-4 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 smartmontools-6.2-8.el7 smartmontools-6.2-4 passed
rhel71 smartmontools-6.2-8.el7 smartmontools-6.2-4 passed
Verifying Package: smartmontools-6.2-4 ...PASSED
Verifying Package: net-tools-2.0-0.17 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 net-tools-2.0-0.22.20131004git.el7 net-tools-2.0-0.17 passed
rhel71 net-tools-2.0-0.22.20131004git.el7 net-tools-2.0-0.17 passed
Verifying Package: net-tools-2.0-0.17 ...PASSED
Verifying Package: compat-libstdc++-33-3.2.3 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 compat-libstdc++-33(x86_64)-3.2.3-72.el7 compat-libstdc++-33(x86_64)-3.2.3 passed
rhel71 compat-libstdc++-33(x86_64)-3.2.3-72.el7 compat-libstdc++-33(x86_64)-3.2.3 passed
Verifying Package: compat-libstdc++-33-3.2.3 (x86_64) ...PASSED
Verifying Package: libxcb-1.11 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 libxcb(x86_64)-1.12-1.el7 libxcb(x86_64)-1.11 passed
rhel71 libxcb(x86_64)-1.12-1.el7 libxcb(x86_64)-1.11 passed
Verifying Package: libxcb-1.11 (x86_64) ...PASSED
Verifying Package: libX11-1.6.3 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 libX11(x86_64)-1.6.5-1.el7 libX11(x86_64)-1.6.3 passed
rhel71 libX11(x86_64)-1.6.5-1.el7 libX11(x86_64)-1.6.3 passed
Verifying Package: libX11-1.6.3 (x86_64) ...PASSED
Verifying Package: libXau-1.0.8 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 libXau(x86_64)-1.0.8-2.1.el7 libXau(x86_64)-1.0.8 passed
rhel71 libXau(x86_64)-1.0.8-2.1.el7 libXau(x86_64)-1.0.8 passed
Verifying Package: libXau-1.0.8 (x86_64) ...PASSED
Verifying Package: libXi-1.7.4 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 libXi(x86_64)-1.7.9-1.el7 libXi(x86_64)-1.7.4 passed
rhel71 libXi(x86_64)-1.7.9-1.el7 libXi(x86_64)-1.7.4 passed
Verifying Package: libXi-1.7.4 (x86_64) ...PASSED
Verifying Package: libXtst-1.2.2 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 libXtst(x86_64)-1.2.3-1.el7 libXtst(x86_64)-1.2.2 passed
rhel71 libXtst(x86_64)-1.2.3-1.el7 libXtst(x86_64)-1.2.2 passed
Verifying Package: libXtst-1.2.2 (x86_64) ...PASSED
Verifying Port Availability for component "Oracle Notification Service (ONS)" ...
Node Name Port Number Protocol Available Status
---------------- ------------ ------------ ------------ ----------------
rhel72 6200 TCP yes successful
rhel71 6200 TCP yes successful
rhel72 6100 TCP yes successful
rhel71 6100 TCP yes successful
Verifying Port Availability for component "Oracle Notification Service (ONS)" ...PASSED
Verifying Port Availability for component "Oracle Cluster Synchronization Services (CSSD)" ...
Node Name Port Number Protocol Available Status
---------------- ------------ ------------ ------------ ----------------
rhel72 42424 TCP yes successful
rhel71 42424 TCP yes successful
Verifying Port Availability for component "Oracle Cluster Synchronization Services (CSSD)" ...PASSED
Verifying Users With Same UID: 0 ...PASSED
Verifying Current Group ID ...PASSED
Verifying Root user consistency ...
Node Name Status
------------------------------------ ------------------------
rhel72 passed
rhel71 passed
Verifying Root user consistency ...PASSED
Verifying Package: cvuqdisk-1.0.10-1 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
rhel72 cvuqdisk-1.0.10-1 cvuqdisk-1.0.10-1 passed
rhel71 cvuqdisk-1.0.10-1 cvuqdisk-1.0.10-1 passed
Verifying Package: cvuqdisk-1.0.10-1 ...PASSED
Verifying Host name ...PASSED
Verifying Node Connectivity ...
Verifying Hosts File ...
Node Name Status
------------------------------------ ------------------------
rhel71 passed
rhel72 passed
Verifying Hosts File ...PASSED
Interface information for node "rhel71"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
enp0s3 192.168.0.93 192.168.0.0 0.0.0.0 192.168.0.102 08:00:27:BD:32:39 1500
enp0s8 192.168.1.93 192.168.1.0 0.0.0.0 192.168.0.102 08:00:27:BD:A9:0A 1500
Interface information for node "rhel72"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
enp0s3 192.168.0.94 192.168.0.0 0.0.0.0 192.168.0.102 08:00:27:31:0F:1F 1500
enp0s8 192.168.1.94 192.168.1.0 0.0.0.0 192.168.0.102 08:00:27:98:22:A6 1500
Check: MTU consistency of the subnet "192.168.1.0".
Node Name IP Address Subnet MTU
---------------- ------------ ------------ ------------ ----------------
rhel71 enp0s8 192.168.1.93 192.168.1.0 1500
rhel72 enp0s8 192.168.1.94 192.168.1.0 1500
Check: MTU consistency of the subnet "192.168.0.0".
Node Name IP Address Subnet MTU
---------------- ------------ ------------ ------------ ----------------
rhel71 enp0s3 192.168.0.93 192.168.0.0 1500
rhel72 enp0s3 192.168.0.94 192.168.0.0 1500
Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED
Source Destination Connected?
------------------------------ ------------------------------ ----------------
rhel71[enp0s8:192.168.1.93] rhel72[enp0s8:192.168.1.94] yes
Source Destination Connected?
------------------------------ ------------------------------ ----------------
rhel71[enp0s3:192.168.0.93] rhel72[enp0s3:192.168.0.94] yes
Verifying subnet mask consistency for subnet "192.168.1.0" ...PASSED
Verifying subnet mask consistency for subnet "192.168.0.0" ...PASSED
Verifying Node Connectivity ...PASSED
Verifying Multicast or broadcast check ...
Checking subnet "192.168.1.0" for multicast communication with multicast group "224.0.0.251"
Verifying Multicast or broadcast check ...PASSED
Verifying Device Checks for ASM ...
Verifying Package: cvuqdisk-1.0.10-1 ...PASSED
Verifying ASM device sharedness check ...
Verifying Shared Storage Accessibility:/dev/oracleasm/gimr,/dev/oracleasm/fra1,/dev/oracleasm/data1,/dev/oracleasm/ocr3,/dev/oracleasm/ocr1,/dev/oracleasm/ocr2 ...PASSED
Device Device Type
------------------------------------ ------------------------
/dev/oracleasm/ocr3 Disk
/dev/oracleasm/data1 Disk
/dev/oracleasm/ocr2 Disk
/dev/oracleasm/fra1 Disk
/dev/oracleasm/ocr1 Disk
/dev/oracleasm/gimr Disk
Verifying ASM device sharedness check ...PASSED
Verifying Access Control List check ...
Checking consistency of device owner across all nodes...
Consistency check of device owner for "/dev/oracleasm/gimr" PASSED
Consistency check of device owner for "/dev/oracleasm/fra1" PASSED
Consistency check of device owner for "/dev/oracleasm/ocr3" PASSED
Consistency check of device owner for "/dev/oracleasm/data1" PASSED
Consistency check of device owner for "/dev/oracleasm/ocr1" PASSED
Consistency check of device owner for "/dev/oracleasm/ocr2" PASSED
Checking consistency of device group across all nodes...
Consistency check of device group for "/dev/oracleasm/gimr" PASSED
Consistency check of device group for "/dev/oracleasm/fra1" PASSED
Consistency check of device group for "/dev/oracleasm/ocr3" PASSED
Consistency check of device group for "/dev/oracleasm/data1" PASSED
Consistency check of device group for "/dev/oracleasm/ocr1" PASSED
Consistency check of device group for "/dev/oracleasm/ocr2" PASSED
Checking consistency of device permissions across all nodes...
Consistency check of device permissions for "/dev/oracleasm/gimr" PASSED
Consistency check of device permissions for "/dev/oracleasm/fra1" PASSED
Consistency check of device permissions for "/dev/oracleasm/ocr3" PASSED
Consistency check of device permissions for "/dev/oracleasm/data1" PASSED
Consistency check of device permissions for "/dev/oracleasm/ocr1" PASSED
Consistency check of device permissions for "/dev/oracleasm/ocr2" PASSED
Verifying Access Control List check ...PASSED
Verifying I/O scheduler ...PASSED
Verifying Device Checks for ASM ...PASSED
Verifying Network Time Protocol (NTP) ...
Verifying '/etc/ntp.conf' ...
Node Name File exists?
------------------------------------ ------------------------
rhel72 no
rhel71 no
Verifying '/etc/ntp.conf' ...PASSED
Verifying '/etc/chrony.conf' ...
Node Name File exists?
------------------------------------ ------------------------
rhel72 no
rhel71 no
Verifying '/etc/chrony.conf' ...PASSED
Verifying '/var/run/ntpd.pid' ...
Node Name File exists?
------------------------------------ ------------------------
rhel72 no
rhel71 no
Verifying '/var/run/ntpd.pid' ...PASSED
Verifying '/var/run/chronyd.pid' ...
Node Name File exists?
------------------------------------ ------------------------
rhel72 no
rhel71 no
Verifying '/var/run/chronyd.pid' ...PASSED
Verifying Network Time Protocol (NTP) ...PASSED
Verifying Same core file name pattern ...PASSED
Verifying User Mask ...
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
rhel72 0022 0022 passed
rhel71 0022 0022 passed
Verifying User Mask ...PASSED
Verifying User Not In Group "root": grid ...
Node Name Status Comment
------------ ------------------------ ------------------------
rhel72 passed does not exist
rhel71 passed does not exist
Verifying User Not In Group "root": grid ...PASSED
Verifying Time zone consistency ...PASSED
Verifying Time offset between nodes ...PASSED
Verifying resolv.conf Integrity ...
Node Name Status
------------------------------------ ------------------------
rhel71 passed
rhel72 passed
checking response for name "rhel72" from each of the name servers specified in
"/etc/resolv.conf"
Node Name Source Comment Status
------------ ------------------------ ------------------------ ----------
rhel72 192.168.0.66 IPv4 passed
checking response for name "rhel71" from each of the name servers specified in
"/etc/resolv.conf"
Node Name Source Comment Status
------------ ------------------------ ------------------------ ----------
rhel71 192.168.0.66 IPv4 passed
Verifying resolv.conf Integrity ...PASSED
Verifying DNS/NIS name service ...PASSED
Verifying Domain Sockets ...PASSED
Verifying /boot mount ...PASSED
Verifying File system mount options for path GI_HOME ...PASSED
Verifying Daemon "avahi-daemon" not configured and running ...
Node Name Configured Status
------------ ------------------------ ------------------------
rhel72 no passed
rhel71 no passed
Node Name Running? Status
------------ ------------------------ ------------------------
rhel72 no passed
rhel71 no passed
Verifying Daemon "avahi-daemon" not configured and running ...PASSED
Verifying Daemon "proxyt" not configured and running ...
Node Name Configured Status
------------ ------------------------ ------------------------
rhel72 no passed
rhel71 no passed
Node Name Running? Status
------------ ------------------------ ------------------------
rhel72 no passed
rhel71 no passed
Verifying Daemon "proxyt" not configured and running ...PASSED
Verifying Grid Infrastructure home path: /opt/app/18.x.0/grid ...
Verifying '/opt/app/18.x.0/grid' ...PASSED
Verifying Grid Infrastructure home path: /opt/app/18.x.0/grid ...PASSED
Verifying User Equivalence ...PASSED
Verifying /dev/shm mounted as temporary file system ...FAILED (PRVE-0421)
Verifying File system mount options for path /var ...PASSED
Verifying zeroconf check ...PASSED
Verifying ASM Filter Driver configuration ...PASSED
Pre-check for cluster services setup was unsuccessful on all the nodes.
Failures were encountered during execution of CVU verification request "stage -pre crsinst".
Verifying Physical Memory ...FAILED
rhel72: PRVF-7530 : Sufficient physical memory is not available on node
"rhel72" [Required physical memory = 8GB (8388608.0KB)]
rhel71: PRVF-7530 : Sufficient physical memory is not available on node
"rhel71" [Required physical memory = 8GB (8388608.0KB)]
Verifying /dev/shm mounted as temporary file system ...FAILED
rhel72: PRVE-0421 : No entry exists in /etc/fstab for mounting /dev/shm
rhel71: PRVE-0421 : No entry exists in /etc/fstab for mounting /dev/shm6. From the GI_HOME directory run gridSetup.sh to begin the installation. Select new cluster configuration option.
Select standard cluster (this would create a flex cluster which is the default for all standard clusters).
Set cluster name and scan name. No
GNS is setup or used for this installation.
Add node and node VIP names that will be used in the cluster configuration. Importantly all node roles are set as HUB nodes.
Select the network interfaces for public and ASM/Private interconnect.
The OCR and vote disks are stored in ASM but ASM could be configured either block devices or using NFS. This setup uses block devices.
A separate disk group is used to store GIMR.
Create the disk group for OCR files.
Create disk group for GIMR
Set the oracle base location. The GI install location is immutable.
The failed pre-reqs are ignorable for test setup.
Summary
7. When prompted run the root script. Following output from running root.sh on the first node.
./root.sh
Performing root user operation.
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /opt/app/18.x.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
/opt/app/oracle/crsdata/rhel71/crsconfig/rootcrs_rhel71_2019-03-19_02-59-46PM.log
2019/03/19 15:00:12 CLSRSC-594: Executing installation step 1 of 20: 'SetupTFA'.
2019/03/19 15:00:12 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2019/03/19 15:00:56 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2019/03/19 15:00:56 CLSRSC-594: Executing installation step 2 of 20: 'ValidateEnv'.
2019/03/19 15:00:57 CLSRSC-363: User ignored prerequisites during installation
2019/03/19 15:00:57 CLSRSC-594: Executing installation step 3 of 20: 'CheckFirstNode'.
2019/03/19 15:01:01 CLSRSC-594: Executing installation step 4 of 20: 'GenSiteGUIDs'.
2019/03/19 15:01:07 CLSRSC-594: Executing installation step 5 of 20: 'SaveParamFile'.
2019/03/19 15:01:28 CLSRSC-594: Executing installation step 6 of 20: 'SetupOSD'.
2019/03/19 15:01:28 CLSRSC-594: Executing installation step 7 of 20: 'CheckCRSConfig'.
2019/03/19 15:01:28 CLSRSC-594: Executing installation step 8 of 20: 'SetupLocalGPNP'.
2019/03/19 15:02:17 CLSRSC-594: Executing installation step 9 of 20: 'CreateRootCert'.
2019/03/19 15:02:30 CLSRSC-594: Executing installation step 10 of 20: 'ConfigOLR'.
2019/03/19 15:02:58 CLSRSC-594: Executing installation step 11 of 20: 'ConfigCHMOS'.
2019/03/19 15:02:58 CLSRSC-594: Executing installation step 12 of 20: 'CreateOHASD'.
2019/03/19 15:03:17 CLSRSC-594: Executing installation step 13 of 20: 'ConfigOHASD'.
2019/03/19 15:03:17 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2019/03/19 15:04:47 CLSRSC-594: Executing installation step 14 of 20: 'InstallAFD'.
2019/03/19 15:05:05 CLSRSC-594: Executing installation step 15 of 20: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel71'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel71' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2019/03/19 15:06:37 CLSRSC-594: Executing installation step 16 of 20: 'InstallKA'.
2019/03/19 15:06:55 CLSRSC-594: Executing installation step 17 of 20: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel71'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel71' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.evmd' on 'rhel71'
CRS-2672: Attempting to start 'ora.mdnsd' on 'rhel71'
CRS-2676: Start of 'ora.mdnsd' on 'rhel71' succeeded
CRS-2676: Start of 'ora.evmd' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rhel71'
CRS-2676: Start of 'ora.gpnpd' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rhel71'
CRS-2672: Attempting to start 'ora.gipcd' on 'rhel71'
CRS-2676: Start of 'ora.cssdmonitor' on 'rhel71' succeeded
CRS-2676: Start of 'ora.gipcd' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rhel71'
CRS-2672: Attempting to start 'ora.diskmon' on 'rhel71'
CRS-2676: Start of 'ora.diskmon' on 'rhel71' succeeded
CRS-2676: Start of 'ora.cssd' on 'rhel71' succeeded
[INFO] [DBT-30001] Disk groups created successfully. Check /opt/app/oracle/cfgtoollogs/asmca/asmca-190319PM030741.log for details.
2019/03/19 15:08:59 CLSRSC-482: Running command: '/opt/app/18.x.0/grid/bin/ocrconfig -upgrade grid oinstall'
CRS-2672: Attempting to start 'ora.crf' on 'rhel71'
CRS-2672: Attempting to start 'ora.storage' on 'rhel71'
CRS-2676: Start of 'ora.storage' on 'rhel71' succeeded
CRS-2676: Start of 'ora.crf' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rhel71'
CRS-2676: Start of 'ora.crsd' on 'rhel71' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk c500a19f07ec4f62bf8ac318166a95ae.
Successful addition of voting disk 9100c10cb7bb4fd1bffbe97b8dc91245.
Successful addition of voting disk 300c2bf8b7094fa6bfaf11e49f826c7e.
Successfully replaced voting disk group with +OCRDG.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE c500a19f07ec4f62bf8ac318166a95ae (/dev/oracleasm/ocr1) [OCRDG]
2. ONLINE 9100c10cb7bb4fd1bffbe97b8dc91245 (/dev/oracleasm/ocr2) [OCRDG]
3. ONLINE 300c2bf8b7094fa6bfaf11e49f826c7e (/dev/oracleasm/ocr3) [OCRDG]
Located 3 voting disk(s).
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel71'
CRS-2673: Attempting to stop 'ora.crsd' on 'rhel71'
CRS-2677: Stop of 'ora.crsd' on 'rhel71' succeeded
CRS-2673: Attempting to stop 'ora.storage' on 'rhel71'
CRS-2673: Attempting to stop 'ora.crf' on 'rhel71'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rhel71'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rhel71'
CRS-2677: Stop of 'ora.crf' on 'rhel71' succeeded
CRS-2677: Stop of 'ora.storage' on 'rhel71' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rhel71'
CRS-2677: Stop of 'ora.drivers.acfs' on 'rhel71' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rhel71' succeeded
CRS-2677: Stop of 'ora.asm' on 'rhel71' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rhel71'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rhel71' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rhel71'
CRS-2673: Attempting to stop 'ora.evmd' on 'rhel71'
CRS-2677: Stop of 'ora.ctssd' on 'rhel71' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rhel71' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rhel71'
CRS-2677: Stop of 'ora.cssd' on 'rhel71' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rhel71'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rhel71'
CRS-2677: Stop of 'ora.gipcd' on 'rhel71' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'rhel71' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel71' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2019/03/19 15:13:36 CLSRSC-594: Executing installation step 18 of 20: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'rhel71'
CRS-2672: Attempting to start 'ora.evmd' on 'rhel71'
CRS-2676: Start of 'ora.evmd' on 'rhel71' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rhel71'
CRS-2676: Start of 'ora.gpnpd' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rhel71'
CRS-2676: Start of 'ora.gipcd' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'rhel71'
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rhel71'
CRS-2676: Start of 'ora.cssdmonitor' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rhel71'
CRS-2672: Attempting to start 'ora.diskmon' on 'rhel71'
CRS-2676: Start of 'ora.diskmon' on 'rhel71' succeeded
CRS-2676: Start of 'ora.crf' on 'rhel71' succeeded
CRS-2676: Start of 'ora.cssd' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rhel71'
CRS-2672: Attempting to start 'ora.ctssd' on 'rhel71'
CRS-2676: Start of 'ora.ctssd' on 'rhel71' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rhel71'
CRS-2676: Start of 'ora.asm' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'rhel71'
CRS-2676: Start of 'ora.storage' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rhel71'
CRS-2676: Start of 'ora.crsd' on 'rhel71' succeeded
CRS-6023: Starting Oracle Cluster Ready Services-managed resources
CRS-6017: Processing resource auto-start for servers: rhel71
CRS-6016: Resource auto-start has completed for server rhel71
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2019/03/19 15:16:19 CLSRSC-343: Successfully started Oracle Clusterware stack
2019/03/19 15:16:19 CLSRSC-594: Executing installation step 19 of 20: 'ConfigNode'.
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'rhel71'
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rhel71'
CRS-2676: Start of 'ora.asm' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.OCRDG.dg' on 'rhel71'
CRS-2676: Start of 'ora.OCRDG.dg' on 'rhel71' succeeded
2019/03/19 15:19:48 CLSRSC-594: Executing installation step 20 of 20: 'PostConfig'.
[INFO] [DBT-30001] Disk groups created successfully. Check /opt/app/oracle/cfgtoollogs/asmca/asmca-190319PM031955.log for details.
2019/03/19 15:23:06 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Root script run on the second (last) node.
./root.sh
Performing root user operation.
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /opt/app/18.x.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
/opt/app/oracle/crsdata/rhel72/crsconfig/rootcrs_rhel72_2019-03-19_03-25-21PM.log
2019/03/19 15:25:38 CLSRSC-594: Executing installation step 1 of 20: 'SetupTFA'.
2019/03/19 15:25:38 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2019/03/19 15:26:17 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2019/03/19 15:26:17 CLSRSC-594: Executing installation step 2 of 20: 'ValidateEnv'.
2019/03/19 15:26:17 CLSRSC-363: User ignored prerequisites during installation
2019/03/19 15:26:17 CLSRSC-594: Executing installation step 3 of 20: 'CheckFirstNode'.
2019/03/19 15:26:19 CLSRSC-594: Executing installation step 4 of 20: 'GenSiteGUIDs'.
2019/03/19 15:26:19 CLSRSC-594: Executing installation step 5 of 20: 'SaveParamFile'.
2019/03/19 15:26:23 CLSRSC-594: Executing installation step 6 of 20: 'SetupOSD'.
2019/03/19 15:26:23 CLSRSC-594: Executing installation step 7 of 20: 'CheckCRSConfig'.
2019/03/19 15:26:25 CLSRSC-594: Executing installation step 8 of 20: 'SetupLocalGPNP'.
2019/03/19 15:26:28 CLSRSC-594: Executing installation step 9 of 20: 'CreateRootCert'.
2019/03/19 15:26:28 CLSRSC-594: Executing installation step 10 of 20: 'ConfigOLR'.
2019/03/19 15:26:31 CLSRSC-594: Executing installation step 11 of 20: 'ConfigCHMOS'.
2019/03/19 15:26:31 CLSRSC-594: Executing installation step 12 of 20: 'CreateOHASD'.
2019/03/19 15:26:34 CLSRSC-594: Executing installation step 13 of 20: 'ConfigOHASD'.
2019/03/19 15:26:35 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2019/03/19 15:27:53 CLSRSC-594: Executing installation step 14 of 20: 'InstallAFD'.
2019/03/19 15:27:56 CLSRSC-594: Executing installation step 15 of 20: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel72'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel72' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2019/03/19 15:29:13 CLSRSC-594: Executing installation step 16 of 20: 'InstallKA'.
2019/03/19 15:29:15 CLSRSC-594: Executing installation step 17 of 20: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel72'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel72' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rhel72'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rhel72'
CRS-2677: Stop of 'ora.drivers.acfs' on 'rhel72' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rhel72' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2019/03/19 15:29:28 CLSRSC-594: Executing installation step 18 of 20: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'rhel72'
CRS-2672: Attempting to start 'ora.evmd' on 'rhel72'
CRS-2676: Start of 'ora.mdnsd' on 'rhel72' succeeded
CRS-2676: Start of 'ora.evmd' on 'rhel72' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rhel72'
CRS-2676: Start of 'ora.gpnpd' on 'rhel72' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rhel72'
CRS-2676: Start of 'ora.gipcd' on 'rhel72' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'rhel72'
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rhel72'
CRS-2676: Start of 'ora.cssdmonitor' on 'rhel72' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rhel72'
CRS-2672: Attempting to start 'ora.diskmon' on 'rhel72'
CRS-2676: Start of 'ora.diskmon' on 'rhel72' succeeded
CRS-2676: Start of 'ora.crf' on 'rhel72' succeeded
CRS-2676: Start of 'ora.cssd' on 'rhel72' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rhel72'
CRS-2672: Attempting to start 'ora.ctssd' on 'rhel72'
CRS-2676: Start of 'ora.ctssd' on 'rhel72' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rhel72'
CRS-2676: Start of 'ora.crsd' on 'rhel72' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rhel72' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rhel72'
CRS-2676: Start of 'ora.asm' on 'rhel72' succeeded
CRS-6017: Processing resource auto-start for servers: rhel72
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rhel71'
CRS-2672: Attempting to start 'ora.ons' on 'rhel72'
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'rhel72'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rhel71' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rhel71'
CRS-2677: Stop of 'ora.scan1.vip' on 'rhel71' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rhel72'
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rhel72' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rhel72'
CRS-2676: Start of 'ora.scan1.vip' on 'rhel72' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rhel72'
CRS-2676: Start of 'ora.ons' on 'rhel72' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rhel72' succeeded
CRS-2676: Start of 'ora.asm' on 'rhel72' succeeded
CRS-2672: Attempting to start 'ora.proxy_advm' on 'rhel71'
CRS-2672: Attempting to start 'ora.proxy_advm' on 'rhel72'
CRS-2676: Start of 'ora.proxy_advm' on 'rhel71' succeeded
CRS-2676: Start of 'ora.proxy_advm' on 'rhel72' succeeded
CRS-6016: Resource auto-start has completed for server rhel72
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2019/03/19 15:32:28 CLSRSC-343: Successfully started Oracle Clusterware stack
2019/03/19 15:32:29 CLSRSC-594: Executing installation step 19 of 20: 'ConfigNode'.
2019/03/19 15:33:03 CLSRSC-594: Executing installation step 20 of 20: 'PostConfig'.
2019/03/19 15:33:13 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
8. Once the root scripts are run click OK on the configuration dialogue box to complete the creation of GIMR and other configuration assistants. At the end this clusterware installation would be completed.
9. Run cluvfy with post crsinst to check the installation.
[grid@rhel71 ~]$ cluvfy stage -post crsinst -allnodes
Verifying Node Connectivity ...
Verifying Hosts File ...PASSED
Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED
Verifying subnet mask consistency for subnet "192.168.1.0" ...PASSED
Verifying subnet mask consistency for subnet "192.168.0.0" ...PASSED
Verifying Node Connectivity ...PASSED
Verifying Multicast or broadcast check ...PASSED
Verifying ASM filter driver configuration consistency ...PASSED
Verifying Time zone consistency ...PASSED
Verifying Cluster Manager Integrity ...PASSED
Verifying User Mask ...PASSED
Verifying Cluster Integrity ...PASSED
Verifying OCR Integrity ...PASSED
Verifying CRS Integrity ...
Verifying Clusterware Version Consistency ...PASSED
Verifying CRS Integrity ...PASSED
Verifying Node Application Existence ...PASSED
Verifying Single Client Access Name (SCAN) ...
Verifying DNS/NIS name service 'rac-scan.domain.net' ...
Verifying Name Service Switch Configuration File Integrity ...PASSED
Verifying DNS/NIS name service 'rac-scan.domain.net' ...PASSED
Verifying Single Client Access Name (SCAN) ...PASSED
Verifying OLR Integrity ...PASSED
Verifying Voting Disk ...PASSED
Verifying ASM Integrity ...
Verifying Node Connectivity ...
Verifying Hosts File ...PASSED
Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED
Verifying subnet mask consistency for subnet "192.168.1.0" ...PASSED
Verifying subnet mask consistency for subnet "192.168.0.0" ...PASSED
Verifying Node Connectivity ...PASSED
Verifying ASM Integrity ...PASSED
Verifying Device Checks for ASM ...
Verifying Access Control List check ...PASSED
Verifying I/O scheduler ...PASSED
Verifying Device Checks for ASM ...PASSED
Verifying ASM disk group free space ...PASSED
Verifying User Not In Group "root": grid ...PASSED
Verifying Clock Synchronization ...PASSED
Verifying VIP Subnet configuration check ...PASSED
Verifying Network configuration consistency checks ...PASSED
Verifying File system mount options for path GI_HOME ...PASSED
Verifying Access control attributes for /var/tmp/.oracle ...PASSED
Verifying Access control attributes for %OCRCONFIGDIR%/maps ...PASSED
Post-check for cluster services setup was successful.10. There are few cluster and local resources that are new in 18c compared to 12.2 Most of them relate the rapid home provisioning feature and are in disable state.
crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.GIMRDG.GHCHKPT.advm
OFFLINE OFFLINE rhel71 STABLE
...
ora.gimrdg.ghchkpt.acfs
OFFLINE OFFLINE rhel71 STABLE
OFFLINE OFFLINE rhel72 STABLE
ora.helper
OFFLINE OFFLINE rhel71 IDLE,STABLE
OFFLINE OFFLINE rhel72 IDLE,STABLE
ora.proxy_advm
ONLINE ONLINE rhel71 STABLE
ONLINE ONLINE rhel72 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
...
ora.rhpserver
1 OFFLINE OFFLINE STABLE
--------------------------------------------------------------------------------For example clusterware installation create a pre-configued ASM volume group call GHCHKPT which is to be used with rapid home provisioning framework.
ASMCMD> volinfo --all
Diskgroup Name: GIMRDG
Volume Name: GHCHKPT
Volume Device: /dev/asm/ghchkpt-493
State: DISABLED
Size (MB): 1536
Resize Unit (MB): 512
Redundancy: UNPROT
Stripe Columns: 8
Stripe Width (K): 1024
Usage: ACFS
Mountpath: /opt/oracle/rhp_images/chkbaseBecause of this volume group the proxy ASM instance will have an active and online status. These would be visible as APX{node number} instances. Oracle docs states "
an Oracle ASM proxy instance is an Oracle instance running on a Hub node with a direct Oracle ASM client. An Oracle ASM proxy instance provides support for Oracle Automatic Storage Management Cluster File System (Oracle ACFS) and Oracle ASM Dynamic Volume Manager (Oracle ADVM)". If no other ASM volume groups are to be created or online then ASM proxy instances could be stopped and disabled.
[grid@rhel71 ~]$ srvctl stop asm -proxy -node rhel71
[grid@rhel71 ~]$ srvctl stop asm -proxy -node rhel72
[grid@rhel71 ~]$ srvctl status asm -proxy
ADVM proxy is not running
The full list of resources after the installation is shown below (before proxy instances were disabled).
Resource Name Type Target State Host
------------- ------ ------- -------- ----------
ora.ASMNET1LSNR_ASM.lsnr ora.asm_listener.type ONLINE ONLINE rhel71
ora.ASMNET1LSNR_ASM.lsnr ora.asm_listener.type ONLINE ONLINE rhel72
ora.GIMRDG.GHCHKPT.advm ora.volume.type OFFLINE OFFLINE
ora.GIMRDG.GHCHKPT.advm ora.volume.type OFFLINE OFFLINE
ora.GIMRDG.dg ora.diskgroup.type ONLINE ONLINE rhel71
ora.GIMRDG.dg ora.diskgroup.type ONLINE ONLINE rhel72
ora.LISTENER.lsnr ora.listener.type ONLINE ONLINE rhel71
ora.LISTENER.lsnr ora.listener.type ONLINE ONLINE rhel72
ora.LISTENER_SCAN1.lsnr ora.scan_listener.type ONLINE ONLINE rhel72
ora.LISTENER_SCAN2.lsnr ora.scan_listener.type ONLINE ONLINE rhel71
ora.LISTENER_SCAN3.lsnr ora.scan_listener.type ONLINE ONLINE rhel71
ora.MGMTLSNR ora.mgmtlsnr.type ONLINE ONLINE rhel71
ora.OCRDG.dg ora.diskgroup.type ONLINE ONLINE rhel71
ora.OCRDG.dg ora.diskgroup.type ONLINE ONLINE rhel72
ora.asm ora.asm.type ONLINE ONLINE rhel71
ora.asm ora.asm.type ONLINE ONLINE rhel72
ora.asm ora.asm.type OFFLINE OFFLINE
ora.chad ora.chad.type ONLINE ONLINE rhel71
ora.chad ora.chad.type ONLINE ONLINE rhel72
ora.cvu ora.cvu.type ONLINE ONLINE rhel71
ora.gimrdg.ghchkpt.acfs ora.acfs.type OFFLINE OFFLINE
ora.gimrdg.ghchkpt.acfs ora.acfs.type OFFLINE OFFLINE
ora.helper ora.helper.type OFFLINE OFFLINE
ora.helper ora.helper.type OFFLINE OFFLINE
ora.mgmtdb ora.mgmtdb.type ONLINE ONLINE rhel71
ora.net1.network ora.network.type ONLINE ONLINE rhel71
ora.net1.network ora.network.type ONLINE ONLINE rhel72
ora.ons ora.ons.type ONLINE ONLINE rhel71
ora.ons ora.ons.type ONLINE ONLINE rhel72
ora.proxy_advm ora.proxy_advm.type ONLINE ONLINE rhel71
ora.proxy_advm ora.proxy_advm.type ONLINE ONLINE rhel72
ora.qosmserver ora.qosmserver.type ONLINE ONLINE rhel71
ora.rhel71.vip ora.cluster_vip_net1.type ONLINE ONLINE rhel71
ora.rhel72.vip ora.cluster_vip_net1.type ONLINE ONLINE rhel72
ora.rhpserver ora.rhpserver.type OFFLINE OFFLINE
ora.scan1.vip ora.scan_vip.type ONLINE ONLINE rhel72
ora.scan2.vip ora.scan_vip.type ONLINE ONLINE rhel71
ora.scan3.vip ora.scan_vip.type ONLINE ONLINE rhel71
11. The default backup location for OCR is the disk group used for GIMR
ocrconfig -showbackuploc
The Oracle Cluster Registry backup location is [+GIMRDG].
12. The clusermode and versions are as follows.
$ crsctl get cluster mode status
Cluster is running in "flex" mode
$crsctl get node role config -all
Node 'rhel71' configured role is 'hub'
Node 'rhel72' configured role is 'hub'
$ crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [18.0.0.0.0]
The next step is to
install the database software and craete the RAC DB.
Related Posts
Installing 12cR2 (12.2.0.1) RAC on RHEL 6 with Role Separation - Clusterware
Installing 12c (12.1.0.2) Flex Cluster on RHEL 6 with Role Separation
Installing 12c (12.1.0.1) RAC on RHEL 6 with Role Separation - Clusterware
Installing 11gR2 (11.2.0.3) GI with Role Separation on RHEL 6
Installing 11gR2 (11.2.0.3) GI with Role Separation on OEL 6
Installing 11gR2 Standalone Server with ASM and Role Separation on RHEL 6
11gR2 Standalone Data Guard (with ASM and Role Separation)
Update on 2020-10-01
Following error was given when intsalling a role separated Oracle restart on 19c. Error was thrown during the Oracle home installaton.
./runInstaller -silent -responseFile /home/oracle/db_install.rsp -ignorePrereqFailure
Launching Oracle Database Setup Wizard...
[FATAL] [INS-30215] Installer has detected that Oracle Grid Infrastructure owner "grid" does not belong to the OS group "racdba" on local node.
CAUSE: The OS group "racdba" does not exist or Oracle Grid Infrastructure owner "grid" is not a member of the OS group "racdba" on local node.
ACTION: Ensure that OS group "racdba" exists and add Oracle Grid Infrastructure owner "grid" as a member of the OS group "racdba" on local node or specify a different OS group in which Oracle Grid Infrastructure owner is a member.
*ADDITIONAL INFORMATION:*
- PRVF-7566 : User "grid" does not belong to group "racdba" on node "ip-172-31-2-77"
- Cause: The specified user is not a member of the specified group on the specified node.
- Action: Make the user a member of the group on the specified node.
It seems with 19c grid user should also be a memeber of the racdba group.
