Monday, June 22, 2015

Can't exec "/usr/bin/lsb_release": No such file or directory at GI_HOME/lib/osds_acfslib.pm line 564

Following was observed during the GI PSU 11.2.4.0.6 patch application
2015-06-15 11:19:01: acfs is supported
2015-06-15 11:19:01: Executing '/opt/app/oracle/product/11.2.0/grid_4/bin/acfsroot install'
2015-06-15 11:19:01: Executing cmd: /opt/app/oracle/product/11.2.0/grid_4/bin/acfsroot install
2015-06-15 11:23:00: Command output:
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  ACFS-9300: ADVM/ACFS distribution files found.
>  ACFS-9312: Existing ADVM/ACFS installation detected.
>  ACFS-9314: Removing previous ADVM/ACFS installation.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  ACFS-9315: Previous ADVM/ACFS components successfully removed.
>  ACFS-9307: Installing requested ADVM/ACFS software.
>  ACFS-9308: Loading installed ADVM/ACFS drivers.
>  ACFS-9321: Creating udev for ADVM/ACFS.
>  ACFS-9323: Creating module dependencies - this may take some time.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  ACFS-9154: Loading 'oracleoks.ko' driver.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  ACFS-9154: Loading 'oracleadvm.ko' driver.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  ACFS-9154: Loading 'oracleacfs.ko' driver.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  ACFS-9327: Verifying ADVM/ACFS devices.
>  ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
>  ACFS-9156: Detecting control device '/dev/ofsctl'.
>  ACFS-9309: ADVM/ACFS installation correctness verified.
>End Command output



The environment is OEL 6 (2.6.32-358.18.1.el6.x86_64).
Reason for this error is bug 17359415. Install redhat-lsb-core-4.0*.el6.x86_64 rpm to fix this problem.
# yum install redhat-lsb-core-4.0*.el6.x86_64
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package redhat-lsb-core.x86_64 0:4.0-7.0.1.el6 will be installed

Sunday, June 14, 2015

"Error: Multilib version problems found" When Installing Required RPMS on OEL 6

Following error could be seen while trying to install required RPMS as part of 12c pre-req task (1529864.1). OS is OEL 6 (2.6.32-358.18.1.el6.x86_64).
yum install libXi.i686
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package libXi.i686 0:1.7.2-2.2.el6 will be installed
--> Processing Dependency: libXext.so.6 for package: libXi-1.7.2-2.2.el6.i686
--> Running transaction check
---> Package libXext.x86_64 0:1.3.1-2.el6 will be updated
---> Package libXext.i686 0:1.3.2-2.1.el6 will be installed
---> Package libXext.x86_64 0:1.3.2-2.1.el6 will be an update
--> Finished Dependency Resolution
Error:  Multilib version problems found. This often means that the root
       cause is something else and multilib version checking is just
       pointing out that there is a problem. Eg.:

         1. You have an upgrade for libXi which is missing some
            dependency that another package requires. Yum is trying to
            solve this by installing an older version of libXi of the
            different architecture. If you exclude the bad architecture
            yum will tell you what the root cause is (which package
            requires what). You can try redoing the upgrade with
            --exclude libXi.otherarch ... this should give you an error
            message showing the root cause of the problem.

         2. You have multiple architectures of libXi installed, but
            yum can only see an upgrade for one of those arcitectures.
            If you don't want/need both architectures anymore then you
            can remove the one with the missing update and everything
            will work.

         3. You have duplicate versions of libXi installed already.
            You can use "yum check" to get yum show these errors.

       ...you can also use --setopt=protected_multilib=false to remove
       this checking, however this is almost never the correct thing to
       do as something else is very likely to go wrong (often causing
       much more problems).

       Protected multilib versions: libXi-1.7.2-2.2.el6.i686 != libXi-1.6.1-3.el6.x86_64
 You could try using --skip-broken to work around the problem
** Found 2 pre-existing rpmdb problem(s), 'yum check' output follows:
oracle-rdbms-server-11gR2-preinstall-1.0-7.el6.x86_64 has missing requires of kernel-uek
oracle-rdbms-server-12cR1-preinstall-1.0-8.el6.x86_64 has missing requires of kernel-uek
In this case the root cause as the output suggested was that i686 version of the rpm was of higher version than already installed x86_64 version.


To resolve this update the currently installed x86_64 version and then install the i686 version.
yum update libXi
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package libXi.x86_64 0:1.6.1-3.el6 will be updated
...

yum install libXi.i686
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package libXi.i686 0:1.7.2-2.2.el6 will be installed
....
Useful metalink notes
Yum update command fails with error around multilib missmatch [ID 1670710.1]
Troubleshooting Common Issues While Registering RHEL Servers With ULN [ID 1984132.1]
Error "Transaction Check Error: file /etc/pki/rpm-gpg/RPM-GPG-KEY from ... " When Upgrading Oracle Linux Server [ID 1996064.1]

Monday, June 8, 2015

ins_emagent.mk Related Error When Installing 11.2.0.4 Database on RHEL 7

Following error could be seen when installing 11.2.0.4 on RHEL 7 (3.10.0-123.el7.x86_64).

To fix edit the ins_emagent.mk file and change the following line in $ORACLE_HOME/sysman/lib/ins_emagent.mk
$(MK_EMAGENT_NMECTL)
to
$(MK_EMAGENT_NMECTL) -lnnz11
and press the retry button again.

Useful metalink notes
Requirements for Installing Oracle Database 12.1on RHEL7 or OL7 64-bit (x86-64) [ID 1961997.1]
Requirements for Installing Oracle 11.2.0.4 RDBMS on RHEL7 or OL7 64-bit (x86-64) [ID 1962100.1]

Related Post
Installing Oracle Database 12.1.0.2 on RHEL 7

Update on 2016-01-25
This error requires patch 19692824 which should be applied after the installation (continue with above error). Refer MOS installation of Oracle 11.2.0.4 Database Software on OL7 fails with “undefined reference to symbol ‘B_DestroyKeyObject’” error [ID 1965691.1]

Monday, June 1, 2015

Creating Data Guard Broker on 12c

The data guard configuration used in this post is a physical standby with standalone servers using ASM for data storage. The primary database is ent12c1 and standby is ent12c1s. This post lists the steps of adding a data guard broker configuration to the existing data guard configuration in 12c.
1. Modify the listener.ora files in both primary and standby add the default static service name entries which are of the "db_unique_name_DGMGRL.db_domain" form. In this case the database domain name is "domain.net". If a non default service name is used then this must be set with StaticConnectIdentifier parameter. Oracle documentation states as of Oracle Database 12.1.0.2 "a static service needs to be defined and registered only if Oracle Clusterware or Oracle Restart is not being used. Broker will use the clusterware to restart an instance". In this case the static _DGMGRL service was created to illustrate the steps that could be used in non-RAC and non Oracle restart setups.
On Primary

SID_LIST_LISTENER =
(SID_LIST =
        (SID_DESC =
                (GLOBAL_DBNAME = ent12c1.domain.net)
                (SID_NAME = ent12c1)
                (ORACLE_HOME = /opt/app/oracle/product/12.1.0/dbhome_2)
        )

        (SID_DESC =
                (GLOBAL_DBNAME = ent12c1_DGMGRL.domain.net)
                (SID_NAME = ent12c1)
                (ORACLE_HOME = /opt/app/oracle/product/12.1.0/dbhome_2)
        )

)

On Standby

SID_LIST_LISTENER =
(SID_LIST =
        (SID_DESC =
                (GLOBAL_DBNAME = ent12c1s.domain.net)
                (SID_NAME = ent12c1s)
                (ORACLE_HOME = /opt/app/oracle/product/12.1.0/dbhome_2)
        )

        (SID_DESC =
                (GLOBAL_DBNAME = ent12c1s_DGMGRL.domain.net)
                (SID_NAME = ent12c1s)
                (ORACLE_HOME = /opt/app/oracle/product/12.1.0/dbhome_2)
        )

)
2. Data guard broker configuration files would be stored in different types of locations (ASM, Cluster FS, ext3). For this setup these files are stored in ASM. For this first create a directory in ASM (if already does not exists) and then the dg broker configuration file location.
On primary 

ASMCMD>cd data/ent12c1
ASMCMD> mkdir dgbroker

ASMCMD>cd flash/ent12c1
ASMCMD> mkdir dgbroker

alter system set dg_broker_config_file1='+DATA/ent12c1/dgbroker/dr1ent12c1.dat' scope=both sid='*';
alter system set dg_broker_config_file2='+FLASH/ent12c1/dgbroker/dr2ent12c1.dat' scope=both sid='*';

On standby

ASMCMD>cd dg_data/ent12c1s
ASMCMD> mkdir dgbroker

ASMCMD>cd dg_flash/ent12c1s
ASMCMD> mkdir dgbroker

alter system set dg_broker_config_file1='+DG_DATA/ent12c1s/dgbroker/dr1ent12c1s.dat' scope=both sid='*';
alter system set dg_broker_config_file2='+DG_FLASH/ent12c1s/dgbroker/dr2ent12c1s.dat' scope=both sid='*';
Finally on both primary and standby set the dg broker start to true.
alter system set dg_broker_start=true scope=both sid='*';
3. In previous versions at this point it would be possible to create the dg broker configuration. But in 12c trying to create the configuration would fail at this stage
DGMGRL> create configuration ent12c1_db as primary database is ent12c1 connect identifier is ENT12c1TNS;
Error: ORA-16698: LOG_ARCHIVE_DEST_n parameter set for object to be added
This is because "as of Oracle Database 12c Release 1 (12.1), for all databases to be added to a broker configuration, any LOG_ARCHIVE_DEST_n parameters that have the SERVICE attribute set, but not the NOREGISTER attribute, must be cleared". To fix this clear the log_archive_dest parameter that does the remote log shipping both on primary and standby.
SQL> alter system set log_Archive_dest_2='' scope=both;
4. Once log_archive_dest is cleared, creation of dg broker configuration succeed.
DGMGRL> create configuration ent12c1_db as primary database is ent12c1 connect identifier is ENT12c1TNS;
Configuration "ent12c1_db" created with primary database "ent12c1"

DGMGRL> show configuration

Configuration - ent12c1_db

  Protection Mode: MaxPerformance
  Members:
  ent12c1 - Primary database

Fast-Start Failover: DISABLED

Configuration Status:
DISABLED


5. Add the standby instance to the configuration
DGMGRL> add database ent12c1s as connect identifier is ENT12c1STNS;
Database "ent12c1s" added

DGMGRL> show configuration

Configuration - ent12c1_db

  Protection Mode: MaxPerformance
  Members:
  ent12c1  - Primary database
    ent12c1s - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
DISABLED
6. Finally enable the configuration
DGMGRL> enable configuration;

DGMGRL> show configuration

Configuration - ent12c1_db

  Protection Mode: MaxPerformance
  Members:
  ent12c1  - Primary database
    ent12c1s - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS   (status updated 7 seconds ago)
Verify the StaticConnectIdentifier is using the default service name
DGMGRL> show database ent12c1s StaticConnectIdentifier
  StaticConnectIdentifier = '(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.86)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=ent12c1s_DGMGRL.domain.net)(INSTANCE_NAME=ent12c1s)(SERVER=DEDICATED)))'

DGMGRL> show database ent12c1 StaticConnectIdentifier
  StaticConnectIdentifier = '(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.85)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=ent12c1_DGMGRL.domain.net)(INSTANCE_NAME=ent12c1)(SERVER=DEDICATED)))'
7. At this stage the dg broker configuration creation is complete and data guard is running in maximum performance mode. However the default log transport mode ASYNC will not allow protection mode to be increased to maximum availability.
DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS MaxAvailability;
Error: ORA-16627: operation disallowed since no standby databases would remain to support protection mode
Failed.
Change the redo transport service to SYNC using LogXptMode. 12c also has a new redo transport service called FASTSYNC (equivalent to setting SYNC NOAFFIRM) which is only available with maximum protection mode.
DGMGRL> show database ent12c1s LogXptMode
  LogXptMode = 'ASYNC'

DGMGRL> edit database ent12c1s SET PROPERTY LogXptMode='SYNC';
Property "logxptmode" updated
DGMGRL>  edit database ent12c1 SET PROPERTY LogXptMode='SYNC';
Property "logxptmode" updated
DGMGRL> show configuration

Configuration - ent12c1_db

  Protection Mode: MaxPerformance
  Members:
  ent12c1  - Primary database
    ent12c1s - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS   (status updated 60 seconds ago)

DGMGRL>  EDIT CONFIGURATION SET PROTECTION MODE AS MaxAvailability;
Succeeded.

DGMGRL>  show configuration

Configuration - ent12c1_db

  Protection Mode: MaxAvailability
  Members:
  ent12c1  - Primary database
    ent12c1s - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS   (status updated 51 seconds ago)


8. 12c has also introduced a new dg broker command "validate database" which checks the readiness of a database to perform a role change.
On primary 

DGMGRL> validate database ent12c1

  Database Role:    Primary database

  Ready for Switchover:  Yes

On Standby

DGMGRL> validate database ent12c1s

  Database Role:     Physical standby database
  Primary Database:  ent12c1

  Ready for Switchover:  Yes
  Ready for Failover:    Yes (Primary Running)


DGMGRL> switchover to ent12c1s
Performing switchover NOW, please wait...
New primary database "ent12c1s" is opening...
Oracle Clusterware is restarting database "ent12c1" ...
Switchover succeeded, new primary is "ent12c1s"

DGMGRL> show configuration;

Configuration - ent12c1_db

  Protection Mode: MaxAvailability
  Members:
  ent12c1s - Primary database
    ent12c1  - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS   (status updated 38 seconds ago)

DGMGRL> validate database ent12c1s

  Database Role:    Primary database

  Ready for Switchover:  Yes

DGMGRL> validate database ent12c1

  Database Role:     Physical standby database
  Primary Database:  ent12c1s

  Ready for Switchover:  Yes
  Ready for Failover:    Yes (Primary Running)
This concludes the creation of data guard broker on 12c.

Useful metalink notes
Create Configuration Failing with ORA-16698 [ID 1582179.1]
12c Create Dataguard Broker Configuration - DGMGRL [ID 1583588.1]

Related Posts
Adding Far Sync Instances to Existing Data Guard Configuration
11gR2 Standalone Data Guard (with ASM and Role Separation)
11gR2 RAC to RAC Data Guard
Upgrading from 11.2.0.2 to 11.2.0.3 with Physical Standby - 1
RAC to Single Instance Physical Standby