Implementing Class of Secure Transport (COST) to Restrict Instance Registration in Oracle 11gR2 SE RAC (Solution mentioned in Oracle Security Alert for CVE-2012-1675)

This post is related to implementing the solution for security vulnerability mentioned in Oracle Security Alert for CVE-2012-1675. There is another post but that is related to version 11gR1 and this one is specific to 11gR2.
The metalink note related to applying the solution in a RAC environment is Using Class of Secure Transport (COST) to Restrict Instance Registration in Oracle RAC [ID 1340831.1]
However this solution fails when tried on a standard edition RAC environment as standard edition Oracle binaries are not linked with TCPS protocol. The standard edition RAC environment created for RHEL 6 is used in this case.
All the steps in section 1 in (1340831.1) up to step 1.4 work same on SE as on EE edition and problem will be encountered trying to implement step 1.5.
As oracle user (since this is a role separated environment) try to set TCPS protocol for the remote listener using scan ips.

show parameter remote_listener

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
remote_listener                      string      rhel6m-scan:1521

This environment only used one scan ip (not recommended by Oracle)

srvctl config scan
SCAN name: rhel6m-scan, Network: 1/192.168.0.0/255.255.255.0/eth0
SCAN VIP name: scan1, IP: /rhel6m-scan/192.168.0.91

Trying to set TCPS will give the following error

SQL> alter system set remote_listener='(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCPS)(HOST=192.168.0.91)(PORT=1523)))' scope=both sid='*';
alter system set remote_listener='(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCPS)(HOST=192.168.0.91)(PORT=1523)))' scope=both sid='*'
*
ERROR at line 1:
ORA-02097: parameter cannot be modified because specified value is invalid
ORA-00119: invalid specification for system parameter REMOTE_LISTENER
ORA-00130: invalid listener address
'(ADDRESS=(PROTOCOL=TCPS)(HOST=192.168.0.91)(PORT=1523))'

Error messages here doesn't help much as it doesn't specifically say which parameter is invalid unlike in 11gR1 which mentioned invalid protocol. It could be tested indeed it's the protocol that is causing the problem by testing the same command with TCP instead of TCPS, in this case it would work without an error

SQL> alter system set remote_listener='(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.91)(PORT=1523)))' scope=both sid='*';

System altered.

Solution is to relink the binaries with tcps same as before. Stop all processes running out of the ORACLE_HOME

srvctl stop home -o $ORACLE_HOME -s status -n `hostname -s`

Change into $ORACLE_HOME/lib and make a backup of the existing libntcps11.a file and copy the libntcps11_ee.a.dbl

cp libntcps11.a libntcps11.a.orig
cp libntcps11_ee.a.dbl libntcps11.a

Execute relink all and monitor the log file

$ relink all
writing relink log to: /opt/app/oracle/product/11.2.0/dbhome_1/install/relink.log

Relinking ends with following text

-L/opt/app/oracle/product/11.2.0/dbhome_1/lib
test ! -f /opt/app/oracle/product/11.2.0/dbhome_1/bin/oracle ||\
           mv -f /opt/app/oracle/product/11.2.0/dbhome_1/bin/oracle /opt/app/oracle/product/11.2.0/dbhome_1/bin/oracleO
mv /opt/app/oracle/product/11.2.0/dbhome_1/rdbms/lib/oracle /opt/app/oracle/product/11.2.0/dbhome_1/bin/oracle
chmod 6751 /opt/app/oracle/product/11.2.0/dbhome_1/bin/oracle

It is important to relink all the oracle homes in the cluster if not the error will still persist

SQL> alter system set remote_listener='(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCPS)(HOST=192.168.0.91)(PORT=1523)))' scope=both sid='*';
alter system set remote_listener='(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCPS)(HOST=192.168.0.91)(PORT=1523)))' scope=both sid='*'
*
ERROR at line 1:
ORA-32008: error while processing parameter update at instance std11g21
ORA-02097: parameter cannot be modified because specified value is invalid
ORA-00119: invalid specification for system parameter REMOTE_LISTENER
ORA-00130: invalid listener address
'(ADDRESS=(PROTOCOL=TCPS)(HOST=192.168.0.91)(PORT=1523))'

But to confirm relinking has worked the above command could be executed at instance level

SQL> alter system set remote_listener='(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCPS)(HOST=192.168.0.91)(PORT=1523)))' scope=both sid='std11g22';

System altered.

Once all oracle homes are relinked execute the original command.

SQL> alter system set remote_listener='(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCPS)(HOST=192.168.0.91)(PORT=1523)))' scope=both sid='*';

System altered.

Rest of the steps in (1340831.1) could be executed without any additional work.
If the libntcps11_ee.a.dbl file is missing it could obtained by extracting the

database/stage/Components/oracle.network.rsf/11.2.0.3.0/1/DataFiles/filegroup5.jar

Related Post
Using Class of Secure Transport (COST) to Restrict Instance Registration in Oracle SE RAC Fails (Solution mentioned in Oracle Security Alert for CVE-2012-1675)




Update on 11th November 2015
For database version 11.2.0.4 Oracle has introduced "Valid Node Checking for Registration (VNCR)" as an alternative for implementing COST, if the sole reason for implementing cost is preventing remote registration. MOS note 1340831.1 has been updated with this information. VNCR related information could be found on following docs.
How to Enable VNCR on RAC Database to Register only Local Instances [ID 1914282.1]
Valid Node Checking For Registration (VNCR) [ID 1600630.1]