HAIP for Private Interconnect in 11gR2

Multiple NICs would be defined of private interconnect during installation (or after) on 11gR2. When there are multiple NICs defined for private interconnect oracle creates highly available IP (HAIP) for interconnect traffic.
From 11.2.0.2 onwards even if single NICs is used for private interconnect Oracle will use HAIP by default. Because of this v$cluster_interconnects view will show HAIP on 11gR2 (11.2.0.2 or later) compared to earlier versions such as 10g or 11gR1.
For example on 11gR1 IPs used for private interconnect could be identified using v$cluster_interconnects view

SQL> select * from gv$cluster_interconnects order by 1;

   INST_ID NAME            IP_ADDRESS       IS_ SOURCE
---------- --------------- ---------------- --- -------------------------------
         1 eth1            192.168.1.87     NO  Oracle Cluster Repository
         2 eth1            192.168.1.88     NO  Oracle Cluster Repository

Or on the database alert log will have following entries

Cluster communication is configured to use the following interface(s) for this instance
  192.168.1.88
cluster interconnect IPC version:Oracle UDP/IP (generic)
IPC Vendor 1 proto 2

However due to HAIP this is not the same on 11gR2, and what you specified during installation is not what you see on v$cluster_interconnects.

SQL> select * from gv$cluster_interconnects order by 1;

   INST_ID NAME            IP_ADDRESS       IS_ SOURCE
---------- --------------- ---------------- --- -------------------------------
         1 eth1:1          169.254.73.209   NO
         2 eth1:1          169.254.180.78   NO

Grid automatically creates the 169.254.*.* range IPs from the link local addresses that will be used for HAIP. These IPs are visible when ifconfig is executed and will appear on the same NIC assigned to private interconnect IP.

eth1      Link encap:Ethernet  HWaddr 08:00:27:7C:F4:94
          inet addr:192.168.1.88  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe7c:f494/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1997903 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1680302 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1349158236 (1.2 GiB)  TX bytes:965427458 (920.7 MiB)

eth1:1    Link encap:Ethernet  HWaddr 08:00:27:7C:F4:94
          inet addr:169.254.73.209  Bcast:169.254.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

Instead of getif option iflist could be used to list the HAIP associated interface.

oifcfg getif
eth0  192.168.0.0  global  public
eth1  192.168.1.0  global  cluster_interconnect

oifcfg iflist -p -n
eth0  192.168.0.0  PRIVATE  255.255.255.0
eth1  192.168.1.0  PRIVATE  255.255.255.0
eth1  169.254.0.0  UNKNOWN  255.255.0.0

Both ASM and DB alert log will also list the HAIP instead of the actual IP used during installation. From 11gR2 DB alert log

Cluster communication is configured to use the following interface(s) for this instance
  169.254.73.209
cluster interconnect IPC version:Oracle UDP/IP (generic)

From ASM alert log

Private Interface 'eth1:1' configured from GPnP for use as a private interconnect.
  [name='eth1:1', type=1, ip=169.254.73.209, mac=08-00-27-7c-f4-94, net=169.254.0.0/16, mask=255.255.0.0, use=haip:cluster_interconnect/62]
Public Interface 'eth0' configured from GPnP for use as a public interface.
  [name='eth0', type=1, ip=192.168.0.86, mac=08-00-27-c1-59-a3, net=192.168.0.0/24, mask=255.255.255.0, use=public/1]

Useful metalink notes
11gR2 Grid Infrastructure Redundant Interconnect and ora.cluster_interconnect.haip [ID 1210883.1]

PRIF-31: Failed to delete the specified network interface because it is the last private interface

Prior to 11.2.0.2 the steps to change the private interconnect subnet, netmask or network interface involved deleting the existing interface related to private interconnect and adding a new one. However from 11.2.0.2 onwards this will fail and it is required to add the new configuration information first before deleting the existing one.
If only the private interconnect IP is changed (192.168.0.10 changed to 192.168.0.22) then shutting down the clusterware stack and changing the IPs on the OS level and starting the clusterware stack backup would suffice. In this case the subnet is changed.
Make a backup (on all the nodes of the cluster) of the profile.xml file located in $GRID_HOME/gpnp/<hostname>/profiles/peer.
Carry out the modification as the grid user (or oracle user if role separation is not used)
Trying to delete the existing cluster interconnect will fail.

$ oifcfg getif
eth0  192.168.0.0  global  public
eth1  192.168.0.0  global  cluster_interconnect

$ oifcfg delif -global eth1
PRIF-31: Failed to delete the specified network interface because it is the last private interface

Add the new configuration

$ oifcfg setif -global eth1/192.168.1.0:cluster_interconnect

$ oifcfg getif
eth0  192.168.0.0  global  public
eth1  192.168.0.0  global  cluster_interconnect
eth1  192.168.1.0  global  cluster_interconnect

Shutdown the clusterware stack and make the changes in the OS level (/etc/hosts file, ifcfg-eth1 file and etc) and start the clusterware stack again

Delete the old configuration by specifying the correct subnet and the interface to delete

$ oifcfg delif -global eth1/192.168.0.0

$ oifcfg getif
eth0  192.168.0.0  global  public
eth1  192.168.1.0  global  cluster_interconnect

Useful Metalink notes
How to Modify Private Network Interface in 11.2 Grid Infrastructure [ID 1073502.1]
How to Modify Public or Private Network Information in Oracle Clusterware [ID 283684.1]

Related Oracle Documentation

Find out Interconnect Protocol

SQL > oradebug setmypid
Statement processed.

SQL>oradebug ipc
Information written to trace file.

Trace file can be found in user_dump_dest directory

SSKGXPT 0x66964c0 flags SSKGXPT_READPENDING socket no 7 IP 10.1.10.22 UDP 52459

This method can be used for both RDMS and ASM instances.

Alert logs of these various types of instances also can be used to find out the interconnect protocol.

cluster interconnect IPC version:Oracle UDP/IP (generic)