Wednesday, July 7, 2010

Migrating OCR to ASM in 11gR2 Clusterware

Before migrating the OCR to ASM there are serveral things to consider. On Oracle Clusterware Administration and Deployment Guide 11g Release 2 (11.2)E10717-03 August 2009 following is said about having an OCR in the ASM.
You can store the OCR in an Oracle ASM disk group that has
external redundancy.
If a disk fails in the disk group, or if you bring down Oracle ASM, then
you can lose the OCR because it depends on Oracle ASM for I/O.
To avoid this issue, add another OCR to a different disk group.
Alternatively, you can store OCR on a block device, or on a shared file
system using OCRCONFIG to enable OCR redundancy.
Oracle does not support storing the OCR on different storage types
simultaneously, such as storing OCR on both Oracle ASM and a shared
file system, except during a migration.
If Oracle ASM fails, then OCR is not accessible on the node on
which Oracle ASM failed, but the cluster remains operational. The
entire cluster only fails if the Oracle ASM instance on the OCR
master node fails, if the majority of the OCR locations are in
Oracle ASM, and if there is an OCR read or write access, then the
crsd stops and the node becomes inoperative.

Ensure that Oracle ASM disk groups that you create are mounted
on all of the nodes in the cluster.

But this has been re-written on Oracle Clusterware Administration and Deployment Guide 11g Release 2 (11.2) E10717-11 April 2010
If OCR is stored in an Oracle ASM disk group with external
redundancy, then Oracle recommends that you add another OCR
location to another disk group to avoid the loss of OCR, if a disk
fails in the disk group.
Oracle does not support storing the OCR on different storage types
simultaneously, such as storing OCR on both Oracle ASM and a shared
file system, except during a migration.
If an Oracle ASM instance fails on any node, then OCR becomes
unavailable on that particular node.
If the crsd process running on the node affected by the Oracle ASM
instance failure is the OCR writer, the majority of the OCR locations are
stored in Oracle ASM, and you attempt I/O on OCR during the time the
Oracle ASM instance is down on this node, then crsd stops and becomes
inoperable. Cluster management is now affected on this particular node.
Under no circumstances will the failure of one Oracle ASM instance on
one node affect the whole cluster.

Ensure that Oracle ASM disk groups that you create are mounted
on all of the nodes in the cluster.

This has not been verified as the setup used in this blog is a cluster setup upgraded to 11.2 and only has one node.
This does not require node to be down.

1. Current setup uses block devices for OCR
ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 148348
Used space (kbytes) : 4480
Available space (kbytes) : 143868
ID : 552644455
Device/File Name : /dev/sdc5
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
2. Created a ASM diskgroup here called CLUSTERDG with Normal Redundancy and explicitly mount it, to store the OCR. To store OCR/Vote disks in an ASM diskgroup the ASM Compatibility must be set to 11.2.0.0
create diskgroup clusterdg normal redundancy failgroup failgp1 disk 'ORCL:CLUS1'  
failgroup failgp2 disk 'ORCL:CLUS2' failgroup failgp3 disk 'ORCL:CLUS3'
ATTRIBUTE 'compatible.asm' = '11.2';

select name,type,compatibility from v$asm_diskgroup;

NAME TYPE COMPATIBILITY
-------------------- -------------------- --------------------
FLASH EXTERN 11.2.0.0.0
CLUSTERDG NORMAL 11.2.0.0.0
DATA EXTERN 11.2.0.0.0
From Oracle Grid Infrastructure installation Guide "For Oracle Clusterware files, Normal redundancy disk groups provide 3 voting disk files, 1 OCR and 2 copies (one primary and one secondary mirror). With normal redundancy, the cluster can survive the loss of one failure group. For most installations, Oracle recommends that you select normal redundancy."

3. Since this only has one OCR location need to use a add/delete approach. If there were multiple OCR locations then
ocrconfig -replace current_location new location
command could be used. Command is run as root
ocrconfig -add +CLUSTERDG
Verify on alerthostname.log
2010-07-07 12:50:01.288
[crsd(7409)]CRS-1007:The OCR/OCR mirror location was replaced by +CLUSTERDG.
On crsd.log
2010-07-07 12:50:00.945: [  OCRRAW][1385945408]propriowv_bootbuf: Vote information on disk 0 [/dev/sdc5] is adjusted from [0/0] to [2/2]
2010-07-07 12:50:01.165: [ OCRASM][1385945408]proprasmo: ASM cache size is [5MB]
2010-07-07 12:50:01.165: [ OCRASM][1385945408]proprasmo: ASM cache [5MB] enabled for disk group [CLUSTERDG].
2010-07-07 12:50:01.187: [ OCRRAW][1385945408]proprioo: for disk 0 (/dev/sdc5), id match (1), total id sets, (2) need recover (0), my votes (2), total votes (2), commit_lsn (19), lsn (19)
2010-07-07 12:50:01.187: [ OCRRAW][1385945408]proprioo: my id set: (2011771966, 1372893121, 0, 0, 0)
2010-07-07 12:50:01.187: [ OCRRAW][1385945408]proprioo: 1st set: (2011771966, 1028247821, 0, 0, 0)
2010-07-07 12:50:01.187: [ OCRRAW][1385945408]proprioo: 2nd set: (2011771966, 1372893121, 0, 0, 0)
2010-07-07 12:50:01.277: [ OCRRAW][1385945408]propriogid:1_2: INVALID FORMAT
2010-07-07 12:50:04.946: [ OCRRAW][1385945408]propriowv_bootbuf: Vote information on disk 1 [+CLUSTERDG] is adjusted from [0/0]
to [1/2]
2010-07-07 12:50:04.966: [ OCRRAW][1385945408]propriowv_bootbuf: Vote information on disk 0 [/dev/sdc5] is adjusted from [2/2] to [1/2]
2010-07-07 12:50:05.027: [ OCRAPI][1385945408]u_masmd:11: clscrs_register_resource2 succeeded [0]. Return [0]
2010-07-07 12:50:05.027: [ OCRAPI][1385945408]proa_modify_dep: ora.crsd resource in OHAS depends on ora.cssd, ora.ctssd and ora.asm




4. Check new OCR location with
ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 148348
Used space (kbytes) : 4500
Available space (kbytes) : 143848
ID : 552644455
Device/File Name : /dev/sdc5
Device/File integrity check succeeded
Device/File Name : +CLUSTERDG
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Oracle doesn't support having OCR in different storage types except for migration.

5. Delete the block device OCR location
ocrconfig -delete /dev/sdc5
Verify on cluster alert log
2010-07-07 12:51:49.413
[crsd(7409)]CRS-1010:The OCR mirror location /dev/sdc5 was removed.
and on crsd.log
2010-07-07 12:51:37.290: [UiServer][1549629760] S(0x2aaab0034e40): set Properties ( root,0x2105c4b0)
2010-07-07 12:51:37.301: [UiServer][1547528512] processMessage called
2010-07-07 12:51:37.301: [UiServer][1547528512] Sending message to PE. ctx= 0x20fcaa10
2010-07-07 12:51:37.302: [ CRSPE][1543326016] Processing PE command id=165. Description: [Stat Resource : 0x2aaaac35bca0]
2010-07-07 12:51:37.302: [ CRSPE][1543326016] PE Command [ Stat Resource : 0x2aaaac35bca0 ] has completed
2010-07-07 12:51:37.302: [ CRSPE][1543326016] UI Command [Stat Resource : 0x2aaaac35bca0] is replying to sender.
2010-07-07 12:51:37.302: [UiServer][1547528512] Done for ctx=0x20fcaa10
2010-07-07 12:51:49.341: [ OCRRAW][1385945408]propriowv_bootbuf: Vote information on disk 1 [] is adjusted from [1/2] to [2/2]
2010-07-07 12:51:49.383: [ OCRASM][1385945408]proprasmo: ASM cache size is [5MB]
2010-07-07 12:51:49.383: [ OCRASM][1385945408]proprasmo: ASM cache [5MB] enabled for disk group [CLUSTERDG].
2010-07-07 12:51:49.412: [ OCRRAW][1385945408]proprioo: for disk 0 (+CLUSTERDG), id match (1), total id sets, (2) need recover (0), my votes (2), total votes (2), commit_lsn (19), lsn (19)
2010-07-07 12:51:49.412: [ OCRRAW][1385945408]proprioo: my id set: (1372893121, 1028247821, 0, 0, 0)
2010-07-07 12:51:49.413: [ OCRRAW][1385945408]proprioo: 1st set: (2011771966, 1372893121, 0, 0, 0)
2010-07-07 12:51:49.413: [ OCRRAW][1385945408]proprioo: 2nd set: (1372893121, 1028247821, 0, 0, 0)
2010-07-07 12:51:49.475: [ OCRAPI][1385945408]u_masmd:11: clscrs_register_resource2 succeeded [0]. Return [0]
2010-07-07 12:51:49.475: [ OCRAPI][1385945408]proa_modify_dep: ora.crsd resource in OHAS depends on ora.cssd, ora.ctssd and ora.asm
6. Finally check the OCR with
ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 148348
Used space (kbytes) : 4500
Available space (kbytes) : 143848
ID : 552644455
Device/File Name : +CLUSTERDG
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded

Related Posts
Moving OCR, Vote and ASM from Raw Devices to Block Devices
Migrating Voting Disk to ASM in 11gR2 Clusterware
Migrating block devices using ASM instance to ASMLib