ASM Diks Resizing After Underlying Storage Disk Expansion

There are two ways to add more space to an ASM diskgroup. One is to add more disks to the group and second is to expand the disk already in the group. This post shows the step for the latter case. The current setup contains two ASM diskgroup each with exactly one disk.

GROUP_NAME DISK_NAME       GROUP_SIZE_MB DISK_SIZE_MB PATH
---------- --------------- ------------- ------------ ----------
DATA       DATA_0000               10236        10236 /dev/sdb1
FLASH      FLASH_0000              10236        10236 /dev/sdc1

Size of the disk added to the flash diskgroup is roughly 10GB and this will be expanded to 12GB. ASM is used for a standalone database (11.2.0.4) created on VirtulBox VM. The expansion of the underlying storage disk is achieved using VBoxManagment. If this is a SAN LUN then vendor provided tools/commands could be used to achieve the same.

# VBoxManage modifyhd rhel5flash.vdi --resize 12288

Shutdown the GI stack on the server before the disk expansion. Once the expansion is complete GI stack could be started but the additional space won't be visible to Oracle and trying to resize the ASM disk would fail.

SQL> alter diskgroup flash resize disk FLASH_0000 size 12000m;
alter diskgroup flash resize disk FLASH_0000 size 12000m
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15057: specified size of 12000 MB is larger than actual size of 10236 MB

This is because even though the disk was expanded the storage disk partition created for the ASM disk is still at the original size.

fdisk -l /dev/sdc

Disk /dev/sdc: 12.9 GB, 12884901888 bytes
255 heads, 63 sectors/track, 1566 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1        1305    10482381   83  Linux

Only way to resize the partition is to re-create. This would entail deleting the current partition as such stop the GI stack before partition recreation.

$ crsctl stop has

# fdisk /dev/sdc

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): d
Selected partition 1

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1566, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1566, default 1566):
Using default value 1566

Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

Once the partition is recreated verify size has increased

# fdisk -l /dev/sdc

Disk /dev/sdc: 12.9 GB, 12884901888 bytes
255 heads, 63 sectors/track, 1566 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1        1566    12578863+  83  Linux

Start the GI stack and resize ASM disk

crsctl start has
CRS-4123: Oracle High Availability Services has been started.

SQL> alter diskgroup flash resize disk FLASH_0000 size 12000m;

If multiple disks are resized then it is possible to add the re-balance option to the alter diskgroup. ASM alert log the resize operation

SQL> alter diskgroup flash resize disk FLASH_0000 size 12000m
NOTE: requesting all-instance membership refresh for group=2
NOTE: requesting all-instance disk validation for group=2
Thu Jul 30 15:42:42 2015
NOTE: disk validation pending for group 2/0x2c5db20f (FLASH)
SUCCESS: validated disks for 2/0x2c5db20f (FLASH)
NOTE: increased size in header on grp 2 disk FLASH_0000
NOTE: membership refresh pending for group 2/0x2c5db20f (FLASH)
Thu Jul 30 15:42:49 2015
GMON querying group 2 at 9 for pid 13, osid 5803
SUCCESS: refreshed membership for 2/0x2c5db20f (FLASH)
NOTE: starting rebalance of group 2/0x2c5db20f (FLASH) at power 1
SUCCESS: alter diskgroup flash resize disk FLASH_0000 size 12000m
Starting background process ARB0
Thu Jul 30 15:42:49 2015
ARB0 started with pid=28, OS id=6019
NOTE: assigning ARB0 to group 2/0x2c5db20f (FLASH) with 1 parallel I/O
cellip.ora not found.
NOTE: stopping process ARB0
SUCCESS: rebalance completed for group 2/0x2c5db20f (FLASH)
NOTE: Attempting voting file refresh on diskgroup FLASH
Thu Jul 30 15:42:52 2015
NOTE: requesting all-instance membership refresh for group=2
Thu Jul 30 15:42:55 2015
NOTE: membership refresh pending for group 2/0x2c5db20f (FLASH)
GMON querying group 2 at 10 for pid 13, osid 5803
SUCCESS: refreshed membership for 2/0x2c5db20f (FLASH)
NOTE: Attempting voting file refresh on diskgroup FLASH

Query the diskgroup and disk for the increase in size.

GROUP_NAME DISK_NAME       GROUP_SIZE_MB DISK_SIZE_MB PATH
---------- --------------- ------------- ------------ ----------
DATA       DATA_0000               10236        10236 /dev/sdb1
FLASH      FLASH_0000              12000        10236 /dev/sdc1

This concludes resizing of ASM disk after underlying storage expansion.

Useful Metalink Notes
How To Resize an ASM Disk? [ID 373426.1]
Master Note for Automatic Storage Management (ASM) [ID 1187723.1]
Resize of ASM Disk Fails with ORA-600 [KFDATSHRINK_NOTFREE] [ID 2015973.1]
Versions 11.2.0.4 and later: When a LUN is expanded, unable to resize ASM disks online [ID 2000490.1]
Master Note For Automatic Storage Management (ASM) For Operating System (OS) Administrators [ID 1345562.1]

Resizing VirtualBox and Linux Partitions with GParted

VirtualBox 4 has an option to modify the logical size of the virtual disk after it's been created without affecting the physical size. The size specified in this option is the new overall size of the disk not the increment. If the current size is 100MB and resizing to 200MB would make the disk 200MB not 300MB (100 + 200). Once the size is increased it cannot be decreased (yet).

Current size of the disk

-rw------- 1 root root  18G Jan 27 16:43 rac4.vdi

The size of disk seen by the OS using this virutal disk

Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Size of the partitions

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              16G   13G  1.8G  88% /
/dev/sda1              99M   12M   83M  13% /boot
tmpfs                1006M     0 1006M   0% /dev/shm

Increase the size with modifyhd option

VBoxManage modifyhd rac4.vdi --resize 40960
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%

Physical size is unaffected. This will grow as the space is used.

-rw------- 1 root root  18G Jan 27 16:45 rac4.vdi

But the size of the disk seen by the OS has changed

Disk /dev/sda: 42.9 GB, 42949672960 bytes
255 heads, 63 sectors/track, 5221 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

The additional space is unallocated and linux partitions are not affected after this increase.

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              16G   13G  1.8G  88% /
/dev/sda1              99M   12M   83M  13% /boot
tmpfs                1006M     0 1006M   0% /dev/shm

Using the Gnome Partition Editor (GParted) it is possible to resize the linux partition and allocate the unused space to it effectively increasing the size of it.

Download the Live CD iso of gparted and boot the virtual box server from it.

Select the default option and complete the startup.

GParted shows the size of current partitions (15.9 GB) and the unallocated space (20 GB).

Right click on the partition to resize and select resize/move item from the popup menu.

Resize the partition by either dragging the highlighted box or adjusting the numerical spinners.

Click apply to complete the resizing.

Restart the virtual server and see the size of linux partition changed.

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              35G   13G   21G  39% /
/dev/sda1              99M   12M   83M  13% /boot
tmpfs                1006M     0 1006M   0% /dev/shm

Size of the new partition is 35 GB (15 GB previously allocated and 20 GB added with GParted).

Tablespace Resize

query to see if tablespaces can be reduced in size

set verify off
column file_name format a50 word_wrapped
column smallest format 999,990 heading "Smallest|Size|Poss."
column currsize format 999,990 heading "Current|Size"
column savings  format 999,990 heading "Poss.|Savings"
break on report
compute sum of savings on report

column value new_val blksize
select value from v$parameter where name = 'db_block_size'
/

select file_name,
       ceil( (nvl(hwm,1)*&&blksize)/1024/1024 ) smallest,
       ceil( blocks*&&blksize/1024/1024) currsize,
       ceil( blocks*&&blksize/1024/1024) -
       ceil( (nvl(hwm,1)*&&blksize)/1024/1024 ) savings
from dba_data_files a,
     ( select file_id, max(block_id+blocks-1) hwm
         from dba_extents
        group by file_id ) b
where a.file_id = b.file_id(+)
/

column cmd format a75 word_wrapped

select 'alter database datafile '''||file_name||''' resize ' ||
       ceil( (nvl(hwm,1)*&&blksize)/1024/1024 )  || 'm;' cmd
from dba_data_files a,
     ( select file_id, max(block_id+blocks-1) hwm
         from dba_extents
        group by file_id ) b
where a.file_id = b.file_id(+)
  and ceil( blocks*&&blksize/1024/1024) -
      ceil( (nvl(hwm,1)*&&blksize)/1024/1024 ) > 0
/