Saturday, August 1, 2015

ASM Diks Resizing After Underlying Storage Disk Expansion

There are two ways to add more space to an ASM diskgroup. One is to add more disks to the group and second is to expand the disk already in the group. This post shows the step for the latter case. The current setup contains two ASM diskgroup each with exactly one disk.
GROUP_NAME DISK_NAME       GROUP_SIZE_MB DISK_SIZE_MB PATH
---------- --------------- ------------- ------------ ----------
DATA       DATA_0000               10236        10236 /dev/sdb1
FLASH      FLASH_0000              10236        10236 /dev/sdc1
Size of the disk added to the flash diskgroup is roughly 10GB and this will be expanded to 12GB. ASM is used for a standalone database (11.2.0.4) created on VirtulBox VM. The expansion of the underlying storage disk is achieved using VBoxManagment. If this is a SAN LUN then vendor provided tools/commands could be used to achieve the same.
# VBoxManage modifyhd rhel5flash.vdi --resize 12288
Shutdown the GI stack on the server before the disk expansion. Once the expansion is complete GI stack could be started but the additional space won't be visible to Oracle and trying to resize the ASM disk would fail.
SQL> alter diskgroup flash resize disk FLASH_0000 size 12000m;
alter diskgroup flash resize disk FLASH_0000 size 12000m
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15057: specified size of 12000 MB is larger than actual size of 10236 MB
This is because even though the disk was expanded the storage disk partition created for the ASM disk is still at the original size.
fdisk -l /dev/sdc

Disk /dev/sdc: 12.9 GB, 12884901888 bytes
255 heads, 63 sectors/track, 1566 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1        1305    10482381   83  Linux


Only way to resize the partition is to re-create. This would entail deleting the current partition as such stop the GI stack before partition recreation.
$ crsctl stop has

# fdisk /dev/sdc

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): d
Selected partition 1

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1566, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1566, default 1566):
Using default value 1566

Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
Once the partition is recreated verify size has increased
# fdisk -l /dev/sdc

Disk /dev/sdc: 12.9 GB, 12884901888 bytes
255 heads, 63 sectors/track, 1566 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1        1566    12578863+  83  Linux
Start the GI stack and resize ASM disk
crsctl start has
CRS-4123: Oracle High Availability Services has been started.

SQL> alter diskgroup flash resize disk FLASH_0000 size 12000m;
If multiple disks are resized then it is possible to add the re-balance option to the alter diskgroup. ASM alert log the resize operation
SQL> alter diskgroup flash resize disk FLASH_0000 size 12000m
NOTE: requesting all-instance membership refresh for group=2
NOTE: requesting all-instance disk validation for group=2
Thu Jul 30 15:42:42 2015
NOTE: disk validation pending for group 2/0x2c5db20f (FLASH)
SUCCESS: validated disks for 2/0x2c5db20f (FLASH)
NOTE: increased size in header on grp 2 disk FLASH_0000
NOTE: membership refresh pending for group 2/0x2c5db20f (FLASH)
Thu Jul 30 15:42:49 2015
GMON querying group 2 at 9 for pid 13, osid 5803
SUCCESS: refreshed membership for 2/0x2c5db20f (FLASH)
NOTE: starting rebalance of group 2/0x2c5db20f (FLASH) at power 1
SUCCESS: alter diskgroup flash resize disk FLASH_0000 size 12000m
Starting background process ARB0
Thu Jul 30 15:42:49 2015
ARB0 started with pid=28, OS id=6019
NOTE: assigning ARB0 to group 2/0x2c5db20f (FLASH) with 1 parallel I/O
cellip.ora not found.
NOTE: stopping process ARB0
SUCCESS: rebalance completed for group 2/0x2c5db20f (FLASH)
NOTE: Attempting voting file refresh on diskgroup FLASH
Thu Jul 30 15:42:52 2015
NOTE: requesting all-instance membership refresh for group=2
Thu Jul 30 15:42:55 2015
NOTE: membership refresh pending for group 2/0x2c5db20f (FLASH)
GMON querying group 2 at 10 for pid 13, osid 5803
SUCCESS: refreshed membership for 2/0x2c5db20f (FLASH)
NOTE: Attempting voting file refresh on diskgroup FLASH
Query the diskgroup and disk for the increase in size.
GROUP_NAME DISK_NAME       GROUP_SIZE_MB DISK_SIZE_MB PATH
---------- --------------- ------------- ------------ ----------
DATA       DATA_0000               10236        10236 /dev/sdb1
FLASH      FLASH_0000              12000        10236 /dev/sdc1
This concludes resizing of ASM disk after underlying storage expansion.

Useful Metalink Notes
How To Resize an ASM Disk? [ID 373426.1]
Master Note for Automatic Storage Management (ASM) [ID 1187723.1]
Resize of ASM Disk Fails with ORA-600 [KFDATSHRINK_NOTFREE] [ID 2015973.1]
Versions 11.2.0.4 and later: When a LUN is expanded, unable to resize ASM disks online [ID 2000490.1]
Master Note For Automatic Storage Management (ASM) For Operating System (OS) Administrators [ID 1345562.1]

Wednesday, July 1, 2015

Converting .ppk to OpenSSH

PuTTY Private Key (.ppk) could be exported to an OpenSSH key allowing it to be used with an OpenSSH client on linux terminal.
1. The private key contained within the ppk is exported to OpenSSH with the use of puttygen.exe
2. Run puttygen.exe and load the existing private key file

Same could be achieved on command line with
puttygen.exe puttykey.ppk  -O private-openssh
When the file is loading it will prompt for the passphrase.
3. Once the correct passphrase is entered the private key will be loaded.

4. Select "Export OpenSSH key" item under the conversions menu and save the key.

5. Copy the key file to linux server to be be used with the OpenSSH client.



6. Change the protection of the key file to 400. Without it ssh client will issue the following warning and will not connect.
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for 'myssh.key' are too open.
It is recommended that your private key files are NOT accessible by others.
This private key will be ignored.
bad permissions: ignore key: myssh.key
Once the file permission is changed, it could be used with the OpenSSH client.
chmod 400 myssh.key
$ ssh oracle@dbserver  -p 2356 -i myssh.key
Enter passphrase for key 'myssh.key':

Monday, June 22, 2015

Can't exec "/usr/bin/lsb_release": No such file or directory at GI_HOME/lib/osds_acfslib.pm line 564

Following was observed during the GI PSU 11.2.4.0.6 patch application
2015-06-15 11:19:01: acfs is supported
2015-06-15 11:19:01: Executing '/opt/app/oracle/product/11.2.0/grid_4/bin/acfsroot install'
2015-06-15 11:19:01: Executing cmd: /opt/app/oracle/product/11.2.0/grid_4/bin/acfsroot install
2015-06-15 11:23:00: Command output:
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  ACFS-9300: ADVM/ACFS distribution files found.
>  ACFS-9312: Existing ADVM/ACFS installation detected.
>  ACFS-9314: Removing previous ADVM/ACFS installation.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  ACFS-9315: Previous ADVM/ACFS components successfully removed.
>  ACFS-9307: Installing requested ADVM/ACFS software.
>  ACFS-9308: Loading installed ADVM/ACFS drivers.
>  ACFS-9321: Creating udev for ADVM/ACFS.
>  ACFS-9323: Creating module dependencies - this may take some time.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  ACFS-9154: Loading 'oracleoks.ko' driver.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  ACFS-9154: Loading 'oracleadvm.ko' driver.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  ACFS-9154: Loading 'oracleacfs.ko' driver.
>  Can't exec "/usr/bin/lsb_release": No such file or directory at /opt/app/oracle/product/11.2.0/grid_4/lib/osds_acfslib.pm line 564.
>  ACFS-9327: Verifying ADVM/ACFS devices.
>  ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
>  ACFS-9156: Detecting control device '/dev/ofsctl'.
>  ACFS-9309: ADVM/ACFS installation correctness verified.
>End Command output



The environment is OEL 6 (2.6.32-358.18.1.el6.x86_64).
Reason for this error is bug 17359415. Install redhat-lsb-core-4.0*.el6.x86_64 rpm to fix this problem.
# yum install redhat-lsb-core-4.0*.el6.x86_64
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package redhat-lsb-core.x86_64 0:4.0-7.0.1.el6 will be installed