Direct NFS Setup

This post shows steps for setting up directNFS for 11.2 and 12.1 (steps are identical). For 11.1 please refer here.
Direct NFS requires a working NFS setup to already exists. Create a NFS setup and mount the NFS on the local server with oracle recommended nfs mount options (refer 359515.1) For this test case (tested with 11.2 and 12.1 both SE, SE2 and EE) nfs mount is as follows

# mount
...
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
192.168.0.104:/opt/backup on /usr/local/ochm/nfsdir type nfs (rw,hard,rsize=32768,wsize=32768,nfsvers=3,nointr,timeo=600,tcp,nolock,actimeo=0,addr=192.168.0.104)

Add an entry to fstab as well

# cat /etc/fstab
...
192.168.0.104:/opt/backup /usr/local/ochm/nfsdir        nfs     rw,hard,rsize=32768,wsize=32768,nfsvers=3,nointr,timeo=600,tcp,nolock,actimeo=0,addr=192.168.0.104 0 0

Test to see if the oracle user has write permission on the nfs mount point by creating dummy files. When the nfs is working without any issues configure the oracle to use direct nfs.
Enable direct nfs by executing the rdbms make with dnfs on.

cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk dnfs_on
rm -f /opt/app/oracle/product/12.1.0/std2/rdbms/lib/odm/libnfsodm12.so; \
    cp /opt/app/oracle/product/12.1.0/std2/lib/libnfsodm12.so /opt/app/oracle/product/12.1.0/std2/rdbms/lib/odm/libnfsodm12.so

To disable direct NFS use

make -f ins_rdbms.mk dnfs_off
rm -f  /opt/app/oracle/product/12.1.0/std2/rdbms/lib/odm/libnfsodm12.so

Verify if the oradism is owned by root and has setuid.

# ls -l $ORACLE_HOME/bin/oradism
-rwsr-x--- 1 root oinstall 71758 Sep 17  2011 /opt/app/oracle/product/12.1.0/std2/bin/oradism

If oradism has wrong permission or ownership correct it as follows

# chown root:oinstall $ORACLE_HOME/bin/oradism
# chmod 4755 $ORACLE_HOME/bin/oradism

Direct NFS searches for mount entries in following files in the given order

1.$ORACLE_HOME/dbs/oranfstab
2. /etc/oranfstab
3. /etc/mtab

In this case an oranfstab file is created as follows

cat oranfstab
server: hpc1nfsmount
local: 192.168.0.66
path:   192.168.0.104
export: /opt/backup mount: /usr/local/ochm/nfsdir

local: 192.168.0.66 is the IP of the local server where the database is running. path: 192.168.0.104 is the IP of the server which provides the NFS storage (i.e. NFS server). export: /opt/backup is the export location on the NFS server. mount: /usr/local/ochm/nfsdir is the mount point local server.

Once the dnfs is enabled start the database. If the dnfs is in use the alert log will have the following line

Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 3.0

However there won't be any dnfs related info available on the views

SQL> select * from v$dnfs_servers;

no rows selected

SQL> select * from v$dnfs_files;

no rows selected

Create a tablespace using the nfs location.

SQL> create tablespace nfstbs datafile '/usr/local/ochm/nfsdir/nfs.dbf'  size 10m;

When the tablespace is being created the alert log will show the dnfs being used

Fri Oct 09 11:16:01 2015
create tablespace nfstbs datafile '/usr/local/ochm/nfsdir/nfs.dbf'  size 10m
Fri Oct 09 11:16:02 2015
Direct NFS: channel id [0] path [192.168.0.104] to filer [192.168.0.104] via local [] is UP
Completed: create tablespace nfstbs datafile '/usr/local/ochm/nfsdir/nfs.dbf'  size 10m

Once the tablespace is created the dnfs views will show the relevant information

SQL> select * from v$dnfs_servers;

        ID SVRNAME              DIRNAME                           MNTPORT    NFSPORT      WTMAX      RTMAX
---------- -------------------- ------------------------------ ---------- ---------- ---------- ----------
         1 192.168.0.104        /opt/backup                         56904       2049    1048576    1048576
   
SQL> select * from v$dnfs_files;

FILENAME                         FILESIZE       PNUM     SVR_ID
------------------------------ ---------- ---------- ----------
/usr/local/ochm/nfsdir/nfs.dbf   10493952         10          1

The same servers and setup was used to test the direct nfs on 11.1 SE as well. The dNFS ODM version is lower than 11.2 and 12.1

Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 2.0

The NFS version was v3

# nfsstat | grep nfs
Server nfs v3:
Client nfs v3:

However this did not succeed creating tablespaces in the 11.1 even after mounting the nfs with a lower version

192.168.0.104:/opt/backup on /usr/local/ochm/nfsdir type nfs (rw,hard,rsize=32768,wsize=32768,nfsvers=2,nointr,timeo=600,tcp,nolock,actimeo=0,addr=192.168.0.104)

Related Post
RMAN Backups on NFS