Saturday, August 24, 2013

Installing 12c (12.1.0.1) RAC on RHEL 6 with Role Separation - Creating CDB & PDB

After installing clusterware and database software remaining step is to create the database. In this post a container database (CDB) and a pluggable database (PDB) will be created using DBCA. CDB and PDB are part of 12c new multitenant feature.
1. Before starting DBCA change the permission on cfgtoollogs and admin folders. cfgtoolslogs permission was an issue in 11.2 when installed with role separation as well(see step 15). In 12c admin directory in ORACLE_BASE also causes issue as it doesn't have write permission for the oinstall group thus making oracle user unable to create necessary directories inside it.

To fix it change to ORACLE_BASE as grid user (or root) and execute the following on all nodes
cd $ORACLE_BASE
chmod 770 cfgtoollogs
chmod 770 admin
2. Verify the database installation pre-req check with cluvfy -pre dbcfg
[oracle@rhel12c1 ~]$ cluvfy stage -pre dbcfg -n rhel12c1,rhel12c2 -d /opt/app/oracle/product/12.1.0/dbhome_1

Performing pre-checks for database configuration

Checking node reachability...
Node reachability check passed from node "rhel12c1"


Checking user equivalence...
User equivalence check passed for user "oracle"

Checking node connectivity...

Checking hosts config file...

Verification of the hosts config file successful

Check: Node connectivity using interfaces on subnet "192.168.0.0"
Node connectivity passed for subnet "192.168.0.0" with node(s) rhel12c2,rhel12c1
TCP connectivity check passed for subnet "192.168.0.0"


Check: Node connectivity using interfaces on subnet "192.168.1.0"
Node connectivity passed for subnet "192.168.1.0" with node(s) rhel12c2,rhel12c1
TCP connectivity check passed for subnet "192.168.1.0"

Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.0.0".
Subnet mask consistency check passed for subnet "192.168.1.0".
Subnet mask consistency check passed.

Node connectivity check passed

Checking multicast communication...

Checking subnet "192.168.1.0" for multicast communication with multicast group "224.0.0.251"...
Check of subnet "192.168.1.0" for multicast communication with multicast group "224.0.0.251" passed.

Check of multicast communication passed.
Group existence check passed for "oper"
Group existence check passed for "dba"
Group existence check passed for "backupdba"
Group existence check passed for "dgdba"
Group existence check passed for "kmdba"
Total memory check passed
Available memory check passed
Swap space check passed
Free disk space check passed for "rhel12c2:/tmp"
Free disk space check passed for "rhel12c1:/tmp"
Check for multiple users with UID value 500 passed
User existence check passed for "oracle"
Group existence check passed for "oinstall"
Group existence check passed for "dba"
Membership check for user "oracle" in group "oinstall" [as Primary] passed
Membership check for user "oracle" in group "dba" passed
Run level check passed
Hard limits check passed for "maximum open file descriptors"
Soft limits check passed for "maximum open file descriptors"
Hard limits check passed for "maximum user processes"
Soft limits check passed for "maximum user processes"
System architecture check passed
Kernel version check passed
Kernel parameter check passed for "semmsl"
Kernel parameter check passed for "semmns"
Kernel parameter check passed for "semopm"
Kernel parameter check passed for "semmni"
Kernel parameter check passed for "shmmax"
Kernel parameter check passed for "shmmni"
Kernel parameter check passed for "shmall"
Kernel parameter check passed for "file-max"
Kernel parameter check passed for "ip_local_port_range"
Kernel parameter check passed for "rmem_default"
Kernel parameter check passed for "rmem_max"
Kernel parameter check passed for "wmem_default"
Kernel parameter check passed for "wmem_max"
Kernel parameter check passed for "aio-max-nr"
Package existence check passed for "binutils"
Package existence check passed for "compat-libcap1"
Package existence check passed for "compat-libstdc++-33(x86_64)"
Package existence check passed for "libgcc(x86_64)"
Package existence check passed for "libstdc++(x86_64)"
Package existence check passed for "libstdc++-devel(x86_64)"
Package existence check passed for "sysstat"
Package existence check passed for "gcc"
Package existence check passed for "gcc-c++"
Package existence check passed for "ksh"
Package existence check passed for "make"
Package existence check passed for "glibc(x86_64)"
Package existence check passed for "glibc-devel(x86_64)"
Package existence check passed for "libaio(x86_64)"
Package existence check passed for "libaio-devel(x86_64)"
Check for multiple users with UID value 0 passed
Current group ID check passed

Starting check for consistency of primary group of root user

Check for consistency of root user's primary group passed

Checking CRS integrity...

Clusterware version consistency passed.

CRS integrity check passed

Checking node application existence...

Checking existence of VIP node application (required)
VIP node application check passed

Checking existence of NETWORK node application (required)
NETWORK node application check passed

Checking existence of ONS node application (optional)
ONS node application check passed

Time zone consistency check passed

Pre-check for database configuration was successful.



3.Start DBCA and create the CDB first.
It was noted that the time to create database using custom option without the use of OLTP and warehouse templates takes longer compared to 11.2
It is possible to create both CDB and PDB in one step but here only the CDB is created initially.
With 12c database vault and label security could be configured with DBCA at the time of database creation.

4. PDBs could be created with DBCA as well as using command "create pluggable database". Time it takes to provision a PDB is much shorter compared to creating a database using DBCA in previous releases. This is one of the reasons why oracle is "marketing/advertising/touting" that 12c is the database for the cloud.
Start DBCA to create the PDB.
Each PDB needs a CDB and earlier created CDB is specified as the CDB for this PDB being created.

SRVCTL could be used to manage PDBs in a RAC environment. Dynamic services must be created for each PDBs. In this for the admin managed CDB and PDBs service is created with following command
srvctl add service -d cdb12c -pdb pdb12c -s pdbsvc -preferred "cdb12c1,cdb12c2"

srvctl config service -d cdb12c -s pdbsvc
Service name: pdbsvc
Service is enabled
Server pool: cdb12c_pdbsvc
Cardinality: 2
Disconnect: false
Service role: PRIMARY
Management policy: AUTOMATIC
DTP transaction: false
AQ HA notifications: false
Global: false
Commit Outcome: false
Failover type:
Failover method:
TAF failover retries:
TAF failover delay:
Connection Load Balancing Goal: LONG
Runtime Load Balancing Goal: NONE
TAF policy specification: NONE
Edition:
Pluggable database name: pdb12c
Maximum lag time: ANY
SQL Translation Profile:
Retention: 86400 seconds
Replay Initiation Time: 300 seconds
Session State Consistency:
Preferred instances: cdb12c1,cdb12c2
Available instances:
Starting the service will open the PDBs in read,write mode.
SQL>  select inst_id,name,open_mode from gv$pdbs;

   INST_ID NAME                           OPEN_MODE
---------- ------------------------------ ----------
         2 PDB12C                         MOUNTED
         1 PDB12C                         MOUNTED

srvctl start service -d cdb12c -s pdbsvc

SQL> select inst_id,name,open_mode from gv$pdbs;

   INST_ID NAME                           OPEN_MODE
---------- ------------------------------ ----------
         2 PDB12C                         READ WRITE
         1 PDB12C                         READ WRITE
on alert log following could be seen
Mon Oct 14 14:14:09 2013
ALTER PLUGGABLE DATABASE pdb12c OPEN  /* svc agent *//* {1:23451:950} */
Mon Oct 14 14:14:09 2013
This instance was first to open pluggable database PDB12C (container=3)
Opening pdb PDB12C (3) with no Resource Manager plan active
Pluggable database PDB12C opened read write
Completed: ALTER PLUGGABLE DATABASE pdb12c OPEN  /* svc agent *//* {1:23451:950} */
Modify the service's auto start status and PDBs will be open in read write mode each time the service is started
# crsctl modify resource ora.cdb12c.pdbsvc.svc -attr "AUTO_START=always"
However stopping the service using srvctl has no effect on the open mode of the PDB. It will stop the service and prevent any new connection coming in but the PDB will remain in read write mode. Even though on Oracle documentation it says stopping the service will stop the PDB.
This text could lead to confusion if one would expect the PDB will be closed when it is stopped. The real behavior of the stop is mentioned in another document which reads (from Admin Guide)"When your database is being managed by Oracle Restart or Oracle Clusterware, and you use the SRVCTL utility to start a service with a PDB property for a PDB that is closed, the PDB is opened in read/write mode on the nodes where the service is started. However, stopping a PDB service does not change the open mode of the PDB."
To stop the PDBs in RAC uses alter pluggable database command with "instances=all".

Creation of the PDB concludes the installing of 12c RAC.

Related Posts
Installing 11gR2 (11.2.0.3) GI with Role Separation on RHEL 6
Installing 11gR2 (11.2.0.3) GI with Role Separation on OEL 6
Installing 11gR2 Standalone Server with ASM and Role Separation on RHEL 6
11gR2 Standalone Data Guard (with ASM and Role Separation)