Database/Instance Recovery in RAC

When a database starts oracle performs two consistency checks (among others).

1. Check if the start SCN value of each datafile header matches the corresponding stop SCN value in the controlfile.
2. Check if the checkpoint counter values matches.
   

If these two checks are successful then no instance recovery is needed.

If datafile header SCNs are out of sync then at least an instance recovery is needed.

If checkpoint counter value check fails then Oracle knows datafile was replaced with a backup copy while it was down and requires a media recovery.

Instance recovery is completed when Oracle has performed

1. Cache recovery : replays the contents of the online redologs of the failed instance.
   
2. Transaction recovery : rollback the uncommitted transactions of the failed instance.

• During the first phase of the recovery GES (Global Enque Service) remaster the enqueues and GCS (Global Cache Service) remaster its resources from the failed instance among the remaining instances.
• First step in GCS remastering is for Oracle to assign a new incarnation number.
  
• Oracle determines how many more nodes are left in the cluster.
• In an attempt to recreate the resource master of the failed node all GCS resource request and writer requests are temporarily suspended. GRD (Global Resource Directory) is frozen.
• All the dead shadow process related to the GCS are cleaned up from the failed instance.
• After enqueues are reconfigured one of the surviving instances grab the instance recovery enqueue.
• Same time GCS resources are remastered SMON determines the blcoks that need recovery. This is known as the recovery set. Due to the nature of the cache fusion SMON needs to merge contents of all online redo logs of each failed instance to determine the blocks that need recovery (recovery set) and the order of apply.
• In this stage buffer space for recovery is allocated and GCS resources identified by reading the online redo logs are claimed as recovery resources. This prevent the other instances accessing these resources.
• SMON performes roll forward (cache recovery) and roll back (transaction recovery).
• A new master node is assigned to the cluster if the failed node was the previous master. All GCS shadow processes are traversed and GRD is removed from the frozen state. This complete the reconfiguration process.

Instance Recovery in Single Instance Database

When a database starts oracle performs two consistency checks (among others).

1. Check if the start SCN value of each datafile header matches the corresponding stop SCN value in the controlfile.
   
2. Check if the checkpoint counter values matches.

If these two checks are successful then no instance recovery is needed.

If datafile header SCNs are out of sync then at least an instance recovery is needed.

If checkpoint counter value check fails then Oracle knows datafile was replaced with a backup copy while it was down and requires a media recovery.

Instance recovery is completed when Oracle has performed

1. Cache recovery : replays the contents of the online redologs of the failed  instance.
   
2. Transaction recovery : rollback the uncommitted transactions of the failed instance.

In a nutshell instance recovery is nothing more than using the contents in the online redo logs to rebuild the database buffer cache to the state it was in before the crash.

At startup Database goes through nomount and mount phases. During the checks if it is found that instance recovery is needed then SMON automatically starts the instance recovery process. This phase is know as the roll forward phase and it reinstates all changes to both data blocks as well as undo blocks, for both committed and uncommitted transactions.

Each redo record has bare minimum information to reconstruct a change (block address, new value). In the roll forward phase blocks are brought into the database buffer cache, changes are applied and then written back to the disk.

At the end of the roll forward phase cache recovery is completed and it as  crash never happened.
Once the roll forward phase is over database is opened for users to connect. 

But there are still uncommitted transactions in the database. 
To complete the transaction recovery these uncommitted transactions needs to be rolled back. 
Oracle does this automatically during rollback phase. If  a user connects and hit a datablock that 
needs to be rolled back then this is done automatically as roll forward phase has populated the undo segments that was protecting the uncommitted transaction. This ensures the read consistency.