Monday, April 19, 2021

Enable/Disable ASM Rebalance Compat Phase

The rebalnce operation in ASM consist of three phases one of which is compact. As per 1902001.1 the compact phase move data closer to outer tracks of the disk. This could be observed during adding or removing disk from ASM diskgroup
INST_ID GROUP_NUMBER OPERA PASS      STAT      POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE EST_MINUTES ERROR_CODE                                       CON_ID
---------- ------------ ----- --------- ---- ---------- ---------- ---------- ---------- ---------- ----------- -------------------------------------------- ----------
         1            1 REBAL COMPACT   RUN           1          1       1378          0        991           0                                                       0
         1            1 REBAL REBALANCE DONE          1          1       3930       3930          0           0                                                       0
         1            1 REBAL REBUILD   DONE          1          1          0          0          0           0                                                       0



   INST_ID GROUP_NUMBER OPERA PASS      STAT      POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE EST_MINUTES ERROR_CODE                                       CON_ID
---------- ------------ ----- --------- ---- ---------- ---------- ---------- ---------- ---------- ----------- -------------------------------------------- ----------
         1            1 REBAL COMPACT   REAP          1          1       1400          0          0           0                                                       0
         1            1 REBAL REBALANCE DONE          1          1       3930       3930          0           0                                                       0
         1            1 REBAL REBUILD   DONE          1          1          0          0          0           0                                                       0
As seen above the COMPACT phase run with some work and come to a conclusing (REAP status).

However, the track concept is not valid for SSD (see here and here). As such the compact phase could be completely avoided to save time on rebalance. This could be done by setting "_DISABLE_REBALANCE_COMPACT=false" on ASM instance (pre-12c systems) or by setting "_rebalance_compact'='FALSE' on the ASM diskgroup (12c and up). The key differences is (beside different names) one is for ASM instance while other is for ASM diskgroup.



When this is done the rebalance operation skip the compact phase and bring it to conclusion straight away. Same operation above with _rebalance_compact set to false on the disk group shows the folllowing.
   INST_ID GROUP_NUMBER OPERA PASS      STAT      POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE EST_MINUTES ERROR_CODE                                       CON_ID
---------- ------------ ----- --------- ---- ---------- ---------- ---------- ---------- ---------- ----------- -------------------------------------------- ----------
         1            1 REBAL COMPACT   WAIT          1          1          0          0          0           0                                                       0
         1            1 REBAL REBALANCE RUN           1          1       3892       3930       1857           0                                                       0
         1            1 REBAL REBUILD   DONE          1          1          0          0          0           0                                                       0



  INST_ID GROUP_NUMBER OPERA PASS      STAT      POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE EST_MINUTES ERROR_CODE                                       CON_ID
---------- ------------ ----- --------- ---- ---------- ---------- ---------- ---------- ---------- ----------- -------------------------------------------- ----------
         1            1 REBAL COMPACT   REAP          1          1          0          0          0           0                                                       0
         1            1 REBAL REBALANCE DONE          1          1       3930       3930          0           0                                                       0
         1            1 REBAL REBUILD   DONE          1          1          0          0          0           0                                                       0
So this looks like a best practice to have on systems using flash disks. But as this is a hidden parameter and consulting Oracle support on the merit of this didn't give confidence. Oracle support's comment was "we don't have official information from development that compact phase can be disabled as per the Best practices". There maybe some validity on this as 2198773.1 shows setting _rebalance_compact to false could cause issues with regard to rebalance operation completing (doesn't mention what type of disks were used). MOS doc 1573768.1 also advises to set _DISABLE_REBALANCE_COMPACT=TRUE. However, testing above scenarios on 19c (multuple times) didn't result in any issues.
All in all there's no conclusive advise on the matter. Lot of Oracle community posts for setting _rebalance_compact to false while offical Oracle support and documents state otherwise.

Related Metalink notes
WARNING: rebalance not completed for group [ID 2198773.1]
What is ASM rebalance compact Phase and how it can be disabled [ID 1902001.1]
Can ASM “_DISABLE_REBALANCE_COMPACT=TRUE" Be Used With NetApp SAN Environment? [ID 1573768.1]