Showing posts with label redo. Show all posts
Showing posts with label redo. Show all posts

Monday, April 5, 2021

LOG_FILE_NAME_CONVERT is not defined, stop clearing ORLs

LOG_FILE_NAME_CONVERT name parameter is used to correct the file location for log files if the database is being created from another database and source database had a different directory structure. This is used in data guard configuration when standby and primary has different file structure. However, there are few limitations to using this parameter and one of them is that it cannot be used with Oracle managed files or OMF (refer 1367014.1).
However, an odd behaviour was observed in a data guard configuration where even when OMF is used there database was looking for LOG_FILE_NAME_CONVERT and if not set stopping the automatic logfile clearing.
The databases (both primary and standby) had following OMF related parameters
NAME                                 TYPE        VALUE
------------------------------------ ----------- -------
db_create_file_dest                  string      +DATA
db_create_online_log_dest_1          string
db_create_online_log_dest_2          string
db_create_online_log_dest_3          string
db_create_online_log_dest_4          string
db_create_online_log_dest_5          string
db_recovery_file_dest                string      +FRA
The *convert* parameters were not set.
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------
db_file_name_convert                 string
log_file_name_convert                string
pdb_file_name_convert                string
During the active database duplication the duplication processed detected use of OMF as seen from below output.
...
sql statement: alter system set  local_listener =  ''LISTENER_DGTEST2'' comment= '''' scope=spfile

Oracle instance shut down

connected to auxiliary database (not started)
Oracle instance started

Total System Global Area    9965665616 bytes

Fixed Size                    12684624 bytes
Variable Size               1543503872 bytes
Database Buffers            8388608000 bytes
Redo Buffers                  20869120 bytes
duplicating Online logs to Oracle Managed File (OMF) location
duplicating Datafiles to Oracle Managed File (OMF) location

contents of Memory Script:
{
   sql clone "alter system set  control_files =
  ''+DATA/DGTEST2/CONTROLFILE/current.264.1067949389'', ''+FRA/DGTEST2/CONTROLFILE/current.418.1067949389'' comment=
  ...
Data guard consists of two databases.
DGMGRL>Configuration - test_dg

  Protection Mode: MaxAvailability
  Members:
  dgtest  - Primary database
    dgtest2 - Physical standby database

Fast-Start Failover:  Disabled

Configuration Status:
SUCCESS
With these setting on, after a switchover following was seen on the alert log of the old primary (new standby)
2021-03-23T12:55:20.314959+00:00
TT02 (PID:8514): LOG_FILE_NAME_CONVERT is not defined, stop clearing ORLs
Validate of the old primary (new standby) showed that online redo logs are not cleared
DGMGRL> validate database dgtest

  Database Role:     Physical standby database
  Primary Database:  dgtest2

  Ready for Switchover:  Yes
  Ready for Failover:    Yes (Primary Running)

  Managed by Clusterware:
    dgtest2:  YES
    dgtest :  YES

  Log Files Cleared:
    dgtest2 Standby Redo Log Files:  Cleared
    dgtest Online Redo Log Files:    Not Cleared
    dgtest Standby Redo Log Files:   Available
It was the same case when the new primary (old standby) did a switchover as well.
2021-03-23T13:02:25.800263+00:00
TT02 (PID:9657): LOG_FILE_NAME_CONVERT is not defined, stop clearing ORLs

DGMGRL> validate database dgtest2

  Database Role:     Physical standby database
  Primary Database:  dgtest

  Ready for Switchover:  Yes
  Ready for Failover:    Yes (Primary Running)

  Managed by Clusterware:
    dgtest :  YES
    dgtest2:  YES

  Log Files Cleared:
    dgtest Standby Redo Log Files:   Cleared
    dgtest2 Online Redo Log Files:   Not Cleared
    dgtest2 Standby Redo Log Files:  Available


This behaviour was observed even after applying release updates (RU) 19.3, 19.4, 19.5, 19.6 and 19.7. If the databases are in one of those RUs then only way to achieve the automatic ORL clearing is to explicitly set the log file name convert parameter. For example in this case for the primary DB dgtest
alter system set log_file_name_convert='/dgtest2/','/dgtest/' scope=spfile;
Once log_file_name_convert is set the automatic ORL clearence was observed even though MOS doc states log_file_name_convert cannot be used with OMF.

The issue is fixed on RU 19.8 and up (tested on 19.9 , 19.10). Once these RU are applied automatic ORL clearing happens as expected. If previously (in one of the affected RUs) log_file_name_convert was set then it could be removed once RU 19.8 or up is applied.
When automatic ORL clearing happens following could be seen on the alert log of the old primary
2021-03-23T13:20:23.135283+00:00
TT02 (PID:10641): Waiting for all non-current ORLs to be archived
2021-03-23T13:20:23.135414+00:00
TT02 (PID:10641): All non-current ORLs have been archived
2021-03-23T13:20:23.273380+00:00
TT02 (PID:10641): Clearing ORL LNO:1 +DATA/DGTEST/ONLINELOG/group_1.264.1067878075
2021-03-23T13:20:24.512745+00:00
TT02 (PID:10641): Clearing ORL LNO:1 complete
TT02 (PID:10641): Clearing ORL LNO:2 +DATA/DGTEST/ONLINELOG/group_2.262.1067878077
Clearing online log 2 of thread 1 sequence number 34
2021-03-23T13:20:26.525504+00:00
TT02 (PID:10641): Clearing ORL LNO:2 complete
TT02 (PID:10641): Clearing ORL LNO:3 +DATA/DGTEST/ONLINELOG/group_3.261.1067878079
Clearing online log 3 of thread 1 sequence number 35
2021-03-23T13:20:28.163683+00:00
TT02 (PID:10641): Clearing ORL LNO:3 complete
2021-03-23T13:20:33.174192+00:00
TT02 (PID:10641): Waiting for all non-current ORLs to be archived
2021-03-23T13:20:33.174386+00:00
TT02 (PID:10641): All non-current ORLs have been archived
Validating the standby shows no log files to be cleared.
DGMGRL> validate database dgtest2

  Database Role:     Physical standby database
  Primary Database:  dgtest

  Ready for Switchover:  Yes
  Ready for Failover:    Yes (Primary Running)

  Managed by Clusterware:
    dgtest :  YES
    dgtest2:  YES

Useful Metalink notes
Usage and Limitation of db_file_name_convert and log_file_name_convert [ID 1367014.1]

Thursday, November 1, 2018

Verifying Redo Transport Compression in 11.2, 12.1 and 12.2 Data Guard Configuraitons

Redo shipped to standby could be compressed for transport in two ways. One is external to the database and uses SSH tunnelling. Other method is enabling compression for the remote log archive destination. Second option requires advance compression license. Once redo transport compression (RTC) is enabled it could verified by enabling log archive tracing. However, there's a difference as to level of tracing and information for DB versions 11.2, 12.1 and 12.2. This posts explore these variations.

Verifying RTC in 11.2.0.4
The DG configuration for 11.2 is as follows
DGMGRL> show configuration

Configuration - db_dg

  Protection Mode: MaxAvailability
  Databases:
    prod  - Primary database
    stdby - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS
Enable RTC on the standby database. This updates the log archive destination setting on the primary by adding compression=enable option.
DGMGRL> edit database stdby set property RedoCompression='enable';
Property "redocompression" updated
On the alert log of the primary following could be seen.
Tue Aug 14 14:29:41 2018
ALTER SYSTEM SET log_archive_dest_3='service="stdbytns"','LGWR SYNC AFFIRM delay=0 optional compression=enable max_failure=10 max_connections=5 reopen=180 db_unique_name="stdby" net_timeout=30','valid_for=(all_logfiles,primary_role)' SCOPE=BOTH;
Enabling log archive tracing level of 1 for primary database
DGMGRL>  edit database prod set property LogArchiveTrace='1';
Property "logarchivetrace" updated
Enable log archive tracing level of 16 for standby
DGMGRL> edit database stdby set property LogArchiveTrace='16';
Property "logarchivetrace" updated
With these setting in place on the primary *nss*.trc files will contain entries similar to following to show that RTC is enabled.
RTC: Compression ratio for this I/O is 81%
RTC: Compression ratio for this I/O is 81%
RTC: Compression ratio for this I/O is 81%
RTC: Compression ratio for this I/O is 80%
On the standby the decompression information is shown in *rsf*.trc files as follows
RTC: decompressed 94 bytes to 512 bytes
RTC: decompressed 93 bytes to 512 bytes
RTC: decompressed 99 bytes to 512 bytes
RTC: decompressed 88 bytes to 512 bytes
RTC: decompressed 5221 bytes to 24576 bytes
If the tracing on the primary is increased to 16 (which Oracle document says as "Track detailed ARCHIVELOG destination activity") only limited RTC information is given out.
edit database prod set property LogArchiveTrace='16';
Both *arc*.trc and *lgwr*.trc files will have entries similar to following.
cat prod_arc7_3372.trc | grep RTC
RTC enabled.
cat prod_lgwr_3297.trc | grep RTC
RTC enabled.
RTC enabled.
Verifying RTC in 12.1.0.2
The 12.1.0.2 DG configuration is as follows
DGMGRL> show configuration;

Configuration - ent12c1_dgb

  Protection Mode: MaxAvailability
  Members:
  ent12c1  - Primary database
    fs12c1   - Far sync instance
      ent12c1s - Physical standby database

  Members Not Receiving Redo:
  fs12c1s  - Far sync instance

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS
On 12.1 setting log archive trace to 1 has no effect on tracing RTC. Setting the value to 16 on primary gives descriptive RTC information which indicates if RTC is enabled or not.
DGMGRL>  edit database ent12c1 set property logarchivetrace='16';
Property "logarchivetrace" updated
The RTC related information appear in *arc*.trc files and *tt*.trc files
cat ent12c1_arc3_15498.trc | grep RTC
RTC enabled.
RTC: Allocating 11534336 buffer bytes for compression.
RTC: header-batch write block 1, count 1 - RTC disabled
RTC: Enable compression for subsequent batches
RTC: actual compressed bytes# 1295

cat ent12c1_tt01_15532.trc | grep RTC
RTC enabled.
Re-enabling RTC following backoff interval 1.
RTC: actual compressed bytes# 10270
RTC: actual compressed bytes# 536
RTC: actual compressed bytes# 1697
RTC: actual compressed bytes# 508
Redo compression could be enabled on far sync instances as well. Setting trace level to 16 on far sync instance will also generate RTC information on far sync trace files.
DGMGRL> edit far_sync fs12c1 set property RedoCompression='enable';
Property "redocompression" updated

DGMGRL> edit far_sync fs12c1 set property LogArchiveTrace='16';
Property "logarchivetrace" updated

cat fs12c1_tt00_2515.trc | grep RTC
RTC: actual compressed bytes# 97
RTC: actual compressed bytes# 79
To get decompression related information from standby set log archive trace value to 1024 (1024: Tracks RFS physical client).
DGMGRL> edit database ent12c1s set property logarchivetrace='1024';
Property "logarchivetrace" updated
Decompression related information is shown in *rfs*.trc files.
cat ent12c1s_rfs_8859.trc | grep RTC
RTC: decompressed 79 bytes to 512 bytes
RTC: decompressed 81 bytes to 512 bytes
RTC: decompressed 97 bytes to 512 bytes
RTC: decompressed 79 bytes to 512 bytes


Verifying RTC in 12.2.0.1 (and 18.3.0.0)
The 12.2 DG configuration is as follows
DGMGRL> show configuration

Configuration - dg12c2

  Protection Mode: MaxAvailability
  Members:
  prodcdb - Primary database
    stbycdb - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS   (status updated 35 seconds ago)
On 12.2 setting log archive trace to 16 on primary generates the RTC information.
DGMGRL> edit database prodcdb set property LogArchiveTrace='16';
Property "logarchivetrace" updated
The RTC information is available on *lgwr*.trc and *nss*.trc files.
cat prodcdb_lgwr_2616.trc | grep RTC
RTC enabled.
RTC enabled.
RTC enabled.

cat prodcdb_nss2_2708.trc | grep RTC
RTC: Enable compression following header-only I/O
RTC: Enable compression following header-only I/O
RTC: Enable compression following header-only I/O
No other values such as 512 (512: Tracks LGWR redo shipping network activity) or combination of values seems to give more descriptive information such as compression ratio or compressed bytes as in the case with previous versions. However, decompression information on standby could be traced with 1024.
DGMGRL> edit database stbycdb set property LogArchiveTrace='1024';
Property "logarchivetrace" updated
Decompression information available in *rfs*.trc file.
cat stbycdb_rfs_2243.trc
krsr_rfs_wda: RTC: decompressed 82 bytes to 512 bytes
krsr_rfs_wda: RTC: decompressed 81 bytes to 512 bytes
krsr_rfs_wda: RTC: decompressed 675 bytes to 1536 bytes
krsr_rfs_wda: RTC: decompressed 1167 bytes to 3072 bytes
krsr_rfs_wda: RTC: decompressed 76 bytes to 512 bytes
Useful Metalink Notes
How to confirm if Redo Transport Compression is used In Oracle Data Guard? [ID 1927057.1]
Redo Transport Compression in a Data Guard Environment [ID 729551.1]
LOG_ARCHIVE_TRACE=16 DOES NOT SHOW REDO COMPRESSION DETAIL IN LOG TRANSPORT in 11gr2 [ID 2193605.1]
How to find out the compression rate of Redo Transport Compression ? [ID 1490751.1]