Thursday, October 15, 2020

UCP Error Even When Not Using Multi-Tenant Shared Pool Feature

Following errors were seen on CDB alert log where JDBC connections were made to a service associated with a PDB.
2020-10-15T12:18:50.823318+00:00
TESTPDB(3):WARNING: too many parse errors, count=100 SQL hash=0x6870fc2d
TESTPDB(3):PARSE ERROR: ospid=12456, error=904 for statement:
2020-10-15T12:18:50.823502+00:00
TESTPDB(3):select dbms_service_prvt.get_topology(:"SYS_B_0") from dual
TESTPDB(3):Additional information: hd=0x6fc6cb80 phd=0x7025df10 flg=0x100476 cisid=80 sid=80 ciuid=80 uid=80 sqlid=bwkv6d1n71z1d
TESTPDB(3):...Current username=ASANGA
TESTPDB(3):...Application: JDBC Thin Client Action:
As per 2489973.1 the reason for this issue is UCP making calls that are associated with Multi-Tenant Shared Pool feature even when the feature is not used.



One way to remedy is to grant the parsing user the execute privilege on dbms_service_prvt. But this means the un-necessary calls would still happen but error won't be raised.
However, there's patch to fix this on 12.2 28643583.
If patch 28643583 is already applied on UCP then use the merge patch 32004255.

Wednesday, September 30, 2020

JDBC Thin Connections to Autonomous Transaction Processing DB

There are two methods for connecting to ATP using a JDBC thin client. One is using Oracle wallet and the other is using java keystore. This post gives a summary of setting up JDBC thin connections using these methods. It is assume that the client credentials zip file is dowloaded from the ATP DB console.

Using Java KeyStore for JDBC Thin Connections

Using java keystore (JKS) is the simplest method for connecting a JDBC thin client to ATP. It requires no new libraries to be added to the classpath and if existing class conforms to using TNS entries then require no code changes either.
1. To use this method to connect to ATP DB following files are needed which are included in the client credentials zip file downloaded from the ATP DB console. 

  •  truststore.jks
  • keystore.jks
  • ojdbc.properties
  • tnsnames.ora
2. Modify the ojdbc.properties file to include JKS related entries (The original file in the client credentail zip will have additonal entries). This include specifying the locations of the two JKS files and their passwords. Password is the same password given when client credentials zip was downloaded from ATP DB console. Below is an example of ojdbc.properties file used in this method.
javax.net.ssl.trustStore=C:\\Asanga\\java\\atpjdbc\\Wallet_ATPFree\\truststore.jks
javax.net.ssl.trustStorePassword=<wallet password here>
javax.net.ssl.keyStore=C:\\Asanga\\java\\atpjdbc\\Wallet_ATPFree\\keystore.jks
javax.net.ssl.keyStorePassword=<wallet password here>
3. Next the location of the tnsnames.ora and ojdbc.properties files must be specified to the java app. This could be done by specifying the oracle.net.tns_admin system property, which could be passed on through the JVM options.
-Doracle.net.tns_admin=./Wallet_ATPFree
This would require no code changes. Other methods of specifying this location includes using the TNS_ADMIN in the JDBC URL (requies driver 18.3 or above) or using connection property OracleConnection.CONNECTION_PROPERTY_TNS_ADMIN. In this example location "./Wallet_ATPFree" containes tnsnames.ora and ojdbc.properties files.

4. With these files and enties in place create a JDBC thin connection using UCP as below. In this case the tpurgent service is used in the URL.
        PoolDataSource ds = PoolDataSourceFactory.getPoolDataSource();
        ds.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
        ds.setConnectionPoolName("ATP_Pool");
        ds.setURL("jdbc:oracle:thin:@atpfree_tpurgent");
        ds.setUser("asanga");
        ds.setPassword("pwd_here");




Using Oracle Wallet for JDBC Thin Connections


1. Using this method requires following files and library jars. 

  •  ewallet.p12 and cwallet.sso
  • ojdbc.properties
  • tnsnames.ora
  • oraclepki.jar, osdt_cert.jar and osdt_core.jar (not included in client credential zip file)

2. The wallet file is doesn't contain any passwords.
mkstore -wrl . -listCredential
Oracle Secret Store Tool Release 21.0.0.0.0 - Production
Version 21.0.0.0.0
Copyright (c) 2004, 2020, Oracle and/or its affiliates. All rights reserved.

Enter wallet password:
List credential (index: connect_string username)
However, it is possible to add a password to the wallet and use it for passwordless login. What is contains is the certificate which allows to make TCPS connections. These certificates could be listed with following (redacted output shown).
orapki wallet display -wallet . -complete
Oracle PKI Tool Release 21.0.0.0.0 - Production
Version 21.0.0.0.0
Copyright (c) 2004, 2020, Oracle and/or its affiliates. All rights reserved.

Requested Certificates:
User Certificates:
Subject:        CN=............................,DNQ=V1
Issuer:         C=US,ST=California,L=Redwood Shores,O=Oracle Corporation Autonomous Data Warehouse Cloud Self-signed CA,CN=Autonomous Data Warehouse Cloud CA
Serial Number:  00
Key Length      2048
MD5 digest:     ....
SHA digest:     ....

Trusted Certificates:
Subject:        CN=DigiCert Global Root CA,OU=www.digicert.com,O=DigiCert Inc,C=US
Issuer:         CN=DigiCert Global Root CA,OU=www.digicert.com,O=DigiCert Inc,C=US
Serial Number:  ....
Key Length      2048
MD5 digest:     ...
SHA digest:     ...

Subject:        C=US,ST=California,L=Redwood Shores,O=Oracle Corporation Autonomous Data Warehouse Cloud Self-signed CA,CN=Autonomous Data Warehouse Cloud CA
Issuer:         C=US,ST=California,L=Redwood Shores,O=Oracle Corporation Autonomous Data Warehouse Cloud Self-signed CA,CN=Autonomous Data Warehouse Cloud CA
Serial Number:  ...
Key Length      2048
MD5 digest:     ...
SHA digest:     ...

Subject:        CN=DigiCert SHA2 Secure Server CA,O=DigiCert Inc,C=US
Issuer:         CN=DigiCert Global Root CA,OU=www.digicert.com,O=DigiCert Inc,C=US
Serial Number:  ...
Key Length      2048
MD5 digest:     ...
SHA digest:     ...
As by default no password contains in the wallet, it must be specified in the JDBC connection. Secondly the auto login wallte included in the downloaded wallet zip file is not a auto login local wallet. For added security a new auto login local wallet could be created. Password of the wallet is the same password given when client credentials zip was downloaded from ATP DB console

3. Modify the ojdbc.properties file to contain the following entry which specify the wallet file location.
oracle.net.wallet_location=(SOURCE=(METHOD=FILE)(METHOD_DATA=(DIRECTORY=C:\\Asanga\\java\\atpjdbc\\Wallet_ATPFree))))
4. Similar to JKS method, specify the location of the tnsnames.ora and ojdbc.properties files. This could be done by specifying the oracle.net.tns_admin system property, which could be passed on through the JVM options.
-Doracle.net.tns_admin=./Wallet_ATPFree
This would require no code changes. Other methods of specifying this location includes using the TNS_ADMIN in the JDBC URL (requies driver 18.3 or above) or using connection property OracleConnection.CONNECTION_PROPERTY_TNS_ADMIN. In this example location "./Wallet_ATPFree" containes tnsnames.ora and ojdbc.properties files.

5. Include the oraclepki.jar, osdt_cert.jar and osdt_core.jar files in the classpath of the java application.

6. With these files and entries in place same JDBC Connection code as shown above in JKS method could be used to make JDBC thin connections to the ATP DB. Example below shows password explicity being specified in the java code rathe than rely on password stoed in wallet.
        PoolDataSource ds = PoolDataSourceFactory.getPoolDataSource();
        ds.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
        ds.setConnectionPoolName("ATP_Pool");
        ds.setURL("jdbc:oracle:thin:@atpfree_tpurgent");
        ds.setUser("asanga");
        ds.setPassword("pwd_here");

Saturday, August 8, 2020

Removing a Failed Standby Database From a Data Guard Configuration

A previous post explained steps for removing a standby instance from a data guard configuration. This post explains steps for the same but when the standby being removed has failed and cannot be reached (or connect into).
In a standby configuration with multiple standby databases once instance is unreachable due to hardware failure. The issue is irrecoverable and only option is to rebuild the node and the standby instance. In mean time the existing standby configuration will give an error state due to the unavailability of the failed instance.
DGMGRL> show configuration

Configuration - fc_pp_dg

  Protection Mode: MaxAvailability
  Members:
  ppdb1  - Primary database
    ppdb2  - Physical standby database
    ppdb3  - Physical standby database
      ppdb4  - Physical standby database (receiving current redo)
    ppfs1  - Far sync instance
      ppdb5  - Physical standby database
      ppdb6  - Physical standby database
      ppdb9  - Physical standby database
      ppdb10 - Physical standby database

  Members Not Receiving Redo:
  ppfs2  - Far sync instance (alternate of ppfs1)
  ppdb8  - Physical standby database
    Error: ORA-12170: TNS:Connect timeout occurred

Fast-Start Failover:  Disabled

Configuration Status:
ERROR   (status updated 96 seconds ago)
As the first step remove any references to the failed instance on RedoRoutes.

Then issue the remove command which will succeed with a warning.
DGMGRL> remove database ppdb8;
Warning: ORA-16620: one or more members could not be reached for a remove operation

Removed database "ppdb8" from the configuration
The warning is due to broker being unable to connect to the failed instance to execute the clean up commands. The dataguard broke log shows this.
2020-07-29T12:33:36.403+00:00
Failed to connect to remote database ppdb8. Error is ORA-12170
Metadata Resync failed. Status = ORA-12170
2020-07-29T12:33:48.691+00:00
Failed to connect to remote database ppdb8. Error is ORA-12170
Failed to send message to member ppdb8. Error code is ORA-12170.
Data Guard Broker Status Summary:
  Type                        Name                             Severity  Status
  Configuration               fc_pp_dg                       Warning  ORA-16607: one or more members have failed
  Primary Database            ppdb1                          Success  ORA-0: normal, successful completion
  Physical Standby Database   ppdb2                          Success  ORA-0: normal, successful completion
  Physical Standby Database   ppdb3                          Success  ORA-0: normal, successful completion
  Physical Standby Database   ppdb4                          Success  ORA-0: normal, successful completion
  Physical Standby Database   ppdb5                          Success  ORA-0: normal, successful completion
  Physical Standby Database   ppdb6                          Success  ORA-0: normal, successful completion
  Far Sync Instance           ppfs1                          Success  ORA-0: normal, successful completion
  Far Sync Instance           ppfs2                          Success  ORA-0: normal, successful completion
  Physical Standby Database   ppdb8                            Error  ORA-12170: TNS:Connect timeout occurred
  Physical Standby Database   ppdb9                          Success  ORA-0: normal, successful completion
  Physical Standby Database   ppdb10                         Success  ORA-0: normal, successful completion
2020-07-29T12:34:00.979+00:00
Failed to connect to remote database ppdb8. Error is ORA-12170
Failed to send message to member ppdb8. Error code is ORA-12170.
2020-07-29T12:34:05.646+00:00
REMOVE DATABASE ppdb8
2020-07-29T12:34:17.939+00:00
Failed to connect to remote database ppdb8. Error is ORA-12170
Failed to send message to member ppdb8. Error code is ORA-12170.
Database ppdb8 (0x0a001000) could not be contacted for database removal, status = ORA-12170
2020-07-29T12:34:31.571+00:00
Failed to connect to remote database ppdb8. Error is ORA-12170
Failed to send message to member ppdb8. Error code is ORA-12170.
2020-07-29T12:34:33.297+00:00
Database ppdb8 removal completed with warning ORA-16620
REMOVE DATABASE  completed with warning ORA-16620
However, all the other databases that are part of the dataguard configuration would have had their log_archive_config parameter updated by removing any reference to the failed database.
NAME                           VALUE
------------------------------ -----------------------------------------
log_archive_config             dg_config=(ppdb1,ppdb2,ppdb3,ppdb4,ppfs1,
                               ppdb5,ppdb6,ppdb9,ppdb10,ppfs2)


Once the failed instance is removed the dataguard broke shows status success.
DGMGRL>  show configuration

Configuration - fc_pp_dg

  Protection Mode: MaxAvailability
  Members:
  ppdb1  - Primary database
    ppdb2  - Physical standby database
    ppdb3  - Physical standby database
      ppdb4  - Physical standby database (receiving current redo)
    ppfs1  - Far sync instance
      ppdb5  - Physical standby database
      ppdb6  - Physical standby database
      ppdb9  - Physical standby database
      ppdb10 - Physical standby database

  Members Not Receiving Redo:
  ppfs2  - Far sync instance (alternate of ppfs1)

Fast-Start Failover:  Disabled

Configuration Status:
SUCCESS   (status updated 55 seconds ago)
Related Posts
Removing a Standby Database From a Data Guard Configuration
Adding a New Physical Standby to Exiting Data Guard Setup