Friday, November 27, 2009

11g Compression and Segment Size

An easy way to observer the compresson in action is to query the user_segments view.

select segment_name,sum(bytes)
from user_extents where segment_type='TABLE'
group by segment_name order by 2 desc;

SEGMENT_NAME SUM(BYTES)
------------------------------ ----------
AIRLINE 196608

Compress the table with
alter table airline move compress for all operations;

But the reduction is always not visible in the user_segments view as the segment is not shrunk beyond the initial storage parameter.

select segment_name,sum(bytes)
from user_extents where segment_type='TABLE'
group by segment_name order by 2 desc;

SEGMENT_NAME SUM(BYTES)
------------------------------ ----------
AIRLINE 196608

It could be observed that compression is being enabled

select table_name,compression,compress_for from user_tables;

TABLE_NAME COMPRESS COMPRESS_FOR
------------------------------ -------- ------------------
AIRLINE ENABLED FOR ALL OPERATIONS

Looking at the storage parameters

... STORAGE(INITIAL 196608 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL KEEP)

But the actual compression could be observed with the dbms_space package.

Before compression

full_blocks 3
full_bytes 24576

total_blocks 24
total_bytes 196608
unused_blocks 18
unused_bytes 147456
last_used_extent_file_id 6
last_used_extent_block_id 1544

After compression

full_blocks 2
full_bytes 16384

total_blocks 24
total_bytes 196608
unused_blocks 19
unused_bytes 155648
last_used_extent_file_id 6
last_used_extent_block_id 128

There's a reduction in the total number of blocks and increase in the number of unused blocks.

The PL/SQL code is listed below

DECLARE
unformatted_blocks NUMBER;
unformatted_bytes NUMBER;
fs1_blocks NUMBER;
fs1_bytes NUMBER;
fs2_blocks NUMBER;
fs2_bytes NUMBER;
fs3_blocks NUMBER;
fs3_bytes NUMBER;
fs4_blocks NUMBER;
fs4_bytes NUMBER;
full_blocks NUMBER;
full_bytes NUMBER;

total_blocks NUMBER;
total_bytes NUMBER;
unused_blocks NUMBER;
unused_bytes NUMBER;
last_used_extent_file_id NUMBER;
last_used_extent_block_id NUMBER;
last_used_block NUMBER;

-- cursor t is select table_name from user_tables order by 1;
-- tname user_tables.table_name%type;
BEGIN
--open t;
--loop
--fetch t into tname;
--exit when t%notfound;

DBMS_SPACE.SPACE_USAGE('TBX',
'AIRLINE',
'TABLE',
unformatted_blocks,
unformatted_bytes,
fs1_blocks,
fs1_bytes,
fs2_blocks,
fs2_bytes,
fs3_blocks,
fs3_bytes,
fs4_blocks,
fs4_bytes,
full_blocks,
full_bytes,
NULL);

-- dbms_output.put_line('table name '|| tname);
-- dbms_output.put_line('fs1_blocks ' || fs1_blocks );
-- dbms_output.put_line('fs1_bytes ' || fs1_bytes );
-- dbms_output.put_line('fs2_blocks '||fs2_blocks );
-- dbms_output.put_line('fs2_bytes '||fs2_bytes );
-- dbms_output.put_line('fs3_blocks '||fs3_blocks );
-- dbms_output.put_line('fs3_bytes '|| fs3_bytes );
-- dbms_output.put_line('fs4_blocks '|| fs4_blocks );
-- dbms_output.put_line('fs4_bytes '|| fs4_bytes );
dbms_output.put_line('full_blocks '|| full_blocks);
dbms_output.put_line('full_bytes '|| full_bytes );
dbms_output.put_line('');
--end loop;
--close t;

dbms_space.unused_space('TBX',
'AIRLINE',
'TABLE',
total_blocks ,
total_bytes ,
unused_blocks ,
unused_bytes ,
last_used_extent_file_id ,
last_used_extent_block_id ,
last_used_block ,
NULL);

dbms_output.put_line('total_blocks '|| total_blocks);
dbms_output.put_line('total_bytes '|| total_bytes );
dbms_output.put_line('unused_blocks '|| unused_blocks);
dbms_output.put_line('unused_bytes '|| unused_bytes );
dbms_output.put_line('last_used_extent_file_id '|| last_used_extent_file_id);
dbms_output.put_line('last_used_extent_block_id '|| last_used_extent_block_id );
dbms_output.put_line('');


end;
/

Wednesday, November 11, 2009

Updating Timezone

Updating 10.2.0.4 timezone to timezone 11. This is specially handy if the database is later upgraded to 11g Release 2.

Couple of metalink documents to go through 840118.1 and 412160.1 and few KEY points

1. Oracle 9, 10 and 11.1 can be upgraded to RDBMS DSTv11 using patch 8524113, DSTv11 is included in 11gR2 standard version.

2. download and run the utltzuv11.sql (url included in the metalink note 412160.1)

further steps are needed depending on the result of utltzuv11.sql script.

3. Patch 7695070 needs to be applied before the RDBMS DSTv11 patch can be installed.
Unlike previous DST patches the DSTv11 patch 8524113 needs to have 2 codefixes in place (Bug 7695070 and bug 7395472) to work. 10.2.0.5 and 11.2.0.2 (and higher) include the needed fixes, no need to apply patch 7695070 before installing the RDBMS DSTv11 patch on these versions.

once updated

WARNING: --> Database is using a timezone file older than version 11.
.... After the release migration, it is recommended that DBMS_DST package
.... be used to upgrade the 10.2.0.4.0 database timezone version
.... to the latest version which comes with the new release.


will not be prompted when upgrading to 11gR2.

Sunday, October 11, 2009

Memory Target on 11g

If MEMORY_TARGET is set to a non-zero value:

  1. If SGA_TARGET and PGA_AGGREGATE_TARGET are set, they will be considered the minimum values for the sizes of SGA and the PGA respectively. MEMORY_TARGET can take values from SGA_TARGET + PGA_AGGREGATE_TARGET to MEMORY_MAX_SIZE.
  2. If SGA_TARGET is set and PGA_AGGREGATE_TARGET is not set, still both parameters will be auto-tuned. PGA_AGGREGATE_TARGET will be initialized to a value of (MEMORY_TARGET-SGA_TARGET).
  3. If PGA_AGGREGATE_TARGET is set and SGA_TARGET is not set, still both parameters will be auto-tuned. SGA_TARGET will be initialized to a value of min(MEMORY_TARGET-PGA_AGGREGATE_TARGET, SGA_MAX_SIZE (if set by the user)) and will auto-tune subcomps.
  4. If neither is set, they will be auto-tuned without any minimum or default values. The policy is to give 60% for sga and 40% for PGA at startup.

If MEMORY_TARGET is not set or set to set to 0 explicitly (default value is 0 for 11g):

  1. If SGA_TARGET is set it will only auto-tune the sizes of the sub-components of the SGA. PGA will be autotuned independent of whether it is explicitly set or not. Though the whole SGA(SGA_TARGET) and the PGA(PGA_AGGREGATE_TARGET) will not be auto-tuned, i.e., will not grow or shrink automatically.
  2. If neither SGA_TARGET nor PGA_AGGREGATE_TARGET is set, it will follow the same policy as it have today; PGA will be auto-tuned and the SGA will not be auto-tuned and parameters for some of the sub-components will have to be set explicitly (for SGA_TARGET).
  3. If only MEMORY_MAX_TARGET is set, MEMORY_TARGET will default to 0 and it will not auto tune sga and pga. It will default to 10gR2 behavior within sga and pga.
  4. If sga_max_size is not user set, it will internally set it to MEMORY_MAX_TARGET.
  5. In a text initialization parameter file, if you omit the line for MEMORY_MAX_TARGET and include a value for MEMORY_TARGET, the database automatically sets MEMORY_MAX_TARGET to the value of MEMORY_TARGET. If you omit the line for MEMORY_TARGET and include a value for MEMORY_MAX_TARGET, the MEMORY_TARGET parameter defaults to zero. After startup, you can then dynamically change MEMORY_TARGET to a non-zero value, provided that it does not exceed the value of MEMORY_MAX_TARGET.

following views can be useful

* V$MEMORY_DYNAMIC_COMPONENTS has the current status of all memory components
* V$MEMORY_RESIZE_OPS has a circular history buffer of the last 800 SGA resize requests

metalink note 452512.1

Monday, October 5, 2009

Java Code Sinppet: JAX-WS & MTOM

To enable MTOM (Message Transmission Optimization Mechanism) there are several things need to be done and in a nutshell

1. add the MTOM annotation on the services class

@javax.xml.ws.soap.MTOM
@WebService(serviceName =


2. Following table list the relationship between MIME type and java type

MIME type Java type
image/gif java.awt.Image
image/jpeg java.awt.Image
text/plain java.lang.String
text/xml javax.xml.transform.Source
application/xml javax.xml.transform.Source
*/* javax.activation.DataHandler


3. Edit the WSDL to and list the correct mime type which will change the schema from

<xs:sequence>
<xs:element name="return" type="xs:base64Binary" minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>

to

<xs:sequence>
<xs:element name="return" type="xs:base64Binary" minOccurs="0" maxOccurs="unbounded"
xmime:expectedContentTypes="image/jpeg" xmlns:xmime="http://www.w3.org/2005/05/xmlmime"/>
</xs:sequence>


4. Make the container use the edited WSDL than generated one

@WebService(serviceName = ...., wsdlLocation="WEB-INF/wsdl/edited.wsdl")


5. Deploy the new service

for more info netbeans tutorial pages 1, page 2, page 3



To do the above without any modification to wsdl (using annotation on the java class)

1. Set the @MTOM as above on the service
@MTOM
@WebService
public class NewWebService {

2. Annotate the data type on the web method as follows
@WebMethod
public void setImage(@XmlMimeType("image/gif")Image img){

3. Can also annotate a return type
@WebMethod
public @XmlMimeType("image/gif")Image getImage( String name){

4. All kind of data types could be used
@WebMethod
public @XmlMimeType("*/*")Image getFiles

Sunday, October 4, 2009

Rolling Forward Image Copy Backups

On the first day create a incremental level 0 copy of the database

backup as copy incremental level 0 tag 'roll_update' database;

On the second day create a incremental level 1 of the database

backup incremental level 1 for reocvery of copy with tag 'roll_update' database;

On the thrid day use the second day incremental level 1 backup to roll forward the first day level 0 backup and then take a new incremental level 1 backup and repeat this step.

run{
recover copy of database with tag 'roll_update';
backup incremental level 1 for reocvery of copy with tag 'roll_update' database;
}


The whole thing could be simplified with


run{
recover copy of database with tag 'roll_update';
backup incremental level 1 for reocvery of copy with tag 'roll_update' database;
}


Points to note.

1. The BACKUP INCREMENTAL LEVEL 1... FOR RECOVER OF COPY WITH TAG... command does not actually always create a level 1 incremental backup. If there is no level 0 image copy backup of an particular datafile, then executing this command creates an image copy backup of the datafile on disk with the specified tag instead of creating the level 1 backup. Thus, the first time the script runs, it creates the image copy of the datafile needed to begin the cycle of incremental updates. In the second run and all subsequent runs, it produces level 1 incremental backups of the datafile.

2. The RECOVER COPY OF DATABASE WITH TAG... command causes RMAN to apply any available incremental level 1 backups to a set of datafile copies with the specified tag. If there is no incremental backup or no datafile copy, the command generates a message but does not generate an error. The first time the script runs, this command has no effect, because there is neither a datafile copy nor a level 1 incremental backup. The second time the script runs, there is a datafile copy (created by the first BACKUP command), but no incremental level 1 backup, so again, the command has no effect. On the third run and all subsequent runs, there is a datafile copy and a level 1 incremental from the previous run, so the level 1 incremental is applied to the datafile copy, bringing the datafile copy up to the checkpoint SCN of the level 1 incremental.

3. Each time a datafile is added to the database, an image copy of the new datafile is created the next time the script runs. The time after that, the first level 1 incremental for that datafile is created, and on all subsequent runs the new datafile is processed like any other datafile.

4. Tags must be used to identify the incremental level 0 datafile copies created for use in this strategy, so that they do not interfere with other backup strategies you implement. If you have multiple incremental backup strategies in effect, RMAN cannot unambiguously create incremental level 1 backups unless you tag level 0 backups. The incremental level 1 backups to apply to those image copies are selected based upon the checkpoint SCNs of the image copy datafiles and the available incremental level 1 backups. (The tag used on the image copy being recovered is not a factor in the selection of the incremental level backups.)

more on incremental backups

Monday, September 21, 2009

Relational Division

Relational division normally requires two not exists sub queries used in three different tables. Below sql snippet shows how division could be employed when all the data is in a single table.


PROJ SUBJ
----- -----
1 1
1 2
1 3
2 3
2 1
3 1
3 2
3 3
4 1
5 2
6 3


Hypothetical situation is that, there are several projects and each project uses number of different subjects. Idea is to find projects that are uses all the subjects.

The theory


theory implementation


select distinct proj from x where proj not in
(select distinct proj from (select * from
(select distinct proj from x) t1,
(select distinct subj from x) t2) t3
where (proj,subj) not in (select proj,subj from x));


Using With clause in Oracle


with t1 as (select DISTINCT proj from x),
t2 as (select distinct subj from x)
select distinct proj from x where proj not in
( select proj from t1,t2 where (proj,subj) not in (select * from x));


Using not exists sub queries

select * from (select distinct proj from x) projects
where not exists(
select * from (select distinct subj from x) subject
where not exists
(select * from x
where x.proj = projects.proj
and x.subj = subject.subj
));

Monday, September 14, 2009

Upgrading 10g to 11g R 2 with DBUA

1. Install Oracle 11gR2 software only (enterprise or standard to match the source db. This blog is on standard version. More on metalink note 870814.1).
2. In order to downgrade Oracle Enterprise Manager Database Control after upgrading to the new Oracle Database 11g release save Database Control files and data before upgrade. The emdwgrd utility resides in the ORACLE_HOME/bin directory in the new Oracle
Database 11g release.
3. Follow these steps to save your Database Control files and data:
1. Install the new Oracle Database 11g release.
This step is not required for an inplace patchset upgrade.
2. Set ORACLE_HOME to your old Oracle home.
This step is not required for an inplace patchset upgrade.
3. Set ORACLE_SID to the SID of the database being upgraded.
4. Set PATH, LD_LIBRARY_PATH, and SHLIB_PATH to point to the Oracle home
from which the database is being upgraded.
5. Go to the Oracle home of the new Oracle Database 11g release.
6. Execute one of the following:
if Single DB
emdwgrd -save -sid old_SID -path save_directory
for RAC
setenv EM_REMCP /usr/bin/scp
emdwgrd -save -cluster -sid old_SID -path save_directory
If the release 10g Oracle home is on a shared device, add -shared to the
previous command line.

create a TNS name entry with the same name as the SID before running the above command

[oracle@server110 std]$ emdwgrd -save -sid std1 -path /home/oracle/emdu
Enter sys password for database std1?

Mon Sep 14 19:14:24 2009 - Verify EM DB Control files ... pass
Mon Sep 14 19:14:24 2009 - Validating DB Connection to std1 ... pass
ENV var EM_REMCP not defined, check if rcp or scp is configured.
RCP = /usr/bin/rcp -rp, REMSH = /usr/bin/rsh
shared = 0
Mon Sep 14 19:14:28 2009 - Creating directory ... created
Mon Sep 14 19:14:29 2009 - Stopping DB Control ... stopped
Mon Sep 14 19:14:34 2009 - Saving DB Control files
... saved
Mon Sep 14 19:14:50 2009 - Recompiling invalid objects ... recompiled
Mon Sep 14 19:14:57 2009 - Exporting sysman schema for std1 ... exported
Mon Sep 14 19:15:56 2009 - DB Control was saved successfully.
Mon Sep 14 19:15:56 2009 - Starting DB Control ... started
Mon Sep 14 19:17:33 2009 - Dump directory was dropped successfully.


7. Copy the Pre-Upgrade Information Tool (utlu112i.sql) from the Oracle Database 11g Release 2 (11.2) ORACLE_HOME/rdbms/admin directory to a directory outside of the Oracle home.
8. run above script and examin the output

SQL> SPOOL upgrade_info.log
SQL> @utlu112i.sql
SQL> SPOOL OFF


9. Gather statistics on schemas instructed by utlu111i.sql
EXEC DBMS_STATS.GATHER_DICTIONARY_STATS;
EXEC DBMS_STATS.GATHER_SCHEMA_STATS(’user_name’);

10. Grant privileges revoked from public to resolve invalid objects
11. If the timezone warning is
Database is using a timezone file older than version 11.
then check the timezone with
select * from v$timezone_file;

if it is not 11 then to fix the problem use the DBMS_DST PL/SQL package and follow the instructions in "Steps to Upgrade Time Zone File and Timestamp with Time Zone Data" in Oracle Database Globalization Support Guide.

if the timezone warning is
Database is using a timezone file greater than version 11.
then before upgrading the database, patch the 11gR2 $ORACLE_HOME/oracore/zoneinfo/ with a timezone data file of the same version as the one used in the source release database.

12. Create a listener for the new 11g R2 Oracle home
13. Run DBUA from new 11g Home





















14. set the compatible parameter.
 alter system set compatible='11.2.0.0.0' scope=spfile ;


15. To upgrade the timezone
shutdown the database and start in upgrade mode
startup upgrade;
exec DBMS_DST.BEGIN_UPGRADE(11);

After PL/SQL executes start the database in normal mode and truncate timezone trigger tables

truncate table sys.dst$error_table;
truncate table sys.dst$trigger_table;

VAR numfail number
BEGIN
DBMS_DST.UPGRADE_DATABASE(:numfail,
parallel => TRUE,
log_errors => TRUE,
log_errors_table => 'SYS.DST$ERROR_TABLE',
log_triggers_table => 'SYS.DST$TRIGGER_TABLE',
error_on_overlap_time => TRUE,
error_on_nonexisting_time => TRUE);
DBMS_OUTPUT.PUT_LINE('Failures:'|| :numfail);
END;
/

after finishes

BEGIN
DBMS_DST.END_UPGRADE(:numfail);
END;
/