Template for AW

#1

How to create in 5 minutes a SQL Tuning Test Case using SQLTXPLAINCarlos SierraConsulting Technical Advisor

Copyright 2013, Oracle and/or its affiliates#2Carlos SierraJob/Hobby: SQL Tuning frequent practitionerExperience: Oracle since 1996Home: Orlando, FloridaBesides SQL Tuning: Spinning, Biking, Racquetball and Parenting!Author of: SQLTXPLAIN (SQLT), TRCANLZR (TRCA), SQLHC, etc.

Consulting Technical Advisor - Center of Expertise (CoE)

#3About SQLTXPLAIN (SQLT)Tool to diagnose SQL statements performing poorlyComprehensive set of diagnostics reportsAutomatic Test Case (TC) extractionNot a product but an add-on toolNeeds to be installedUsed by Oracle Support since 1999Works on 11g and 10.2 (UNIX, Linux and Windows)Prior versions for 10.1, 9i and 8.1 are available

Copyright 2013, Oracle and/or its affiliates

#Some SQLTXPLAIN (SQLT) characteristics and attributes :Self-contained in its own schema SQLTXPLAIN.Does not read or modify application data.Facilitates communication within Oracle and between Oracle and its customers.Provides in one place everything needed to analyze a SQL statement.Consistent view of the environment around one SQL statement.Strong and easy-to-use Test Case (TC) functionality.Performs many health-checks that may affect performance of a SQL being analyzed.About 100K lines of code in 14 PL/SQL libraries.Source code is not wrapped so it can be read and reverse engineered.Installation takes a few minutes and its footprint is small.Improves Oracle Support productivity and turnaround in SQL tuning issues.Works on 10.2-11.2 databases. Versions for 8.1-10.1 are available.Works on UNIX, LINUX and Windows platforms.Makes use of extended functionality if licenses exist for Tuning Pack or Diagnostics Pack.4TopicsA Test Case (TC) for SQL TuningHow SQLTXPLAIN (SQLT) helps with a TC?What is included in a SQLT TC?Demo of SQLT TC implementationBeyond a SQLT TCQ&ACopyright 2013, Oracle and/or its affiliates

#5A Test Case (TC) for SQL TuningFirst ObjectiveReproduce an Execution Plan from Source into Target systemWhat for?Apply WHAT-IF scenarios which are not feasible to test in Source systemInvestigate unexpected resultsTwo-phasesExtraction and implementationCopyright 2013, Oracle and/or its affiliates

#A Test Case for SQL Tuning requires two phases:Extract from a Source system everything needed to reproduce a known Execution Plan.Implement in a Target system everything needed to get that Plan.Once a Plan is reproduced into a test system, then a trained person can do diagnostics and apply what-if scenarios, for example newer releases, patches, different parameters, different statistics, etc.Sometimes a 10053 is not enough to understand WHY the CBO decided by a particular plan, for example when there are multiple predicates for a Table.

6What is needed to create a TC?A similar system in terms of database releaseBe aware of possible one-offsSQL textMetadata to create schema objectsTables, Indexes, Views Functions, Packages, ConstraintsCBO StatisticsSchema ObjectsCopyright 2013, Oracle and/or its affiliates

#At a very high level, a Test Case requires these 3:SQL Text.Tables and Indexes structure (or metadata to create them).CBO schema object statistics.Quite common other elements are also needed:Metadata for dependencies (views, functions, other tables, constraints, etc.)CBO System Statistics.CBO Environment (Parameters and Fix Control).Binds and their values.It is common practice to re-map schema owners into one TC user.7What else is needed to create a TC?Binds declaration and valuesCBO StatisticsSystem StatisticsCBO EnvironmentParametersFix ControlSchema owner(s) re-map Common but not required

Copyright 2013, Oracle and/or its affiliates

#At a very high level, a Test Case requires these 3:SQL Text.Tables and Indexes structure (or metadata to create them).CBO schema object statistics.Quite common other elements are also needed:Metadata for dependencies (views, functions, other tables, constraints, etc.)CBO System Statistics.CBO Environment (Parameters and Fix Control).Binds and their values.It is common practice to re-map schema owners into one TC user.8

Some Methods to extract a TCManualEasy if Target contains same schema objectsExport and Import of CBO schema object statisticsWhat about CBO environment and System stats?11g Test Case Builder (TCB)Overloaded DBMS_SQLDIAG.EXPORT_SQL_TESTCASEInputs SQL from Memory, Text or IncidentCopyright 2013, Oracle and/or its affiliates#If there are two systems where one (test) was cloned from the other (production) then manually migrating just the CBO schema object statistics would be the easiest approach.11g TCB is monolithic. When a TC is imported then it goes through entirely or rejected. SQLT TC allows to implement a TC step by step with great control.9

Some Methods to extract a TCSQLTXPLAIN (SQLT) Test Case (TC)Available for 11g, 10g and 9iInputs SQL from Memory, Text or AWRSome SQLT dependenciesNew internal SQLT Test Case Xpress (TCX)Available for 11g and 10g (since November 2012)No SQLT dependenciesAlmost as complete as SQLT TCCopyright 2013, Oracle and/or its affiliates#If there are two systems where one (test) was cloned from the other (production) then manually migrating just the CBO schema object statistics would be the easiest approach.11g TCB is monolithic. When a TC is imported then it goes through entirely or rejected. SQLT TC allows to implement a TC step by step with great control.10

How SQLTXPLAIN helps with a TC?

SQLT XTRACT, XECUTE and XTRXEC Automatic 11g TCB export on every 11g executionIf SQL is found in memoryAutomatic SQLT TC/TCX extraction on every 10g/11g execution If SQL is found in memory or AWRSQLT XTRSBY and XPLAINAutomatic SQLT TC/TCX extraction on every 10g/11g execution SQLT on 9i provides a rudimentary TCCopyright 2013, Oracle and/or its affiliates#11g TCB when executed by SQLT may take a long time to extract. You can verify this by looking at the SQLT log file. If that were the case in your system, and if you are not implementing the TC using 11g TCB, then you can disable this functionality from SQLT by using the following command, which only affects SQLT:EXEC sqltxadmin.sqlt$a.set_param('test_case_builder', 'N');When implementing a SQLT TC try using first the XPRESS mode. CUSTOM mode is available in cases where the SQLT TC XPRESS implementation gets interrupted and you want to restart it from an intermediate step.11

SQLT TC/TCX implementation

SQLT TC provides two implementation modesCUSTOMSeven steps for better control during TC implementationXPRESSOne driver script for a fast TC implementationSQLT TCX provides one implementation modeINSTALLOne driver script for a fast TCX implementation

Copyright 2013, Oracle and/or its affiliates#11g TCB when executed by SQLT may take a long time to extract. You can verify this by looking at the SQLT log file. If that were the case in your system, and if you are not implementing the TC using 11g TCB, then you can disable this functionality from SQLT by using the following command, which only affects SQLT:EXEC sqltxadmin.sqlt$a.set_param('test_case_builder', 'N');When implementing a SQLT TC try using first the XPRESS mode. CUSTOM mode is available in cases where the SQLT TC XPRESS implementation gets interrupted and you want to restart it from an intermediate step.12Where do I find the SQL TC?Inside sqlt_sNNNNN.zipReview sqlt_sNNNNN_main.htmlSQLT TC implementation requiresReadme file sqlt_sNNNNN_readme.htmlSQLT TC zip file sqlt_sNNNNN_tc.zipLook for Implement SQLT Test Case (TC) on readme fileCopyright 2013, Oracle and/or its affiliates

#SQLT TC is included inside the main zip file that SQLT produces every time one of the main methods is used. A SQLT TC is identified by *_tc.zip. Be aware that 11g TCB is identified by *_tcb.zip.Before implementing a SQLT TC, be sure you are familiar with the SQL and its environment. You can use the main html report for this effect. Then use the readme html or text file to review Preparation and Implementation steps.13Files inside sqlt_sNNNNN_tc.zipCopyright 2013, Oracle and/or its affiliates

XECUTEXTRACTXPLAINreadme

sqlt tc

#XECUTE, XTRACT, XTRXEC and XPLAIN all extract a SQLT TC from the Source system.The SQLT TC is made of two files:A dynamic readme file with SQLT TC implementation instructions.A TC zip file with several files needed by the SQLT TC implementation. SQLT TC allows a failed TC implementation to be manually fixed and restarted from intermediate steps. This makes the SQLT TC very popular among Oracle Support engineers.A SQLT TC generated on 10g can be implemented on 10g or 11g.

14

What is included in a SQLT TC?TC zip contains several filesMost used by TC implementationSome miscellaneous utilitiesA readme file in text formatCopyright 2013, Oracle and/or its affiliates

#When you unzip the tc.zip file you will discover several files. Some are used during the SQLT TC implementation and some are used later if needed. You will also find a text-format readme which contains the same information than the html-format readme inside the main zip. The reason to have this text-format readme is to make it easy to access it from the server without having to use a PC to read the html-format.

15What is included in a SQLT TC?Metadata scriptCreate schema objectsPurge scriptAvoid collisions on statement_idExport DMP fileSQLT repository from SourceRestore scriptMap and import CBO schema stats

Copyright 2013, Oracle and/or its affiliates

#The metadata script could be customized if needed. All others require no changes. The names of these scripts loosely match what they do. Also, the CUSTOM mode shows clearly the step name and the script that is used.16What is included in a SQLT TC?System Statistics scriptSet system stats in TargetSet CBO environment scriptSet CBO ParametersSet Fix ControlTest Case scriptExecute QueryProduce Plan

Copyright 2013, Oracle and/or its affiliates

#The metadata script could be customized if needed. All others require no changes. The names of these scripts loosely match what they do. Also, the CUSTOM mode shows clearly the step name and the script that is used.17What is included in a SQLT TC?XPRESS scriptsXPRESS.shXPRESS.sqlMiscellaneous utilitiesReadme fileCopyright 2013, Oracle and/or its affiliates

#The metadata script could be customized if needed. All others require no changes. The names of these scripts loosely match what they do. Also, the CUSTOM mode shows clearly the step name and the script that is used.18About Metadata scriptCreates TC user TCNNNNNSchema owner re-map into TCNNNNNSuffix to branch a TCTC12345CS, TC12345ADefaults (recommended)Re-map: yesSuffix: noCopyright 2013, Oracle and/or its affiliates

#This is the most important script on the SQLT TC. It contains the create commands for schema objects your SQL statement depends on.In most cases we simply allow the script to re-map schema owner(s) from Source system into just one TC user in our test Target system. The name of this TC user becomes then TCNNNNN, where NNNNN matches the statement_id from sqlt_sNNNNN.zip.Also, in most cases you do not need to provide a TC suffix. This is only needed when you need or want to identify your TC or need to branch it. So from one Source case you can generate two or more TC in Target.When you execute the metadata script it shows at the end some invalid objects. If this list is empty that means there were no invalid objects. In some cases you can ignore invalid synonyms, but in most cases you cannot ignore invalid Tables, Indexes, Views or PL/SQL Libraries. You can use then the metadata log file to find out WHY you got that error. You can fix it then and re-execute the metadata script a second time.19About Metadata scriptAlways review Invalid section at end of executionInvalid synonymsMostly safe to ignoreInvalid librariesSometimes require attentionFix codeInvalid viewsAlways require attentionEliminate references to schema names on FROMCopyright 2013, Oracle and/or its affiliates

#This is the most important script on the SQLT TC. It contains the create commands for schema objects your SQL statement depends on.In most cases we simply allow the script to re-map schema owner(s) from Source system into just one TC user in our test Target system. The name of this TC user becomes then TCNNNNN, where NNNNN matches the statement_id from sqlt_sNNNNN.zip.Also, in most cases you do not need to provide a TC suffix. This is only needed when you need or want to identify your TC or need to branch it. So from one Source case you can generate two or more TC in Target.When you execute the metadata script it shows at the end some invalid objects. If this list is empty that means there were no invalid objects. In some cases you can ignore invalid synonyms, but in most cases you cannot ignore invalid Tables, Indexes, Views or PL/SQL Libraries. You can use then the metadata log file to find out WHY you got that error. You can fix it then and re-execute the metadata script a second time.20Metadata Customization SectionCopyright 2013, Oracle and/or its affiliates

#TC suffix can be specified in advance or avoided altogether. By default SQLT TC implementation will ask for TC suffix. If you dont need one then just hit enter and none will be used. A TC suffix must be in upper-case and contain only letters and numbers.By default, all schema objects referenced by the SQL statement analyzed by SQLT, are re-mapped into a TC user. If you want to preserve original schema names then remove from DEFINE commands at the bottom of the customization section. Be aware that you will then have to import the CBO statistics using sqlt/utl/sqltimp.sql and not provide a schema owner so the original schema owner name(s) get used.21About Purge scriptSQLT repository for statement_id (NNNNN) is about to be importedPurge script deletes prior repository entries for statement_id NNNNNAvoids possible but very infrequent collisions on NNNNNCopyright 2013, Oracle and/or its affiliates

#This purge script removes from the local (Target) SQLT repository all entries pertinent to statement_id NNNNN. In most cases your Target system will not contain such NNNNN since each system starts with a random number. This script is a safeguard against possible collisions on NNNNN. You can also use this script to wipe out all references to NNNNN in the local SQLT repository in cases where you want to restart your TC from scratch. In such case you may also need to manually drop the TC user with cascade. 22About Purge scriptPurge script becomes important when TC is refreshedIn such case manually drop user TCNNNNN before handDROP USER TCNNNNN CASCADE;Purge script only executes a SQLT libraryEXEC sqltxadmin.sqlt$a.purge_repository(93896, 93896);Copyright 2013, Oracle and/or its affiliates

#This purge script removes from the local (Target) SQLT repository all entries pertinent to statement_id NNNNN. In most cases your Target system will not contain such NNNNN since each system starts with a random number. This script is a safeguard against possible collisions on NNNNN. You can also use this script to wipe out all references to NNNNN in the local SQLT repository in cases where you want to restart your TC from scratch. In such case you may also need to manually drop the TC user with cascade. 23About Exported DMP fileDMP file is created on Source when SQLT is executedXTRACTXECUTEXTRXECXTRSBYXPLAINContains SQLT repository for statement_id NNNNNNo application data is exported

Copyright 2013, Oracle and/or its affiliates

#In some cases the dmp file is missing from the sqlt_sNNNNN_tc.zip file. That is possible if the user of SQLT main methods does not provide a valid SQLTXPLAIN password when XTRACT, XECUTE, XTRXEC or XPLAIN requested it. If that were the case refer to the readme for instructions in how to export it after the fact. Look for Export SQLT repository.Be aware that a dmp file must be treated as binary. If you FTP it as text then it becomes corrupted.24About Exported DMP fileImport gets SQLT repository for NNNNN including a STATTAB staging table imp sqltxplain FILE=sqlt_s93896_exp.dmp TABLES=sqlt% IGNORE=YDefault utility is original export/importSQLT can be configured to use Data Pump instead

Copyright 2013, Oracle and/or its affiliates

#In some cases the dmp file is missing from the sqlt_sNNNNN_tc.zip file. That is possible if the user of SQLT main methods does not provide a valid SQLTXPLAIN password when XTRACT, XECUTE, XTRXEC or XPLAIN requested it. If that were the case refer to the readme for instructions in how to export it after the fact. Look for Export SQLT repository.Be aware that a dmp file must be treated as binary. If you FTP it as text then it becomes corrupted.25About Restore scriptRestore (import) schema object CBO statistics for all tables accessed by SQLRe-maps CBO statistics to TCNNNNN userRe-maps extended statistics columnsExpression or column groupUses DBMS_STATS.IMPORT_TABLE_STATSIf re-map of owner is not desired use instead sqlt/utl/sqltimp.sqlCopyright 2013, Oracle and/or its affiliates

#One of the tables making the SQLT repository is STATTAB which contains the schema object statistics. This restore script calls a SQLT library that re-maps all entries on STATTAB for a given NNNNN statement_id so they match the new TC user. Then it uses standard DBMS_STATS import libraries to restore into the Target data dictionary the schema statistics for the TC objects.This restore script also re-maps columns that are hidden, like the ones making the extended statistics. This is needed since the column name is system generated and Source and Target may have different names for the same extension.If you are not using the default TC user, but the original schema names from Source, then skip this restore script and use sqltimp.sql instead.This restore script indicates how many CBO statistics rows were successfully imported into the Target data dictionary.26

About System Statistics scriptDeletes CBO System Statistics on TargetRestores (set) using values from SourceCalls DBMS_STATS.DELETE_SYSTEM_STATSFollowed by DBMS_STATS.SET_SYSTEM_STATSCopyright 2013, Oracle and/or its affiliates#System Statistics have usually a minor impact in an Execution Plan, unless of course the values of Source and Target are very different. This system statistics script deletes the local statistics first, then restore the values captured from the Source system.If working on a TC in a Target server used by other users, you may want to restore system statistics to the original values, or just delete them, once you finish with your TC.27

About Set CBO Environment scriptCaptures CBO Environment from Source SQLT sessionExecuted on Target after connecting as TC userSets Optimizer Features Enable (OFE)Sets CBO ParametersUsing ALTER SESSION and ALTER SYSTEM commandsSets Bug Fix ControlCopyright 2013, Oracle and/or its affiliates#After connecting as the TC user, this script sets the CBO environment as it was captured from the session in Source where SQLT was executed. This script includes both CBO Parameters and Bug Fix Control.It first sets the OFE as per Source, the proceeds with non-default values (Parameters and Fix Control) followed by default values.The reason to do default values is because the Target system may have some non-default values itself, so we have to synchronize it with Source by changing those to their default value.If you Target system always uses default values, you can eliminate from this script all ALTER SESSION/SYSTEM for the default values sections.28

About TC script (tc.sql)Executes q.sql followed by plan.sqlScript q.sql contains SQL being analyzedCBO environment for SQL when captured from memory Bind variables (if applicable)Declaration and values assignmentSQL textScript plan.sql displays execution plan Calls DBMS_XPLAN.DISPLAY_CURSORCopyright 2013, Oracle and/or its affiliates#The tc.sql script execute two other scripts:The q.sql script with the SQL statement.The plan.sql script to display the Execution Plan for q.sql.If the SQL statement was captured from memory, then q.sql contains ALTER commands from the SQL CBO environment.Script q.sql also contains the commands to define and assign values to bind variables. These values try to produce the most expensive Execution Plan in terms of average elapsed time. This is not always possible.29About XPRESS scripts (sh and sql)Shell script xpress.sh executes SQL script xpress.sql connected as SYSDBA

Script xpress.sql executes all same 7 steps from CUSTOM modeTC implementation can be restarted from any intermediate stepCopyright 2013, Oracle and/or its affiliates

#XPRESS mode simply executes the 7 steps from the CUSTOM mode one after another. You can always start with XPRESS mode, and if you need to restart from an intermediate step move to the CUSTOM mode.30

About Miscellaneous Utilities and FilesSQL statement script q.sqlDisplay plan script plan.sqlFlush shared pool script flush.sqlEVENT 10053 script 10053.sqlSelectivity script sel.sql Auxiliary selectivity script sel_aux.sqlDelete TC Histograms sqlt_sNNNNN_del_hgrm.sqlText readme sqlt_sNNNNN_readme.txtSetup script setup.sql for stand-alone TCCopyright 2013, Oracle and/or its affiliates

#Scripts q.sql and plan.sql are used by the TC implementation when step 7 tc.sql is executed.Scripts 10053.sql, flush.sql and sel.sql can be used during the TC exploration (diagnostics). Script sel_aux.sql is called by sel.sql. The latter is used to compute single-table selectivity of individual predicates, when validating the CBO estimated cardinality.The delete histograms script is provided to delete all histograms out of the TC but preserving all other column statistics.The setup.sql is only used if you need to create a stand-alone TC out of this SQLT TC, removing all dependencies on SQLT. This is useful when a TC is requested by Oracle Development.31SQLT TC implementation stepsRequirementsA sqlt_sNNNNN.zip file from Source systemA Target system with same database version (or higher)Be aware of possible one-offsSQLTXPLAIN installed on Target systemCopyright 2013, Oracle and/or its affiliates

#The starting point is a sqlt_sNNNNN.zip file from any SQLT method from a Source system. Next step is to identify a test system with same or similar database release.SQLTXPLAIN must be installed on that test system (Target). Always try using in Target the latest version of SQLTXPLAIN.You need to copy into Target the sqlt_sNNNNN_tc.zip which is part of sqlt_sNNNNN.zip.Unzip in Target that tc.zip file. You may want to see the readme file for exact unzip command which creates also a TC directory.Navigate to that TC directory and proceed with XPRESS mode.32

SQLT TC implementation stepsPreparationUnzip sqlt_sNNNNN.zipReview main diagnostics reportFind Implement SQLT Test Case (TC) section in readmeCopy sqlt_sNNNNN_tc.zip file to Target serverFollow Preparation then XPRESS or CUSTOM modeCopyright 2013, Oracle and/or its affiliates#The starting point is a sqlt_sNNNNN.zip file from any SQLT method from a Source system. Next step is to identify a test system with same or similar database release.SQLTXPLAIN must be installed on that test system (Target). Always try using in Target the latest version of SQLTXPLAIN.You need to copy into Target the sqlt_sNNNNN_tc.zip which is part of sqlt_sNNNNN.zip.Unzip in Target that tc.zip file. You may want to see the readme file for exact unzip command which creates also a TC directory.Navigate to that TC directory and proceed with XPRESS mode.33Typical menu from a SQLT readmeCopyright 2013, Oracle and/or its affiliates

#The menu of the SQLT readme is dynamic. Depending on the database release and the objects around your SQL, you may find instructions to handle SQL Profiles, SQL Sets, SQL Plan Baselines, etc.For SQLT TC implementation refer to pertinent entry on the menu.34SQLT TC PreparationCopyright 2013, Oracle and/or its affiliates

#SQLT TC Preparation is quite easy. Just copy and unzip the tc.zip file into your test server (Target), then navigate to new TC directory.35CUSTOM implementation modeCopyright 2013, Oracle and/or its affiliates

#SLQT TC CUSTOM implementation requires 7 steps. You can stop at any time and continue later. For example, if the metadata script results in some invalid objects, you can fix them and proceed with next step.If you open the xpress.sql script you will discover it contains the same 7 steps. Both XPRESS and CUSTOM modes are synchronized. 36CUSTOM implementation modeCopyright 2013, Oracle and/or its affiliates

#You may notice that XPRESS and CUSTOM modes both have the same sequence of steps, so you can move out of XPRESS and continue with CUSTOM at any point during the SQLT TC implementation.37XPRESS implementation modeCopyright 2013, Oracle and/or its affiliates

#SQLT TC XPRESS implementation is done with one of two options. Try using option 1 from your OS. If it does not work, then use option 2.If you look inside xpress.sh you will notice that all it does is the execution of option 2.38

XPRESS script xpress.sqlCopyright 2013, Oracle and/or its affiliates

#Using XPRESS mode allows to see what is being done on each step. There is a pause between steps (scripts) so just hit enter to continue. Pay attention on the outcome of each step.39DemonstrationImplementing a SQLT TC using Xpress method

#Beyond a SQLT TCA TC enables a SQL Tuning environment on TestAfter reproducing Execution PlanApply any WHAT-IFIf a regression try pre-upgrade OFEIf OFE produces good plan consider SQLT XPLOREIf suspecting statistics consider SQLT XHUME Only on a disposable Test systemIf suspecting histograms consider SQLT XGRAMCopyright 2013, Oracle and/or its affiliates

#Once the main goal of the SQLT TC implementation has been reached, which was to reproduce on Target the same Execution Plan seen on Source, then the actual SQL Tuning exercise begins. Now you can start the debugging process using your TC. Common tests include using a prior OFE and verify the effects, or testing with different values of binds, different parameters, prior statistics, different statistics, etc.SQLT includes some utilities that can be used to explore some scenarios in your test case. SQLT XPLORE will toggle CBO Parameters and Fix Control, identifying how the Plan changes. SQLT XHUME restores statistics to multiple points in time, but this method can only be used on a disposable test system since it affects the data dictionary directly by resetting the TC objects creation date, leaving your test system on an unsupported state.SQLT XGRAM allows you to play with histograms. This is important since the TC does not include application data so we cannot just gather statistics in different ways.

41Removing SQLT dependencies from Test CaseTo isolate TC from any SQLTXPLAIN dependencies refer to Create TC with no SQLT dependencies on readmeCopyright 2013, Oracle and/or its affiliates

#Once your SQLT TC reproduces the issue of concern at will, you may need to create a new TC out of it. This new TC would not have dependencies on SQLT. To do this follow instructions on readme.42Create TC with no SQLT dependenciesCopyright 2013, Oracle and/or its affiliates

#If using these instructions, always test your stand-alone TC on another system, and be sure that by following your own readme.txt you can reproduce the desired Plan on your stand-alone TC.43ReferencesMOS 215187.1 SQLTXPLAIN (SQLT)

carlos.sierra@oracle.comhttp://carlos-sierra.net

Copyright 2013, Oracle and/or its affiliates

##45