high performance persistence - software...

91
2005 JavaOne SM Conference | Session XXXX Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation Donald Smith — High Performance Persistence Page High Performance Persistence Donald Smith Oracle Corporation

Upload: others

Post on 17-Jun-2020

6 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

2005 JavaOneSM Conference | Session XXXX

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

High PerformancePersistence

Donald Smith

Oracle Corporation

Page 2: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Show how you build high performance

J2EE applications that scale using an

object-relational persistence layer

Goal

Page 3: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

AgendaPersistence Architecture Overview

Performance Points:Models and Mapping

Inheritance

Locking

Querying

Lazy Reading

N+1 Reads

Projections

Transactions

Caching

Leveraging the Database and JDBC

Page 4: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

AgendaPersistence Architecture Overview

Performance Points:Models and Mapping

Inheritance

Locking

Querying

Lazy Reading

N+1 Reads

Projections

Transactions

Caching

Leveraging the Database and JDBC

Page 5: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Java Access of Relational Data

Direct JDBC

Direct SQL calls hard coded in Java

Use result sets (“rows”) directly

Persistence layer

Accessed as objects or components

Transparent that the data is stored in RDB

Persistence layer in middle tier handlesobject-relational mapping and infrastructure

Required if doing business logic in the middle tier!

Page 6: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

JDBC—Java DatabaseConnectivity

Java standard foraccessing databases

JDBC is simply thedatabase connectionutilities Javadevelopers need tobuild upon

SQLrows

JDBC

SQLrows

Java &Java &

WebWeb

ServicesServices

Page 7: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

JDBC

Simply – data source connection

OK with:“Window on data” applications

Business logic entrenched ondatabase

Java nothing more than GUI tool

Is it your only option for “highperformance persistence”? –NO!

SQLrows

JDBC

SQLrows

Java &Java &

WebWeb

ServicesServices

Page 8: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Persistence Layer

Abstracts persistence details from theapplication layer

SQLrows

Objects

Persistence Layer

Objects

object-level

querying and creation

results are objects

results are

returned as

raw data

API uses SQL

or database

specific calls

object creation and

updates through

object-level API

Java &Java &

WebWeb

ServicesServices

Page 9: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Persistence Layer

SQLrows

Objects

JDBC

Objects

object-level

querying and creation

results are objects

results are

returned as

raw data

API uses SQL

or database

specific calls

object creation and

updates through

object-level API

J2EE &J2EE &

WebWeb

ServicesServices

Persistence Layer

Page 10: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

ORM Persistence Entity Types

EJB 2 Persistence: Entity Beans

BMP: Developer must hand code persistence “lifecycle” calls generate by J2EE Container

CMP: More Automatic Persistence

“Plain Old Java Objects” (POJO) Persistence

Oracle TopLink

Open Source: JBoss Hibernate

DAO Pattern: Custom, Generators

JDO Implementations

EJB 3.0 Persistence: Entity Beans == POJOs

Page 11: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

iBATIS – Somewhere inbetween…

iBATIS is not meant to replace Object-Relational Mapping(ORM) tools. It is a low-level framework that allows you todirectly hand-code your SQL statements and map them toJava object. You don’t need to lookup DataSource, getconnections, create prepared statements, parse ResultSetor even cache the results – iBATIS does it all for you.Under the covers, iBATIS creates a PreparedStatement, setsthe parameters (if any), executes the statements and buildsa Map or JavaBean object from the ResultSet.

Summarized from Vinny Carpenter’s BLOG entry:

http://www.j2eegeek.com/blog/2005/01/31/ibatis-where-have-you-been-all-my-life/

Page 12: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

AgendaPersistence Architecture Overview

Performance Points:Models and Mapping

Inheritance

Locking

Querying

Lazy Reading

N+1 Reads

Projections

Transactions

Caching

Leveraging the Database and JDBC

Page 13: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Impedance Mismatch

Difference in relational and objecttechnology know as “object-relationalimpedance mismatch”

Challenging problem

Requires relational and object expertise

Page 14: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Impedance Mismatch

Factor J2EE RDBMSLogical Data

Format

Objects, methods,

inheritance

Tables, SQL, stored

procedures

Scale Hundreds of megs Gigabytes, terabytes

Relationship Object references Foreign keys

Uniqueness Internal object identity Primary keys

Key Skills Java development,

object modeling

SQL, Stored Procedures, data

management, Data modeling

Tools IDE, Source code

management, Object

Modeler

Schema designer, query

manager, performance

profilers, database config

Corporate

Org.

Structure

“Newer technology”

often with weak

organizational ties to

database mgmt

Often mature infrastructure

with significant legacy

considerations

Technic

al

Busin

ess

Polit

ical

Page 15: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Mapping

Object model and Schema must be mapped

True for any persistence approach

Most contentious issue facing designers

Which classes map to which table(s)?

How are relationships mapped?

What data transformations are required?

Page 16: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Mapping Process

Map Object Model to schema

Java IDEJava source

MappingTool

Schema

Mapping

Metadata

(XML/Annotation)

Page 17: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Mapping Tools

Theunderlyingmappingflexibility isveryimportant

Page 18: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Modeling and Mapping

Options:

Generate Object Model from schema

Generate schema from Object Model

Meet in the Middle

• Existing Object Model

• Existing Schema

Page 19: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Modeling and MappingGeneration

Excellent productivity boost at start ofproject

Don’t take the generated code/schema asgospel!

Tools are smart not psychic

Many legacy schemas will lead to classicproblems discussed shortly…

Page 20: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Recommendations

Ensure ORM layer allows customization ofmappings

Match the approach to current and long termproject requirements

New/Flexible schema versus legacy staticschema

Future enhancements

Maintenance

If absolutely tied to a schema/model with nohope of change – reconsider strategy

Page 21: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

AgendaPersistence Architecture Overview

Performance Points:Models and Mapping

Inheritance

Locking

Querying

Lazy Reading

N+1 Reads

Projections

Transactions

Caching

Leveraging the Database and JDBC

Page 22: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Impedance Mismatch:Inheritance

Java

Used to extend (share) common state &behavior

Relational

Shared data can be normalized into commontable

Adds additional unique constraint withincommon table

Page 23: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Inheritance

Very straight forward at first, but canbecome challenging because of the numberof possible solutions

Compounded with relationships

Can be source of contention with DBA –generally leads to lots of joining on queries!

Page 24: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

“Leaf Table Mapping”

Sub1

sub1var: String

Root {a}

id: int

var: String

Sub2 {a}

sub2var: int

Sub22

sub22var: String

Map only concrete classes

SUB1

ID VAR

SUB22

7 Foo

SUB1VAR

Pish

ID VAR

13 Tim

SUB2VAR

677

SUB22VAR

Flim

Page 25: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

“Leaf Table” Issues

How do you query for the Root class?

Data is not normalized

Performance Impact:

PRO: No joins or table contention between types

CON: No heterogeneous query results

Is this your fastest option? If you only do leafqueries, perhaps it is…

Page 26: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

“All Table Mapping”

Sub1

sub1var: String

Root {a}

id: int

var: String

Sub2 {a}

sub2var: int

Sub22

sub22var: String

Map allclasses ROOT

ID VAR

SUB2

7 Foo

13 Bar

ID

13

SUB2VAR

677

SUB22VAR

Flim

SUB1

ID

7

SUB1VAR

Pish

ID

13

SUB22

Problem!How to telltype?!

Page 27: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Side Bar: Identifying Type

Sub1

sub1var: String

Root {a}

id: int

var: String

Sub2 {a}

sub2var: int

Sub22

sub22var: String

Map allclasses ROOT

ID VAR

SUB2

7 Foo

13 Bar

ID

13

SUB2VAR

677

SUB22VAR

Flim

SUB1

ID

7

SUB1VAR

Pish

ID

13

SUB22

Need TypeIdentifier!

TYPE

S1

S22

Page 28: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Side Bar: Identifying Type

ROOT

ID VAR

7 Foo

13 Bar

Lots of strategies exist withvarious performanceimpacts…

TYPE

S1

S22 ROOT

ID VAR

7 Foo

13 Bar

TYPE

com.foo.Sub1

com.foo.Sub22

ROOT

ID VAR

7 Foo

13 Bar

if id < 10

type = Sub1

else

type = Sub22

Page 29: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Side Bar: Identifying Type

Another option to “type identifier fields”exists Just read through ALL Leaf classesand figure it out in memory… Performance?

Sub1sub1var: String

Root {a}

id: int

var: String

Sub2 {a}sub2var: int

Sub22sub22var: String

ROOTID VAR

SUB2

7 Foo

13 Bar

ID

13

SUB2VAR

677

SUB22VAR

Flim

SUB1ID

7

SUB1VAR

Pish

ID

13

SUB22

Page 30: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

“All Table” Issues

Lots (and lots) of joining

Type identification can be inefficient

Performance Impact:

PRO: Root Level/Abstract Class query resultspossible

CON: Additional queries or joins required

CON: Type Identification

Page 31: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

“Single Table Mapping”

Sub1

sub1var: String

Root {a}

id: int

var: String

Sub2 {a}

sub2var: int

Sub22

sub22var: String

Map to one table

ROOT

ID VAR

7 Foo

13 Bar

SUB1VAR

Pish

null

SUB2VAR

null

677

SUB22VAR

null

Flim

Page 32: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Identifying Type

Sub1

sub1var: String

Root {a}

id: int

var: String

Sub2 {a}

sub2var: int

Sub22

sub22var: String

ROOT

ID VAR

7 Foo

13 Bar

SUB1VAR

Pish

null

SUB2VAR

null

677

SUB22VAR

null

Flim

TYPE

S1

SUB22

if SUB1VAR != null

type = Sub1

else

type = Sub22

Two options fortype detection

Page 33: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

“Single Table” Issues

DBA will freak out – de-normalized database

Type identification can be inefficient

Performance Impact:PRO:• Heterogeneous query results possible• No extra queries or joins for subclasses

CON: Additional table size, unused columns

Page 34: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

“Combination Table Mapping”

Sub1

sub1var: String

Root {a}

id: int

var: String

Sub2 {a}

sub2var: int

Sub22

sub22var: String

Mapappropriateclasses

ROOT

ID VAR

7 Foo

13 Bar

SUB2VAR

677

SUB22VAR

Flim

SUB1

ID

7

SUB1VAR

Pish

ID

13

SUB22

TYPE

S1

S22

Page 35: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Which Strategy to Use?

Leaf TableWhen never querying at abstract class level

All TableWhen not doing much querying• Too many joins

Single TableWhen one subclass is much more prevalent, andyour DBA doesn’t care about normalization

CombinationUse to optimize above situations

Page 36: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Wouldn’t It Be Cool?

Sub1

sub1var: String

Root {a}

id: int

var: String

Sub2 {a}

sub2var: int

Sub22

sub22var: String

Map to view for reads…ROOT (View)

ID VAR

7 Foo

13 Bar

SUB1VAR

Pish

null

SUB2VAR

null

677

SUB22VAR

null

Flim

ROOT

ID VARSUB2

7 Foo

13 BarID

13

SUB2VAR

677

SUB22VAR

Flim

SUB1ID

7

SUB1VAR

Pish ID

13

SUB22

Map to tables for writes…

Page 37: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

AgendaPersistence Architecture Overview

Performance Points:Models and Mapping

Inheritance

Locking

Querying

Lazy Reading

N+1 Reads

Projections

Transactions

Caching

Leveraging the Database and JDBC

Page 38: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Locking

J2EE Developers want to think of locking atthe object level

Databases may need to manage lockingacross many applications

Persistence layer or application server mustbe able to respect and participate in locks atdatabase level

Page 39: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Optimistic Locking

DBA may wish to use version, timestampand/or last update field to representoptimistic lock

Java developer may not want this in theirbusiness model

Persistence layer must be able to abstract this

Must be able to support using any fieldsincluding business domain

Page 40: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Optimistic Locking

EMP

ID NAME

7 Don

13 Trisha

VER

3

8

SAL

40,000

55,000

UPDATE EMP SET NAME=‘DONALD’,

VER=4 WHERE ID=7 AND VER=3

UPDATE EMP SET SAL=50,000,

VER=4 WHERE ID=7 AND VER=3

SQL from Transaction B

SQL from Transaction A

ID NAME

7 Don

13 Trisha

VER

4

8

SAL

50,000

55,000

ID NAME

7 Donald

13 Trisha

VER

4

8

SAL

40,000

55,000

If A “Wins”:

B gets “Opt Lock Exception

If B “Wins”:

A gets “Opt Lock Exception

OR

Page 41: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Pessimistic Locking

Requires careful attention as a JDBCconnection is required for duration ofpessimistic lock

Should support SELECT FOR UPDATE[NOWAIT] semantics

Tim

e

Begin UOW

Commit UOW

Begin Txn

Commit Txn

Pess Lock

J2EE &J2EE &

WebWeb

ServicesServices

Page 42: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Concurrency Protection:Locking

Proper locking semantics is a must

Recommendations:

Use optimistic locking

• Reads can avoid unnecessary refresh by comparing lock values

Minimize pessimistic locking

• Reduce database resource contention with longer transactions

• Use when retry is more costly then locking

• Typically used in data processing or non-user interactivemodules

Page 43: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

AgendaPersistence Architecture Overview

Performance Points:Models and Mapping

Inheritance

Locking

Querying

Lazy Reading

N+1 Reads

Projections

Transactions

Caching

Leveraging the Database and JDBC

Page 44: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Querying

Critical element of ORM tuning

Most applications have more reading then writing

GOALS

Reducing the amount of SELECT statements

Optimize the speed of each SELECT statement

RECOMMENDED SOLUTIONS

Just in time reading

Result caching – object or result container

Optimized SQL generation

Data projections versus full object loading

Page 45: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

AgendaPersistence Architecture Overview

Performance Points:Models and Mapping

Inheritance

Locking

Querying

Lazy Reading

N+1 Reads

Projections

Transactions

Caching

Leveraging the Database and JDBC

Page 46: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Object Traversal – Lazy Reads

J2EE Applications work on the scale of afew hundreds of megabytes

Databases routinely manage gigabytesand terabytes of data

Persistence layer must be able totransparently fetch data “just in time”

Page 47: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Classic Support Case

Customer – “Uh, Hi. I am trying to read 1object and get 3,000 SQL statements.”

Support – “Have you turned on indirection?”

Customer – *silence*

Support – “Turn on Proxies and let us knowwhat happens”

Customer – “Oh, 3 SQL statements now.Thanks ”

Page 48: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Just in Time Reading – FaultingProcess

Customer

Orders

Proxy

1. Accessing relationshipfor first time

2. Get relatedobject basedon FK3b. SQL if

not cached3a. CheckCache

4. PlugresultintoProxy

Page 49: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Lazy Loading – Just in TimeReading

Use of proxy to defer reading until required

Very valuable performance feature

Several Implementation Options

Explicit proxy

Dynamic proxy (java.lang.reflect.Proxy)

Development time class enhancement (source or byte codes)

Dynamic weaving (AOP)

Customer AddressValueHolder

List PhoneNumber

Page 50: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

AgendaPersistence Architecture Overview

Performance Points:Models and Mapping

Inheritance

Locking

Querying

Lazy Reading

N+1 Reads

Projections

Transactions

Caching

Leveraging the Database and JDBC

Page 51: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

N+1 Reads Problem

Many persistence layers and applicationservers have an N+1 reads problem

Causes N subsequent queries to fetchrelated data when a collection is queried for

A side effect of the impedance mismatch andpoor mapping and querying support inpersistence layers

Page 52: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

N+1 Reads Problem

Pool of Created

Objects or Beans

Persistence

Layer or EJB

Container

findByCity()

1

findByCity()

2

For each Customer

Fetch their Address

Address

46

4

n

5 5

3 Returns collection

n

If Address had related

objects, they too may be

fetched 2n+1 Reads!

Container returns results

C C C C

Page 53: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Classic Support Case #2

Customer – “Uh, Hi. I am trying to read 1object and get 3,000 SQL statements.”

Support – “Have you turned on indirection?”

Customer – “Yes, and it’s still at 3,000 SQLs.”

Support – “Turn on join/batch reading”

Customer - * SILENCE *

Customer – “Oh, 3 SQL statements now.Thanks ”

Page 54: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Example Object Model

PurchaseOrder

id : int

customer : Customer

Customer

id : int

address : Address

phoneNumbers : List

Address

id : int

street : String

city : String

state : String

postalCode : String

country : String

PhoneNumber

customer : Customer

number : String

type : String

1

1

1

*

Page 55: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Query Example

Find all pending PO’s with a customerin San Francisco and display customerdetailsEJB QL

SELECT po FROM PurchaseOrder po

WHERE po.status = 'ACTIVE' AND

po.customer.address.city = 'SFO‘

Page 56: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Query Example: Initial SQL

SELECT PO.* FROM PO, CUST, ADDR WHEREPO.STATUS = ‘ACTIVE’ AND PO.CUST_ID =CUST.ID AND CUST.ADDR_ID = ADDR.ID ANDADDR.CITY = ‘SFO’

{Returns N Purchase Orders}

SELECT * FROM CUST WHERE CUST.ID = 1 …{N}

SELECT * FROM ADDR WHERE ADDR.ID = 100 …{N}

SELECT * FROM PHONE WHERE PHONE.CUST_ID =1 … {N}

RESULT: 3N+1 queries (100 PO’s = 301)

“N+1 Query – explosion of SQL”

Page 57: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

N+1 Reads

Must have solution to minimize queries

Need flexibility to reduce to 1 query, 1+1query or N+1 query where appropriate

1 Query when displaying list of customers andaddresses – known as a “Join Read”

1+1 Query when displaying list of customers anduser may click button to see addresses – known asa “Batch Read”

N+1 Query when displaying list of customers butonly want to see address for selected customer

Page 58: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Where Do I Define This?

Many options exist, depends on persistencelayer and spec

In the query language syntax

On the mapping

Development time/runtime

Page 59: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Join Reading 1:1’s

Join ReadingJoin rows together in a single SELECT

Example of generated SQLSELECT PO.*,CUST.*,ADDR.* FROM PO, CUST,ADDR WHERE PO.STATUS = ‘ACTIVE’ ANDPO.CUST_ID = CUST.ID AND CUST.ADDR_ID =ADDR.ID AND ADDR.CITY = ‘SFO’

{Returns N Purchase Orders with Customers& Addresses}

SELECT * FROM PHONE WHERE PHONE.CUST_ID = 1

…{N calls}

RESULT: N+1 queries (100 Phone #’s = 101)

Page 60: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Join Reading All Relationships

Join ReadingRead in rows that can be joined together in asingle SELECT

ExampleSELECT PO.*,CUST.*,ADDR.*, PHONE.*FROM PO, CUST, ADDR, PHONEWHERE PO.STATUS = ‘ACTIVE’ AND PO.CUST_ID= CUST.ID AND CUST.ADDR_ID = ADDR.ID ANDADDR.CITY = ‘SFO’ AND PHONE.CUST_ID =CUST.ID

{Returns N Purchase Orders withCustomers, Addresses, and PhoneNumbers}

RESULT: 1 query (100 PO’s)

Page 61: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Batch and Join Reading

Use SELECT per child with join to originalclause

ExampleSELECT PO.*,CUST.*,ADDR.* FROM PO, CUST, ADDR WHERE

PO.STATUS = ‘ACTIVE’ AND PO.CUST_ID = CUST.ID ANDCUST.ADDR_ID = ADDR.ID AND ADDR.CITY = ‘SFO’

{Returns N Purchase Orders with Customers &Addresses}

SELECT PHONE.* FROM PHONE, PO, CUST, ADDR WHEREPO.STATUS = ‘ACTIVE’ AND PO.CUST_ID = CUST.ID ANDCUST.ADDR_ID = ADDR.ID AND ADDR.CITY = ‘SFO’ ANDPHONE.CUST_ID = CUST.ID

RESULT: 2 queries (100 PO’s)

Page 62: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Joining + Batch Reading

Use SELECT per child with join to originalclause

ExampleSELECT PO.* FROM PO, CUST, ADDR WHERE PO.STATUS =

‘ACTIVE’ AND PO.CUST_ID = CUST.ID AND CUST.ADDR_ID= ADDR.ID AND ADDR.CITY = ‘SFO’

SELECT CUST.* FROM PO, CUST, ADDR WHERE PO.STATUS =‘ACTIVE’ AND PO.CUST_ID = CUST.ID AND CUST.ADDR_ID= ADDR.ID AND ADDR.CITY = ‘SFO’

SELECT ADDR.* FROM PO, CUST, ADDR WHERE PO.STATUS =‘ACTIVE’ AND PO.CUST_ID = CUST.ID AND CUST.ADDR_ID= ADDR.ID AND ADDR.CITY = ‘SFO’

SELECT PHONE.* FROM PHONE, PO, CUST, ADDR WHEREPO.STATUS = ‘ACTIVE’ AND PO.CUST_ID = CUST.ID ANDCUST.ADDR_ID = ADDR.ID AND ADDR.CITY = ‘SFO’ ANDPHONE.CUST_ID = CUST.ID

RESULT: 100 PO’s = 4 SQL

Page 63: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Join & Batch Reading

Reducing the quantity of SQL calls willproduce a significant performance gain

Joining and batching can be combined

Be careful not to extend the depth too deep

The cost of the join could become too expensive

Page 64: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Query Optimization Options Reading 100 PO’s with relationships

Source: Simple test environment

# SQL Time

Default 301 200 ms

Join 1:1’s 101 150 ms

Join All 1 60 ms

Batch All 4 40 ms

Batch & Join 2 20 ms

Fewer SQL ! Always = Faster Execution

Page 65: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

AgendaPersistence Architecture Overview

Performance Points:Models and Mapping

Inheritance

Locking

Querying

Lazy Reading

N+1 Reads

Projections

Transactions

Caching

Leveraging the Database and JDBC

Page 66: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Queries: Minimize Data Read

Only read the entire object when required

When modifying the object

When caching will assist concurrent clients

All fields of the object are required

Consider projections versus object queries

Page 67: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Query: Projection Example

EJB QL: SELECT new PODetails(po.id, po.num,

po.amount, c.lastName, a.city)

FROM PurchaseOrder po JOIN po.customer c,

JOIN po.customer.address a

WHERE po.status = 'ACTIVE‘ AND a.city =

'SFO‘

SQL: SELECT PO.ID, PO.NUM, PO.AMOUNT, CUST.L_NAME,

ADDR.CITY FROM PO, CUST, ADDR WHERE PO.STATUS =

‘ACTIVE’ AND PO.CUST_ID = CUST.ID AND

CUST.ADDR_ID = ADDR.ID AND ADDR.CITY = ‘SFO’

New YorkJones500.00CD4562

OttawaSmith100.00AB1231

CityLast$PO #ID

Page 68: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Object Traversals

How would a developer using a typicalpersistence layer find “The averageorder amount for customers in city X”?

In other words, how do Java developerswrite “report” style queries?

CustomerOrder OrderItem

Productname

city

orders

orderDate

cost

orderItems

quantity

discount

product

productName

cost

weight

1

1 11* *

Page 69: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Example Codepublic double findAverageOrderSizeFor(String city) {

int count=0; double value=0.0;

Customer c = new Customer();

c.setCity(city);

Collection custs = pLayer.executeQueryByExample(c);

iterate i over custs {

Collection ords = custs[i].getOrders();

iterate j over ords {

value += ords[j].getValue(); count++;

}

}

return value/count;

}

This approach causes unnecessary

interactions with the database and

brings too much data into the

appserver…

Page 70: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

A Better Solution

EJB QL: SELECT avg(ord.amount)

FROM Customer cu JOIN cu.order o,

WHERE cu.city = :city

SELECT AVG(t0.VALUE) FROM ORDERS t0, ADDRESS t2, CUSTOMER t1

WHERE ((t2.NAME = city) AND ((t0.CUST_ID = t1.ID) AND (t2.ID = t1.ADD_ID)))

Generates

Projections allow Java developers to leverage

the database instead of trying to do all the

computation in JVM

Page 71: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Query: Projection Optimizations

Specify the required fields in terms of theobject model

Use relationships to indicate joins

Use aggregate functions

SUM, MIN, MAX, COUNT, AVERAGE

Produce easy to render results

Page 72: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

AgendaPersistence Architecture Overview

Performance Points:Models and Mapping

Inheritance

Locking

Querying

Lazy Reading

N+1 Reads

Projections

Transactions

Caching

Leveraging the Database and JDBC

Page 73: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Transactions: Short & Sweet

J2EE apps typically support many clients sharingsmall number of db connections

Ideally would like to minimize length of transactionon database

Tim

e

Begin UOW

Commit UOWBegin Txn

Commit Txn

J2EEJ2EE

AppApp

Page 74: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Transactions: Minimal Writes

Persistence layer tracks/calculates changes

Only UPDATE modified columns

Order UPDATE, INSERT, & DELETES usingreferential integrity rules and consistentorder

Benefits

Simplified development

Reduced TX costs

Minimize deadlock occurrences

Page 75: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Transactions: Minimize Context

Each object in a transaction context has acost

Only include objects which may be changedin the transaction context

Symptoms

Long commit cycles

Increased garbage creation

Page 76: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Minimize Context

Bad:

Better:

begin UOWread 10,000 objectspick 1change 1

commit UOW

read 10,000 objectspick 1begin UOW

change 1commit UOW

Page 77: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Transactions: Other Optimizations

Data Access Optimizations

Parameter Binding

ResultSet streaming

Streaming of large objects (BLOB/CLOB)

Cache Interaction

Transaction updates to shared object cache

Distribution of changes to clustered caches

Page 78: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

AgendaPersistence Architecture Overview

Performance Points:Models and Mapping

Inheritance

Locking

Querying

Lazy Reading

N+1 Reads

Projections

Transactions

Caching

Leveraging the Database and JDBC

Page 79: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Why Have a Cache?

Object Identity

Performance – Avoid extra database trips

Reference Read-Only

Read-mostly

Page 80: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Caching

OO Query SQL Query

Results(s)

Does PK for rowexist in cache?

YES – Getfrom Cache

NO – Buildbean/objectfrom results

Returnobjectresults

Page 81: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

ORM Caching Options

Isolated cache

Per Client/Thread

Transactional

Conforming

Shared cache

EntityManager

Session

EntityManager

Factory

Server

UnitOfWork

Cache Cache

Cache

J2EEJ2EE

AppApp

Page 82: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Caching Architecture

ServerShared

Cache

SessionIsolated

Cache

UnitOfWork

TX

Cache

ServerShared

CacheSession Bus

JDBC/JCA

JMS (MDB)

RMI

CORBA

IIOP

Distributed/Coordinated

Configurable/Flexible

Page 83: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Cache Configuration

Categorize your persistent entity types

Isolated: short cache life

Shared: medium to long cache life (app/dbspecific)

Reference: Long cache life

Configure each type

Cache type

Initial and/or maximum size

Invalidation (time-to-live)

Page 84: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Caching –Stale Cache Management

Non-locking read = stale data

Trying to eliminate this issue can lead tomajor performance penalties

Addressing stale cache

Configurable cache life-cycle

Refresh using query when required

Use optimistic lock value to avoid extra refresh

Avoid turning off the cache

Page 85: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Distributed/CoordinatedCaching

Following successful database commit

Changes are merged into shared cache

Changes messaged to clustered sessions

Ensure messaging is warranted

Fail-over

Shared objects between users

Server

Shared

Cache

Server

Shared

CacheSession Bus

Page 86: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

AgendaPersistence Architecture Overview

Performance Points:Models and Mapping

Inheritance

Locking

Querying

Lazy Reading

N+1 Reads

Projections

Transactions

Caching

Leveraging the Database and JDBC

Page 87: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Leverage the RelationalDatabase

Schema adjustments

Indexes

Order of calls & columns

Stored Procedures

PK Sequencing (pre-allocation)

Hints in SQL

Page 88: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Leverage JDBC Features

Prepared Statement Caching

Parameter binding

Large data element streaming

Result scrolling

Page 89: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Work with Your DBA

Some ORM approaches ignore RDBMS best-practices

Don’t restrict the evolution of the relationalschema

Allow for the use of indices and storedprocedure where necessary

Leverage advanced features of the databaseHints in SQL

Custom types (ORM data types)

XML storage

Page 90: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Summary

Know or learn your application data andusage

Know your ORM persistence layer

Leverage the database & JDBC capabilities

General Performance Tuning

Page 91: High Performance Persistence - Software Summitsoftwaresummit.com/2005/speakers/SmithDonaldHigh... · Donald Smith — High Performance Persistence Page iBATIS – Somewhere in between…

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation

Donald Smith — High Performance Persistence Page

Q&A