© 2007 adp 1 java capacity planning & performance measurements dr. carl j. de pasquale...

© 2007 ADP© 2007 ADP1

Java Capacity Planning & Performance Measurements

Dr. Carl J. De [email protected]

Electronic Numerical Integrator and

Computer Electronic Discrete Variable Automatic Computer

© 2007 ADP© 2007 ADP2


• Agenda– Capacity vs. Performance– Java capacity planning

• Java black box– Tools

• Java application– Tools

– Performance Measurements• Coding standards

• Byte code instrumentation– Method performance measurements using management library

– Method performance measurements using ARM & management library

• Tools

– Conclusions

© 2007 ADP© 2007 ADP3


Hardware Does Not Always Solve Capacity Problems • Capacity of a given hardware configuration is subject to the

application constraints

Capacity = f(Hardware, Application Performance)• Assume a given hardware configuration

– is 70% utilized, – Provides a 3 second application response,– When the arrival rate is two hundred transactions per second.– Management upgrades the hardware configuration with one that is

twice as fast

What could be the affect on the response time?

© 2007 ADP© 2007 ADP4


What could be the affect on the response time?• Response Time Could Improve

– If there isn’t an application CPU bottleneck– If there isn’t an application threading bottleneck– If there isn’t a network or I/O bottleneck

• Response Time Could Degrade– Work arrives faster

• Saturates non CPU peripherals• CPU experiences I/O Waits

– If there is an application threading bottleneck– If SQL is a poorly written

• More physical I/O executed than required• table scans/small reads

– If there is a Network Bottleneck• Small packet size

© 2007 ADP© 2007 ADP5


• Black box Java Capacity Planning– The Java Process

• CPU

• Disk

• Network

• Memory

– No application visibility

– Many java classes running in one JVM

• Application Capacity Planning– Process java

• Application Class/Method Data– CPU time by method

– Elapsed time by method

– Method invocation count

– Memory usage heap

– Workflow trace

– SQL timing

© 2007 ADP© 2007 ADP6


• Capacity Planning Black Box (java) - No Application Visibility– Collect process data– Identify busy period– Analyze data– Review reports to identify hot spots

• Computer CPU utilization

• Computer summary

• Disk IO summary

• Network summary

• Memory summary

• Process summary – Java will usually be the top resource consumer

© 2007 ADP© 2007 ADP7


• Capacity Planning Black Box (java) - No Application Visibility– Identify and eliminate all bottlenecks (analytically)– Apply and monitor changes to physical infrastructure– Once bottlenecks are eliminated

• Recollect performance data

• Use Analytical modeling to help determineCapacity = f(Hardware, Application Performance)

– Tools • Opnet IT GURU

• Hyperformix Capacity Manager

• Team Quest

• Performance Assurance

© 2007 ADP© 2007 ADP8


• Application Capacity Planning (java) - Application Visibility– Coding Standards

• Application Specific Log– Non standard format

– Each application log must be parsed by capacity planner

– May or may not contain necessary data

– Better than nothing

– Tools• Byte Code Engineering Library (BCEL)

• Java ClassLoader

© 2007 ADP© 2007 ADP9


Sample Codepublic final class test {

test () { }

public void runtest () {

for ( int i = 0 ; i < 9999999 ; i++ ) {}

}

public static void main(String[] argv) {

try {

for (int i = 0 ; i < 1; i++){

test t1 = new test();

t1.runtest();

for (int j = 0 ; j < 2 ; j++)

new oohrah ( argv[0], argv[1] ) ;

}

} catch ( Exception e ) {

e.printStackTrace();

}

}

}

Sample Codepublic class oohrah

{

public oohrah( String a, String b ) {

spin();

}

public static void spin() {

int i;

new hello();

for (i=0; i < 500000000; i++);

}

}

public class hello

{

public hello( ) {

printHello();

}

public static void printHello() {

System.out.println ("HelloWorld (from hello)");

}

}

© 2007 ADP© 2007 ADP10


• Dynamic Byte Code Instrumentation - JDK 1.5Start

End

Parse Java Class

Create ClassGen, Get Constant Pool, Get Methods

Get Method

Are All Methods processed

If Main

Crete Print Count Method

Yes

Instrument MethodsNo

Yes

No

© 2007 ADP© 2007 ADP11


• Dynamic Byte Code Instrumentation - JDK 1.5Start Class Name=test,Method Name main Elapsed 1180537593616 CPU 515625000Start Class Name=test,Method Name runtest Elapsed 1180537593632 CPU 515625000End Class Name=test,Method Name runtest Elapsed 1180537593663 CPU 546875000ClassName=test runtestCount==> 1ClassName=test runtestCpu==> 31250000ClassName=test mainCount==> 1ClassName=test mainCpu==> 0Start Class Name=oohrah,Method Name spin Elapsed 1180537593679 CPU 562500000Start Class Name=hello,Method Name printHello Elapsed 1180537593679 CPU 562500000HelloWorld (from hello)End Class Name=hello,Method Name printHello Elapsed 1180537593679 CPU 562500000ClassName=hello printHelloCount==> 1ClassName=hello printHelloCpu==> 0End Class Name=oohrah,Method Name spin Elapsed 1180537595053 CPU 1890625000ClassName=oohrah spinCount==> 1ClassName=oohrah spinCpu==> 1328125000Start Class Name=oohrah,Method Name spin Elapsed 1180537595053 CPU 1890625000Start Class Name=hello,Method Name printHello Elapsed 1180537595053 CPU 1890625000HelloWorld (from hello)End Class Name=hello,Method Name printHello Elapsed 1180537595053 CPU 1890625000ClassName=hello printHelloCount==> 2ClassName=hello printHelloCpu==> 0End Class Name=oohrah,Method Name spin Elapsed 1180537596366 CPU 3203125000ClassName=oohrah spinCount==> 2ClassName=oohrah spinCpu==> 2640625000End Class Name=test,Method Name main Elapsed 1180537596381 CPU 3218750000ClassName=test runtestCount==> 1ClassName=test runtestCpu==> 31250000ClassName=test mainCount==> 1ClassName=test mainCpu==> 2703125000

© 2007 ADP© 2007 ADP12


• Application Capacity Planning (java) - Application Visibility• ARM Instrument

– Elapsed time

– Invocation counts

– Correlation

– ArmTransactionsWithMetrics

• CPU time

– Tools• Byte Code Engineering Library (BCEL)

• Java ClassLoader

© 2007 ADP© 2007 ADP13


• Dynamic Byte Code Instrumentation - ARMStart

End

Parse Java Class

Create ClassGen, Get Constant Pool, Get Methods

Get Method

Are All Methods processed

ARM Instrument Method

Yes

No

© 2007 ADP© 2007 ADP14


• Dynamic Byte Code Instrumentation – ARMarm_start_transaction()

time=03:09:34 06/13/07

transaction id=02000000 00000000 00000000 00000000

parent correlator: none

flags=NONE

application name=ApplicationDefinition test

transaction name=TransactionDefinition test.main

transaction instance handle=1

© 2007 ADP© 2007 ADP15


• Dynamic Byte Code Instrumentation – ARMarm_start_transaction() time=03:09:34 06/13/07 transaction id=04000000 00000000 00000000 00000000 parent correlator: length=44 format=126 flags=0 app name=ApplicationDefinition test app handle=1 tran name=TransactionDefinition test.main tran instance id=1 flags=NONE application name=ApplicationDefinition test transaction name=TransactionDefinition test.runtest transaction instance handle=2

arm_stop_transaction() time=03:09:34 06/13/07 transaction instance handle=2 transaction status=GOOD flags=NONE application name=ApplicationDefinition test transaction name=TransactionDefinition test.runtest

© 2007 ADP© 2007 ADP16


• Dynamic Byte Code Instrumentation – ARMarm_start_transaction() time=03:09:34 06/13/07 transaction id=06000000 00000000 00000000 00000000 parent correlator: length=44 format=126 flags=0 app name=ApplicationDefinition test app handle=1 tran name=TransactionDefinition test.main tran instance id=1 flags=NONE application name=ApplicationDefinition oohrah transaction name=TransactionDefinition oohrah.spin transaction instance handle=3

© 2007 ADP© 2007 ADP17


• Dynamic Byte Code Instrumentation – ARMarm_start_transaction() time=03:09:34 06/13/07 transaction id=08000000 00000000 00000000 00000000 parent correlator: length=44 format=126 flags=0 app name=ApplicationDefinition oohrah app handle=3 tran name=TransactionDefinition oohrah.spin tran instance id=3 flags=NONE application name=ApplicationDefinition hello transaction name=TransactionDefinition hello.printHello transaction instance handle=4

© 2007 ADP© 2007 ADP18


• Dynamic Byte Code Instrumentation – ARMarm_stop_transaction()

time=03:09:34 06/13/07


transaction status=GOOD

flags=NONE

application name=ApplicationDefinition hello

transaction name=TransactionDefinition hello.printHello

arm_stop_transaction()

time=03:09:35 06/13/07



flags=NONE

application name=ApplicationDefinition oohrah

transaction name=TransactionDefinition oohrah.spin

© 2007 ADP© 2007 ADP19



time=03:09:35 06/13/07

transaction id=0a000000 00000000 00000000 00000000

parent correlator:

length=44

format=126

flags=0

app name=ApplicationDefinition test

app handle=1

tran name=TransactionDefinition test.main

tran instance id=1

flags=NONE




© 2007 ADP© 2007 ADP20



time=03:09:35 06/13/07

transaction id=0c000000 00000000 00000000 00000000

parent correlator:

length=44

format=126

flags=0

app name=ApplicationDefinition oohrah

app handle=5

tran name=TransactionDefinition oohrah.spin

tran instance id=5

flags=NONE




© 2007 ADP© 2007 ADP21


• Dynamic Byte Code Instrumentation – ARMarm_stop_transaction()

time=03:09:35 06/13/07



flags=NONE




time=03:09:35 06/13/07



flags=NONE




time=03:09:35 06/13/07



flags=NONE

application name=ApplicationDefinition test

transaction name=TransactionDefinition test.main

© 2007 ADP© 2007 ADP22


• Analyzing the Raw Data

TT -hput)me)/ThrougthodBusyTi((Users/Me sedTimeMethodElap

s/TCompletion Throughput

otalVisitspsedTime/T MethodEla Service

psedTime/T MethodElan Utilizatio

nPeriodObservatio T

nt MethodCou sCompletion

© 2007 ADP© 2007 ADP24


• Operational Analysis – Overloading• Process Method• Method Process• Valid for

• Black box, and application capacity planning

Denning, and J. Buzen, The operational analysis of queuing network models. Computer Surveys, 10(3),

225 - 261 (1978)

© 2007 ADP© 2007 ADP25


• Third Party Performance Analysis Tools- Unable to implement custom application logging - Unable to implement Dynamic Byte Code

Instrumentation- Third party alternatives

• Mercury

• Opnet Panorama/ACE

• Quest Perform Assure

• Wily

© 2007 ADP© 2007 ADP26


• ConclusionBy adding a bit of sophistication during application development, performance testing, or production deployment the often overlooked non-functional

requirement of “application performance” requirement can be addressed

© 2007 adp 1 java capacity planning & performance measurements dr. carl j. de pasquale...

Documents

capacity problems capacity

capacity planner

application response

java classes

application log

application cpu bottleneck

io bottleneck response

process data