adaptive instrumentation of java-applications for ... · national research center of the helmholtz...

KIT – University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association

KIT – SOFTWARE DESIGN AND QUALITY GROUP UNIVERSITY OF STUTTGART – RELIABLE SOFTWARE SYSTEMS GROUP

www.kit.eduwww.uni-stuttgart.deUniversity of Stuttgart

Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis

Symposium on Software Performance 2014 28.11.2014

Henning Schulz, Albert Flaig,Alexander Wert and André van Hoorn

KIT – Software Design and Quality GroupUniversity of Stuttgart – Reliable Software Systems Group

2/22 28-11-2014

Motivation


Performance?

Performance Engineer Instrumentation

Application

Application

Kieker & AIM ConclusionIntegration of Kieker & AIMExperiment-Based ScenariosIntroduction


3/22 28-11-2014

Instrumentation Overhead


methodA()

methodB() methodD()

methodC()

time

methodA()

methodB()

methodC()

methodD()

start end (no instrumentation) end (full instrumentation)



4/22 28-11-2014

time

start end (no instrumentation) end (full instrumentation)

Adaptive Instrumentation


methodA()

methodB()

methodC()

methodD()

methodA()

methodB()

methodC()

methodD()

methodA()

methodB() methodD()

methodC()



5/22 28-11-2014

Kieker vs. AIM

Kieker (http://kieker-monitoring.net) AIM (http://sopeco.github.io/AIM)Instrumentation at application start-up Instrumentation on demandStatically adaptive instrumentation Dynamically adaptiveExtensible (writer, etc.) Extendable (writer, etc.)Analysis infrastructure No analysis at allFocus on production Focus on experimentsRegexps for adaptive monitoring Instrumentation description language


Use dynamic adaptiveinstrumentation from AIMUse instrumentation descriptionlanguage

Use Kieker‘s monitoring &analysis infrastructure



6/22 28-11-2014

AIM IN EXPERIMENT-BASED CONTEXT


Henning Schulz: Adaptive Instrumentation of Java-Applications for Experiment-Based Performance Analysis, Bachelor‘s Thesis, Karlsruhe Institute of Technology, 2014


7/22 28-11-2014

AIM

Henning Schulz - AIM in Experiment-Based Context

JVM

Target Application

InjectedProbes

Java Process

MeasurementData

AIM Agent

Instrumentation Description

:Instrumentation Description

:Instrumentation Entity

:Method Scopepattern = my.package.*(int)

:Response Time Probe

instrumentationentity

probe

where

how

scope



8/22 28-11-2014

Evaluation

AIM is comparable to existing

approaches

• Compare to Kieker and DiSL• Instrumentation process:

• fast for < 500 methods• comparatively slow for > 500 methods

• Instrumentation overhead: smaller than Kieker’s

Benefits of adaptapable

instrumentation

• Automated Palladio Resource Demand Estimation




9/22 28-11-2014

Automated Estimation of Palladio Resource Demands

Derive Call Tree

Measure Response

Times of All Methods in

the Call Tree

Derive Resource

Demands by Subtraction

of Response Times

TPC-W DeploymentPalladio Model of TPC-W

Insert ResourceDemandsinto PCM

Model

SimulateCompareSimulation

Results

methodA()

methodB()

methodD()

methodC()

enter A

enter B

enter C

enter D

exit Cexit B

exit Dexit A

methodA()

methodB()

methodD()

methodC()

enter A

enter B

enter C

enter D

exit Cexit B

exit Dexit A

10%

100%

90%

80%70%

50%

60%

40%30%

20%

Pr

response time




10/22 28-11-2014

Comparing PRD Qualities

mean[ms]

meanerror [%]

Reference 4.52 -

AIM (adaptive) 4.52 0.01

AIM (full) 6.21 37.3

Kieker (adaptive) 4.95 9.4

Kieker (full) 6.57 45.4

Legend: R: Reference Aa: AIM adaptive Af: AIM full Ka: Kieker adaptive Kf: Kieker full

Measurement Simulation




11/22 28-11-2014

INTEGRATION OF KIEKER & AIM


Albert Flaig: Dynamic Instrumentation in Kieker Using Runtime Bytecode Modification, Bachelor‘s Thesis, University of Stuttgart, 2014


12/22 28-11-2014

Statically Adaptive Monitoring in Kieker

Albert Flaig - Integration AIM/Kieker

Expensive

Checks all Instrumentation

Patterns

Result iscached

Illustration based on Waller et al. (2012)



13/22 28-11-2014

Dynamically Adaptive Monitoring in Kieker


Illustration based on Waller et al. (2012)

MonitoredClass

monitoredMethod()



14/22 28-11-2014

Kieker

AIM Integration in Kieker

Monitoring Controller

Instrumentation Patterns

Java Process

Process of External Tool

JVM Instr. API

AIM

Target Application

BCInjector

JMX Client

Execution ProbeKieker

Instrumentor

Instrumentation

HotswapProbe Snippet

Javassist

Instrumentation Pattern

Instrumented Classes

Methods toinstrument

LoadedClasses

PatternList

Data FlowComponentPiece of Data

Record




15/22 28-11-2014

Evaluation

Bytecode instrumentation vs. Manual1. Overhead

• RQ1: How does it fare in terms of performance?

2. Turnaround• RQ2a: How does it fare in

terms of turnaround time?• RQ2b: Does it scale well?

3. Lost Transactions• RQ3: Is it reliable?

MooBench




16/22 28-11-2014

Evaluation 1: Overhead Comparison


0

2

4

6

8

10

12

14

Manual Instrumentation Bytecode Instrumentation

No Probe Deactivated Probe Collecting Data Writer

Exe

cutio

nTi

me

(µs)



17/22 28-11-2014

Evaluation 2: Turnaround (Setting)

Bytecode InstrumentationTurnaround = Activating probe + first execution of probeActivating probe invokes BCI

Manual InstrumentationTurnaround = Activating probe + first execution of probeFirst execution of probe invokes caching

Instrumenting 1,000 dummy classesComparison runtime BCI vs. caching




18/22 28-11-2014

Evaluation 2a: Turnaround Time

Bytecode Instrumentation

Manual Instrumentation




19/22 28-11-2014

Evaluation 2b: Scalability Comparison

5 502515 35 5 502515 35

# of classes

Turn

arou

nd T

ime

(ms)

Bytecode Instrumentation Manual Instrumentation

500

1000

1500

Turn

arou

nd T

ime

(ms)

0,3

0,5

0,7

500

1000

1500




20/22 28-11-2014

Conclusion

Core ResultsKieker & AIM synergiesSmall overhead reduction and higher flexibilityHigh turnaround time relevant for production needs optimization

Future WorkKieker supporting the instrumentation description languageOptimize Performance of AIM‘s instrumentation stepUsage in bigger context (e.g., performance problem diagnostics)Integration into Kieker release




21/22 28-11-2014

References

[Dmitriev, 2004]M. Dmitriev, “Profiling java applications using code hotswapping and dynamic call graph revelation”

[Ehlers et al, 2011]J. Ehlers, A. van Hoorn, J. Waller, and W. Hasselbring, “Self-adaptive software system monitoring for performance anomaly localization”

[Moret et al, 2011]P. Moret, W. Binder, and E. Tanter, “Polymorphic bytecode instrumentation”

[Marek et al, 2012]L. Marek, A. Villazon, Y. Zheng, D. Ansaloni, W. Binder, and Z. Qi, “Disl: A domain-specific language for bytecode instrumentation”

[van Hoorn et al, 2009]A. van Hoorn, M. Rohr, W. Hasselbring, J. Waller, J. Ehlers, S. Frey, and D. Kieselhorst, “Continuous monitoring of software services: Design and application of the kiekerframework”



22/22 28-11-2014

Kieker

AIM Integration in Kieker Detailed

Monitoring Controller JMX Controller

Probe ControllerInstrumentation Patterns

Kieker Class Loader

Java Process

Process of External Tool

JVM Instr. API

AIM

Target Application

BCInjector

JMX Client

Execution Probe Kieker Instrumentor

Instrumentation

HotswapProbe Snippet

Javassist

Instrumentation Pattern

Class Loaded

InstrumentedClasses

Classes toinstrument

Methods toinstrument

LoadedClasses

PatternList

Data FlowComponentPiece of Data

Probe Snippets

Record



23/22 28-11-2014

Evaluation 1: Overhead ComparisonE

xecu

tion

Tim

e (µ

s)

500 1000750

Manual Instrumentation

500 1000750

Bytecode Instrumentation



26/22 28-11-2014

Instrumentation Duration (TPC-W)


Legend: A: Instr. with AIM K: Instr. with Kieker D: Instr. With DiSL

(8 methods) (251 methods) (380 methods)


27/22 28-11-2014

Instrumentation Duration (Liferay)


Legend: A: Instr. with AIM K: Instr. with Kieker D: Instr. With DiSL

(6 methods) (2810 methods) (3270 methods)


28/22 28-11-2014

Overhead (MooBench & TPC-W)


Single Method

Legend: N: No Instr. A: Instr. with AIM K: Instr. with Kieker

(8 methods) (251 methods)(380 methods)


29/22 28-11-2014

Overhead (Liferay)


Legend: N: No Instr. A: Instr. with AIM K: Instr. with Kieker

adaptive instrumentation of java-applications for ... · national research center of the helmholtz...

Documents