java monitoring stuff you can get for free (and stuff you ... · java monitoring stuff you can get...

Java Monitoring Stuff You Can Get For Free (And Stuff You Can’t) Paul Jasek Sales Engineer

A Bit About Me

•  Current: •  Past:

•  Pre-Sales Engineer (1997 – present) •  WaveMaker •  Wily •  Persistence •  GemStone

•  Application Developer (1988 – 1997) •  Dow Jones •  Dun & Bradstreet •  General Electric / RCA

•  Hobbies •  Vinyl •  Lionel trains •  iPhone Development

2 March 5, 2012 | Copyright © 2010 AppDynamics. All rights reserved. "

Why should I Pay for a Monitoring Tool ?

The changing Application Landscape

Try out the Free Monitoring Tools

How people are benefitting from APM?

3

4

2

Today’s Agenda

53 Copyright © AppDynamics. All rights reserved. "

Let’s debug a Slow Application 1

App is running Slow … AND … •  It’s running in Production

•  Can’t restart the servers

•  Can’t reproduce in non prod environment •  Data sets and load are not alike

•  Just released a new version and it includes •  several refactoring, a new deployment architecture

•  Revenue critical app, can’t make it any slower by •  overloading it with intrusive monitors, additional logging

•  Development team is at a different location and time zone


Home grown - Logs, JMX Dashboards etc.

•  When it works •  First level of defense

•  Often added based on learning from outages and critical infrastructure components (pools, caches)

•  Systemic issues, e.g. recurring error / warn messages

•  When it doesn’t •  Sheer volume, don’t know where to start, after all logs are flat •  The issue is triggered by specific users, data, regions, etc. •  Logging libraries are often synchronous, and heavy on string

operations, hence can lead to performance issues •  What if you forgot to add “isDebugEnabled()”

•  Time and Server correlation is not easy •  Server Time vs. Actual Outage time


Let’s try some free tools

•  JConsole – ships with all JVMs

•  Visual VM – Exclusive to Oracle (Sun) JVM

•  Eclipse Memory Analyzer – Heap Dump Analysis

The Game Changer – AppDynamics Lite 2.0


Let’s go classic - JConsole JVM Health


Let’s go classic - JConsole Thread Dumps Mbean Browser


Let’s go classic - JConsole Mbean Browser


JConsole – Summary

•  When it Works •  JVM needs tuning

•  Get metrics for Memory, CPU, Threads and Garbage Collection •  Experiment with Memory Pools and GC Algorithms

•  Systemic slowness, thread dumps point at areas •  Server MBean and Custom JMX provides immediate clues

•  When it doesn’t •  Metrics have no contextual information so no clue why thread is

stuck on a socket ! •  You don’t know / own the source code and dev is not next to you •  It’s not a JVM tuning issue ? •  Passive, always have to be on watch •  Overhead


New Age - VisualVM How are my threads doing ?


New Age - VisualVM What are my Threads munching on? My App Classes


Visual VM– Summary

•  When it Works •  JVM Metrics overview : Memory, CPU, Threads, GC etc. •  Thread Summary View – Wait, Sleep, Monitor, Run Time •  Sampling provides some initial clues to the problems

•  Thread Sampling – Where time is spent most often •  Memory Sampling – What objects are created most often

•  When it doesn’t •  All the drawbacks of JConsole •  The profiler option (instruments all classes and methods) has way

too much overhead even for trivial apps, so not an option •  Ships only for Sun (Oracle) JVM

•  JRockit ships with a similar tool named JRA •  Nothing notable for IBM JVMs for free


Out of Memory ! - Eclipse Memory Analyzer

Leak Suspects Retained Heap Table


Eclipse Memory Analyzer - Summary •  When it works

•  You have a Heap Dump generated though –XX options •  Open up Leak Suspect Report and

•  Check out the actual content of the Object (Map <Key, Value>) •  Trace the outgoing and incoming references on the object (Who is

holding the map) •  It happens often and you know why it happens (you have some

smart engineers working with you !)

•  When it doesn’t ! •  Knowing what’s big is less important, what you need is

•  How it got big ? •  Who is adding / putting elements

•  Doesn’t happen all the time and you can’t associate with a specific event, it grows in lock step, not linear !


AD LITE 2.0 The Game Changer

AppDynamics Lite 2.0 •  Application flow map

•  Identifies key components – Servlets, Struts, EJBs •  External calls – HTTP, RMI, JMS, JDBC, THRIFT etc.

•  Top Down View of Application •  Recognizable artifacts e.g. URI vs. Class Name •  Aggregated at the top, as experienced by real user

•  Dynamic Thresholds •  Learns by itself what’s slow for Login vs. Checkout

•  Continuous visibility and Deep Dive when needed •  Every click is monitored, capture outliers – Slow, Errors, Stalls

•  Your very own dashboard – App, BT, JMX Metrics •  Always ON, alerts when things go wrong

Built for Production < 2% Overhead


Shameless Plug


http://www.appdynamics.com/products-free-download.php

When does it make sense to Pay for a Monitoring Tool?

If your app … •  Deployed on more than 5 JVMs

•  Need to see data at one place, be able to navigate easily

•  Request processing is multi threaded, distributed •  Need correlation across threads and services

•  Agile releases, Now vs. Then •  Last 5 mins ! Last Week ! Last Quarter

•  Custom Data Capture, Slicing and dicing of the data •  Capture Parameters, Select where session_id =‘’, All slow SQLs

•  Advanced Baselines and Policy Engine •  Static / Dynamic / Seasonal Baselines •  Making sense from the metrics deluge

•  Dashboards, Business Metrics •  Pre built, custom widgets, live business metrics

•  Moving towards Cloud Architecture 21 March 5, 2012 | Copyright © 2010 AppDynamics. All rights reserved. "

The Commercial Landscape

•  The Big 5s

•  IBM •  CA •  Compuware •  HP •  BMC

•  The New Age Players •  AppDynamics •  New Relic


Must haves for your Paid Tool •  No professional services

•  Plug and Play, no experts needed to configure and tune the tool

•  You don’t tell the Tool, the Tools tells you what you got •  Self adaptive learning

•  Always ON in production with very low overhead •  Not a factor of how much traffic

•  Easy to learn the tool and extend too •  Start with the basics and be an expert over time

•  Easy to share and collaborate across the Organization •  Operation, Dev, QA, Performance, Architects

•  Total Cost of Ownership – include everything •  Software, Hardware, Deployment, Learning, Maintenance

•  Continuous Innovation •  Cloud, NO SQL, Async Transactions


AppDynamics Pro

Custom Data Collection API / Scripts

Custom Action API / Scripts

Instrumentation Point API

Business Metric Collection API

Custom Header Correlation API

Controller

Application Server Agent

(JVM/CLR)

Machine Agent Alert&

Integra+on&API&/&Scripts&

Data&Feed&API&

Data&Collec+on&

API&

Cloud&Connector&

API&

How is APM Helping Organizations ?

•  A nice problem to have... •  http://techblog.netflix.com/2011/02/redesigning-netflix-api.html

•  A “Blockbuster in the Cloud” •  World’s largest Java public cloud deployment

AppDynamics in Action: Netflix

37x Growth Jan 2010-Jan 2011


1ST CLICK: THE APP TOPOLOGY


2ND CLICK: REVIEW SLOW TRANSACTIONS


3RD CLICK: TRIAGE LOCATION


4TH CLICK: FIND THE LINE OF CODE


•  E – Travel Site

•  Tomcat, Active MQ, My SQL

•  200 Business Transactions

AppDynamics in Action: Karavel, France


CUSTOM DASHBOARDS


DRILL DOWN ON SLOW TRANSACTIONS


FIND THE CODE AND THE BAD SQL


VALIDATE THE FIX !


FUTURE PROOF WITH ALERTS


Q&A

Q&A


java monitoring stuff you can get for free (and stuff you ... · java monitoring stuff you can get...

Documents