put your java apps to sleep? find out how - john matthew holt (waratek)

© Copyright Waratek 2013

!

Put your Java Apps to Sleep? Find out How… John Ma(hew Holt | JAX London 2013


John Matthew Holt!•  CTO and co-‐Founder of Waratek •  Been doing JVM engineering for 10 years •  Invented “Java VirtualizaCon” to transform how JVMs operate in virtualized and cloud-‐compuCng environments

•  Lead the design of the first Type-‐1 VMM for a JVM


What will we cover?!•  First some background…

– Why virtualize INSIDE the JVM? – Think “VMware for Java”

•  PuSng you Java Apps to Sleep… – Why are Java apps Cred? (or restated: why should they be slept?)

– Let your apps nap: “wake-‐on-‐LAN for JVCs”


First some background…!

• Why virtualize INSIDE the JVM?


Because…!•  Because JVMs make bad bedfellows with HW hypervisors (HV)

•  A JVM thwarts HW hypervisor features like: –  The GC in a JVM thwarts memory-‐overcommit in a HW HV

–  The GC’s heap memory, and the JIT’s permgen/codecache, thwarts ‘transparent page sharing’ (memory de-‐dup) in a HW HV

– Dedicated GC and JIT for EVERY Java applicaCon wastes lots and lots of CPU Cme and memory capacity doing the same thing over, and over, and over, and…


Today’s JVMs are OLD!•  Were designed ~20 years ago, and haven’t changed design since then

•  JVM design PREDATES the major 2 trends of last decade: virtualizaCon and cloud-‐compuCng

•  Only 2 innovaCons in JVMs in 20 years: – Beder JIT compilers – Concurrent GCs – ….all blindly focused on single-‐tenant performance at the expense of mulC-‐tenant efficiency


Server Java has problems!•  JVMs are grossly inefficient users of CPU and memory

–  A JVM assumes it can use all available CPU and memory capacity for itself and its single applicaCon, eg:

–  1 JBoss can consume up to 500MB RSS with no apps –  1 Tomcat can consume up to 250MB RSS with no webapps –  Running mulCple JVMs on a server rapidly exhausts physically

memory while CPU uClisaCon averages 5% •  >100 different JVM versions makes compliance/support a

NIGHTMARE •  Security vulnerabiliCes and compromises are EVERYWHERE

inside Java


VMWare does not virtualize Java well!

•  Realworld stats using VMware: –  >80% of Server JVMs have <=2GB of heap memory

•  Yet actual applicaCon heap uClisaCon averages <20% – A 2GB JVM heap needs ~3GB of OS RSS memory, which in turn needs ~4GB of VMWare tenant memory •  …reducing actual applicaCon memory uClisaCon to <10%

–  CPU uClisaCon running Java on Vmware averages <7% •  Yet physical memory is exhausted because of running mulCple JVMs


Java is a compliance nightmare!•  Realworld stats:

–  93% of organisaCons are running versions of Java at least 5 years old

–  The average organisaCon has 51 disCnct versions of Java installed

–  <1% of installed Java is upgraded to the latest version •  Recurring enterprise conversaCon:

– QuesCon: “what version of Java do you run?” – Answer: “every version except the latest version!”


Java is a security nightmare!•  Realworld stats:

–  In last 12 months alone, >250 CVE alerts issued by US-‐CERT/NIST for Java

–  The most prevalent version of Java has 96 “perfect 10” vulnerabiliCes

–  “There are virtually no modern versions of Java without any known severe vulnerabiliCes” (source: Bit9)

•  CriCcal Security Patch Updates are released so oven that developers and admins can’t keep up


Virtualization is the answer…!•  …but HIGHER up the Server Stack •  VirtualizaCon is creeping up the Server Stack already –  x86 HW got virtualized in 1999 (VMware et al) –  x86 OS got virtualized in 2006 (OpenVZ, LXC, et al) – …now it’s Java’s turn in 2013

•  Virtualizing Java means virtualizing at the TOP of the Server Stack, whereas VMware virtualizes at the BOTTOM


Cost-per-tentant to zero !•  The higher up virtualizaCon goes up the Server Stack, the cheaper a “tenant” becomes

•  Virtualizing INSIDE the JVM gets close to “zero-‐cost tenants”: – Helloworld in <1MB of tenant memory – Examples.war (Tomcat demo webapps) in <5MB of tenant memory

– …Java becomes CHEAPER than python/ruby/perl when the JVM itself gets virtualized!


Virtualization == Security!•  Virtualizing the JVM makes Server Java secure and compliant –  Virtualized Java Apps run as “tenants” in a shared JVM –  Tenants run in Java Virtual Container (JVC) “jails” –  Tenants are isolated and “locked-‐down” –  Tenants can run any Java version

•  Doesn’t mader what “Java version” a tenant uses, it gets the latest JVM

•  Means a “Java version” becomes just a tenant API for the convenience of developers

•  Admins control the REAL version at the JVM level so that tenants (developers) can stop caring about “versions” altogether


First some background…!

• Think “VMWare for Java”


A hypervisor INSIDE a JVM!•  Waratek has wriden a type-‐1 VMM that runs INSIDE the

Oracle HotSpot and OpenJDK JVMs –  >300,000 lines of Java code, <5K lines of C-‐code –  Can easily run on any OS/CPU pair (only Linux/x86 supported so

far) –  Does EVERYTHING VMware does, but INSIDE the JVM

•  Virtual FS •  Virtual networks/IPs/NICs •  CPU pinning and quotas •  Memory overcommit, elasCc memory, memory quotas •  I/O QoS, rate-‐limiCng •  Etc…

–  …all with ZERO code change to Java applicaCons


A real VMM inside the JVM!


JVC

Java Code

C/C++ Code

VMWare for Java!

Waratek JVM

java.* APIs

SecurityManager

Waratek Java Hypervisor

Java App

•  Waratek virtualizes Java exactly like VMware virtualizes HW –  Waratek inserts a “Java

Hypervisor” between the JVM and the Java App + java.* APIs + SecurityManager

–  The Java app PLUS java.* APIs and SecurityManager run inside a Java Virtual Container (JVC) “jail”

–  For a Java App to go outside of its JVC “jail” it has to “trap” to the Java Hypervisor

JNI Libs

Process


JVC

Java Code

C/C++ Code

VMWare for Java!

Waratek JVM

java.* APIs

SecurityManager

•  Every incoming and outgoing access by a JVC is intercepted and checked by the Java Hypervisor, e.g: –  Class loading intercepted and checked –  JNI invocaCon intercepted –  Parameter and return value checks –  File access (delete/copy/move) checks –  Network operaCon checks –  IO rate-‐limiCng and flow-‐control –  CPU Cme control –  Memory consumpCon control –  Buffer overflow checks –  Null pointer checks –  Type safety checks –  OS access checks –  JNI Libraries isolated –  …


Java App

JNI Libs

Process


Slots into existing VM tools!•  You can manage the Java hypervisor exactly like you manage

exisCng HW hypervisors: –  SSH CLI –  RedHat LibVirt/KVM integraCon (beta) –  VMWare vCAC integraCon shortly –  OpenStack integraCon shortly

•  Plus other management interfaces –  HTTP REST API –  Local and remote JMX APIs –  /proc/PID/* pseudo-‐filesystem for JVM and each JVC –  Vmware CloudFoundry integraCon (beta) –  RedHat OpenShiv integraCon soon


A HelloWorld JVC in LibVirt XML!


JVC

Java Code

C/C++ Code

JVCs make Java secure!

Waratek JVM

java.* APIs

SecurityManager

•  If a Java App inside a JVC compromises either/both of java.* APIs or SecurityManager, then only that JVC is effected and NOT the Java Hypervisor or JVM or other JVCs

•  JNI Libraries are isolatable in their own OS process

•  Thus a compromised JVC cannot:

–  Insert malicious code outside its JVC –  Delete/move/copy files outside its VROOT –  Trigger buffer overflows –  Do null-‐pointer segmentaCon faults –  Perform unsafe heap memory access –  Fork OS processes –  etc…


Java App

JNI Libs

Process


“Come in, we’re open”!•  The Java Hypervisor has an open API for unlimited customizaCon and extensibility

•  Anyone can write their own PURE-‐JAVA plugins to the Java hypervisor to customize ANYTHING about JVCs, eg: – Make a DropBox.com account the virtual filesystem for a JVC

–  Create a virtual LAN between distributed JVCs anywhere in the world

–  Clone live network I/O of an acCve JVC to a test JVC for side-‐by-‐side applicaCon tesCng


Example hypervisor plugin!•  Waratek wrote ElasCcat as one example of a Java Hypervisor plugin

•  ElasCcat is open source, is <1000 lines of Java code, and loads into the Java hypervisor at startup

•  ElasCcat virtualizes Apache Tomcat so it becomes the world’s first mulC-‐tenant JavaEE container: –  Tomcat runs in one JVC and each webapp runs in it’s own dedicated JVC

–  If one webapp crashes it won’t crash any other JVC –  …all this achieved with ZERO code change to Tomcat or webapps, and without them knowing they’ve been virtualized!


Putting Java to jSleep…!

• Why are Java apps Cred? (or: why should they be slept?)


Java Apps are tired!!•  Realworld stats:

– >50% of PaaS apps are “dead bone idle” at any given Cme

– The average Dev/Test environment runs at <10% sustained uClizaCon

– Lots and lots of enterprise Java apps are only used periodically, e.g. end-‐of-‐week, end-‐of-‐month…

•  But these apps have SLAs and can’t be shut-‐off!


Fact from the field:!•  So frustrated was one Fortune500 admin that he shut-‐off a bunch of Java apps that had been idle for 2 weeks (no TCP traffic) and waited to see what would happen… the results were: – …a small number of app-‐owners noCced immediately (1-‐2 days)

– …the largest number of app-‐owners noCced within 1-‐2 weeks

– …and a small number of app-‐owners NEVER noCced at all!


Wouldn’t it be great if…!•  …you could “compress” an idle app and re-‐awaken it instantly WITHOUT THE APP KNOWING?

•  …and without the END-‐POINTS knowing? •  …and without ever RESTARTING the app later?


Actually it exists…almost!!•  The technology for this nearly already exists:

–  It’s called “wake-‐on-‐LAN” (WoL)

•  …but there’s a problem: WoL only works at the HARDWARE level, so is of NO USE in mulCtenant virtualized cloud-‐compuCng environments

•  Think about it: –  As long as ONE tenant on a server is acCve then the server HAS to stay alive and WoL CANNOT be used!


Putting Java to jSleep…!

• Let your apps nap: wake-‐on-‐LAN for JVCs


“Wake-on-LAN for JVCs”!•  What’s needed is a NEW kind of WoL that “naps” single idle tenants on a mulCtenant server…

•  …and reclaims the memory/CPU/IO quotas that were provisioned for the now-‐napped app, to be used to run other new apps

•  This is what Waratek jSleep does for Java apps!


jSleep in a Slide!•  Waratek jSleep works inside the Java Hypervisor to do four things: –  1) monitor the acCvity of individual apps (JVCs) –  2) put idle JVCs to sleep aver defined inacCvity thresholds and reclaim their CPU/mem/IO quotas

–  3) monitor the I/O connecCons (TCP/UDP Sockets, ServerSockets, Files, etc) of a sleeping JVC

–  4) and awaken a sleeping JVC when I/O acCvity is detected, reinstaCng reclaimed quotas

•  …it’s wake-‐on-‐LAN for JVCs!


jSleep is Very Powerful!•  Lots of “knobs and dials” make it possible to define sophisCcated sleep/awake profiles that respond to: –  CPU acCvity –  Network/IO acCvity –  IO Cmeouts – Wall-‐clock Cme –  …or just about anything else

•  THERE’S ALSO a built-‐in network profile that allows for immediate configuraCon via the JIRSH CLI


Simple jSleep Example!•  Set a jSleep policy based on network traffic of defined

sockets/ports, eg: –  jirsh # sleepinfo 1 –-set network=120000,include_local=:8080,10.1.0.8:4444

–  Above command: •  Suspends JVC 1 aver the network interface is idle 120s •  The include_local param is a series of host:port combinaCons that specify what traffic of remote hosts and/or local ports is allowed to wake the JVC, so in above:

–  Traffic from any remote host to local port 8080 will wake the JVC –  Traffic from remote host 10.1.0.8 to local port 4444 will wake the JVC


jSleep API in 60 Seconds!•  2 packages, 8 classes:

–  cloudvm.jsleep.event.* •  BytesReceivedEvent •  CpuEvent •  NetworkIdleEvent •  TimeoutEvent

–  cloudvm.jsleep.profile.* •  JSleepProfile •  JSleepProfile.MemoryType •  JSleepProfileState •  NetworkProfile


jSleep API in 60 Seconds!•  Main class: cloudvm.jsleep.profile.JSleepProfile

–  Every JVC can have one of these –  Defines the two CORE methods:

•  checkSuspend( VirtualContainerMXBean ) •  checkResume( VirtualContainerMXBean )

–  Is an abstract class so anyone can write their own custom sleep/resume profile

–  Loaded and configured using JMX (local or remote)


Custom jSleep API Example!•  Here’s a custom jSleep profile…

public boolean checkSuspend(VirtualContainerMXBean paramVirtualContainerMXBean) { Date currentDate = new Date(); if(currentDate.aver(sleepDate) && currentDate.before(resumeDate)) { return true; } else { return false; } } public boolean checkResume(VirtualContainerMXBean paramVirtualContainerMXBean) { Date currentDate = new Date(); if(currentDate.before(sleepDate) || currentDate.aver(resumeDate)) { return true; } else { return false; } }


What happens when…!•  …a JVC sleeps?:

–  The threads of the JVC are suspended on the OS kernel (but NOT destroyed)

–  The JVC’s dead objects are asynchronously reclaimed by the GC

–  If the JSleepProfile was configured with MemoryType.VARIABLE, the JVCs unused memory quota will be reclaimed for use by other JVCs

–  All open file descriptors will be kept open –  …all this in <1 second


What happens when…!•  …a JVC wakes up?:

– The threads of the JVC are resumed on the OS kernel

–  If the JSleepProfile was configured with MemoryType.VARIABLE, the JVM will reinstate the prior memory quota of the JVC

– Any file descriptors with received but unread data will now be “readable” by the applicaCon

– …all this in <1 second


Why you should care…!•  Dev/Test, PaaS, and middleware environments are

characterized by LOTS of “applicaCon idleness” •  Idle apps today are treated NO DIFFERENTLY than acCve,

busy apps •  Years ago H/W wake-‐on-‐LAN was an effecCve technology for

powering down idle servers… but it no longer works in mulCtenant cloud infrastructure

•  jSleep is “wake-‐on-‐LAN for JVCs” and can increase the number of Java apps-‐per-‐server by up to 10x –  …this means a reducCon in server sprawl and OpEx up to 90%


Summary!

• Recap and links…


Biggest change to Java in yrs!•  A hypervisor inside the JVM transforms Java with: – Ultra-‐liteweight JVCs for near zero-‐cost tenants – JVC “jails” for Java security and isolaCon – Mixed-‐version JVCs for version consolidaCon and legacy compliance

–  jSleep (“Wake-‐On-‐LAN for JVCs”) for near-‐zero footprint of idle tenants


Links!•  General informaCon about the Waratek JVM

– hdp://www.waratek.com

•  jSleep, JVI (Java VirtualizaCon Interface), and JMX APIs – hdp://javadoc.waratek.com

•  ElasCcat source code – hdps://github.com/waratek/elasCcat-‐driver


!

put your java apps to sleep? find out how - john matthew holt (waratek)

Technology

lanforjvcs copyright

vmmfora jvm copyright

server java

andthejitspermgen codecache

john matthew holt

todays jvms