put your java apps to sleep? find out how

© Copyright Waratek 2013

Put your Java Apps to Sleep?Find out How…John Matthew Holt | JAX London 2013


John Matthew Holt

• CTO and co-Founder of Waratek• Been doing JVM engineering for 10 years• Invented “Java Virtualization” to transform how

JVMs operate in virtualized and cloud-computing environments

• Lead the design of the first Type-1 VMM for a JVM


What will we cover?

• First some background…– Why virtualize INSIDE the JVM?– Think “VMware for Java”

• Putting you Java Apps to Sleep…– Why are Java apps tired?

(or restated: why should they be slept?)– Let your apps nap: “wake-on-LAN for JVCs”


First some background…

• Why virtualize INSIDE the JVM?


Because…

• Because JVMs make bad bedfellows with HW hypervisors (HV)

• A JVM thwarts HW hypervisor features like:– The GC in a JVM thwarts memory-overcommit in a HW HV– The GC’s heap memory, and the JIT’s permgen/codecache,

thwarts ‘transparent page sharing’ (memory de-dup) in a HW HV

– Dedicated GC and JIT for EVERY Java application wastes lots and lots of CPU time and memory capacity doing the same thing over, and over, and over, and…


Today’s JVMs are OLD

• Were designed ~20 years ago, and haven’t changed design since then

• JVM design PREDATES the major 2 trends of last decade: virtualization and cloud-computing

• Only 2 innovations in JVMs in 20 years:– Better JIT compilers– Concurrent GCs– ….all blindly focused on single-tenant performance

at the expense of multi-tenant efficiency


Server Java has problems

• JVMs are grossly inefficient users of CPU and memory– A JVM assumes it can use all available CPU and memory capacity for

itself and its single application, eg:– 1 JBoss can consume up to 500MB RSS with no apps– 1 Tomcat can consume up to 250MB RSS with no webapps– Running multiple JVMs on a server rapidly exhausts physically memory

while CPU utilisation averages 5%

• >100 different JVM versions makes compliance/support a NIGHTMARE

• Security vulnerabilities and compromises are EVERYWHERE inside Java


VMWare does not virtualize Java well

• Realworld stats using VMware:– >80% of Server JVMs have <=2GB of heap memory

• Yet actual application heap utilisation averages <20%

– A 2GB JVM heap needs ~3GB of OS RSS memory, which in turn needs ~4GB of VMWare tenant memory

• …reducing actual application memory utilisation to <10%

– CPU utilisation running Java on Vmware averages <7%• Yet physical memory is exhausted because of running multiple

JVMs


Java is a compliance nightmare

• Realworld stats:– 93% of organisations are running versions of Java at

least 5 years old– The average organisation has 51 distinct versions of

Java installed– <1% of installed Java is upgraded to the latest version

• Recurring enterprise conversation:– Question: “what version of Java do you run?”– Answer: “every version except the latest version!”


Java is a security nightmare

• Realworld stats:– In last 12 months alone, >250 CVE alerts issued by US-

CERT/NIST for Java– The most prevalent version of Java has 96 “perfect 10”

vulnerabilities– “There are virtually no modern versions of Java without

any known severe vulnerabilities” (source: Bit9)• Critical Security Patch Updates are released so

often that developers and admins can’t keep up


Virtualization is the answer…

• …but HIGHER up the Server Stack• Virtualization is creeping up the Server Stack

already– x86 HW got virtualized in 1999 (VMware et al)– x86 OS got virtualized in 2006 (OpenVZ, LXC, et al)– …now it’s Java’s turn in 2013

• Virtualizing Java means virtualizing at the TOP of the Server Stack, whereas VMware virtualizes at the BOTTOM


Cost-per-tentant to zero

• The higher up virtualization goes up the Server Stack, the cheaper a “tenant” becomes

• Virtualizing INSIDE the JVM gets close to “zero-cost tenants”:– Helloworld in <1MB of tenant memory– Examples.war (Tomcat demo webapps) in <5MB of

tenant memory– …Java becomes CHEAPER than python/ruby/perl

when the JVM itself gets virtualized!


Virtualization == Security

• Virtualizing the JVM makes Server Java secure and compliant– Virtualized Java Apps run as “tenants” in a shared JVM– Tenants run in Java Virtual Container (JVC) “jails”– Tenants are isolated and “locked-down”– Tenants can run any Java version

• Doesn’t matter what “Java version” a tenant uses, it gets the latest JVM• Means a “Java version” becomes just a tenant API for the convenience

of developers• Admins control the REAL version at the JVM level so that tenants

(developers) can stop caring about “versions” altogether


First some background…

• Think “VMWare for Java”


A hypervisor INSIDE a JVM

• Waratek has written a type-1 VMM that runs INSIDE the Oracle HotSpot and OpenJDK JVMs– >300,000 lines of Java code, <5K lines of C-code– Can easily run on any OS/CPU pair (only Linux/x86 supported so far)– Does EVERYTHING VMware does, but INSIDE the JVM

• Virtual FS• Virtual networks/IPs/NICs• CPU pinning and quotas• Memory overcommit, elastic memory, memory quotas• I/O QoS, rate-limiting• Etc…

– …all with ZERO code change to Java applications


A real VMM inside the JVM


JVC

Java Code

C/C++ Code

VMWare for Java

WaratekJVM

java.* APIs

SecurityManager

Waratek Java Hypervisor

Java App

• Waratek virtualizes Java exactly like VMware virtualizes HW– Waratek inserts a “Java

Hypervisor” between the JVM and the Java App + java.* APIs + SecurityManager

– The Java app PLUS java.* APIs and SecurityManager run inside a Java Virtual Container (JVC) “jail”

– For a Java App to go outside of its JVC “jail” it has to “trap” to the Java Hypervisor

JNILibs

Process


JVC

Java Code

C/C++ Code

VMWare for Java

WaratekJVM

java.* APIs

SecurityManager

• Every incoming and outgoing access by a JVC is intercepted and checked by the Java Hypervisor, e.g:

– Class loading intercepted and checked– JNI invocation intercepted– Parameter and return value checks– File access (delete/copy/move) checks– Network operation checks– IO rate-limiting and flow-control– CPU time control– Memory consumption control– Buffer overflow checks– Null pointer checks– Type safety checks– OS access checks – JNI Libraries isolated– …


Java App

JNILibs

Process


Slots into existing VM tools

• You can manage the Java hypervisor exactly like you manage existing HW hypervisors:– SSH CLI– RedHat LibVirt/KVM integration (beta)– VMWare vCAC integration shortly– OpenStack integration shortly

• Plus other management interfaces– HTTP REST API– Local and remote JMX APIs– /proc/PID/* pseudo-filesystem for JVM and each JVC– Vmware CloudFoundry integration (beta)– RedHat OpenShift integration soon


A HelloWorld JVC in LibVirt XML


JVC

Java Code

C/C++ Code

JVCs make Java secure

WaratekJVM

java.* APIs

SecurityManager

• If a Java App inside a JVC compromises either/both of java.* APIs or SecurityManager, then only that JVC is effected and NOT the Java Hypervisor or JVM or other JVCs

• JNI Libraries are isolatable in their own OS process

• Thus a compromised JVC cannot:– Insert malicious code outside its JVC– Delete/move/copy files outside its VROOT– Trigger buffer overflows– Do null-pointer segmentation faults– Perform unsafe heap memory access– Fork OS processes– etc…


Java App

JNILibs

Process


“Come in, we’re open”

• The Java Hypervisor has an open API for unlimited customization and extensibility

• Anyone can write their own PURE-JAVA plugins to the Java hypervisor to customize ANYTHING about JVCs, eg:– Make a DropBox.com account the virtual filesystem for a JVC– Create a virtual LAN between distributed JVCs anywhere in

the world– Clone live network I/O of an active JVC to a test JVC for side-

by-side application testing


Example hypervisor plugin

• Waratek wrote ElastiCat as one example of a Java Hypervisor plugin

• ElastiCat is open source, is <1000 lines of Java code, and loads into the Java hypervisor at startup

• ElastiCat virtualizes Apache Tomcat so it becomes the world’s first multi-tenant JavaEE container:– Tomcat runs in one JVC and each webapp runs in it’s own

dedicated JVC– If one webapp crashes it won’t crash any other JVC– …all this achieved with ZERO code change to Tomcat or webapps,

and without them knowing they’ve been virtualized!


Putting Java to jSleep…

• Why are Java apps tired?(or: why should they be slept?)


Java Apps are tired!

• Realworld stats:– >50% of PaaS apps are “dead bone idle” at any

given time– The average Dev/Test environment runs at <10%

sustained utilization– Lots and lots of enterprise Java apps are only used

periodically, e.g. end-of-week, end-of-month…• But these apps have SLAs and can’t be shut-off!


Fact from the field:

• So frustrated was one Fortune500 admin that he shut-off a bunch of Java apps that had been idle for 2 weeks (no TCP traffic) and waited to see what would happen… the results were:– …a small number of app-owners noticed immediately

(1-2 days)– …the largest number of app-owners noticed within 1-2

weeks– …and a small number of app-owners NEVER noticed at

all!


Wouldn’t it be great if…

• …you could “compress” an idle app and re-awaken it instantly WITHOUT THE APP KNOWING?

• …and without the END-POINTS knowing?• …and without ever RESTARTING the app later?


Actually it exists…almost!

• The technology for this nearly already exists:– It’s called “wake-on-LAN” (WoL)

• …but there’s a problem: WoL only works at the HARDWARE level, so is of NO USE in multitenant virtualized cloud-computing environments

• Think about it:– As long as ONE tenant on a server is active then the server

HAS to stay alive and WoL CANNOT be used!


Putting Java to jSleep…

• Let your apps nap: wake-on-LAN for JVCs


“Wake-on-LAN for JVCs”

• What’s needed is a NEW kind of WoL that “naps” single idle tenants on a multitenant server…

• …and reclaims the memory/CPU/IO quotas that were provisioned for the now-napped app, to be used to run other new apps

• This is what Waratek jSleep does for Java apps!


jSleep in a Slide

• Waratek jSleep works inside the Java Hypervisor to do four things:– 1) monitor the activity of individual apps (JVCs)– 2) put idle JVCs to sleep after defined inactivity thresholds

and reclaim their CPU/mem/IO quotas– 3) monitor the I/O connections (TCP/UDP Sockets,

ServerSockets, Files, etc) of a sleeping JVC– 4) and awaken a sleeping JVC when I/O activity is

detected, reinstating reclaimed quotas

• …it’s wake-on-LAN for JVCs!


jSleep is Very Powerful

• Lots of “knobs and dials” make it possible to define sophisticated sleep/awake profiles that respond to:– CPU activity– Network/IO activity– IO timeouts– Wall-clock time– …or just about anything else

• THERE’S ALSO a built-in network profile that allows for immediate configuration via the JIRSH CLI


Simple jSleep Example

• Set a jSleep policy based on network traffic of defined sockets/ports, eg:– jirsh # sleepinfo 1 –-set network=120000,include_local=:8080,10.1.0.8:4444

– Above command:• Suspends JVC 1 after the network interface is idle 120s• The include_local param is a series of host:port combinations

that specify what traffic of remote hosts and/or local ports is allowed to wake the JVC, so in above:

– Traffic from any remote host to local port 8080 will wake the JVC– Traffic from remote host 10.1.0.8 to local port 4444 will wake the

JVC


jSleep API in 60 Seconds

• 2 packages, 8 classes:– cloudvm.jsleep.event.*

• BytesReceivedEvent• CpuEvent• NetworkIdleEvent• TimeoutEvent

– cloudvm.jsleep.profile.*• JSleepProfile• JSleepProfile.MemoryType• JSleepProfileState• NetworkProfile


jSleep API in 60 Seconds

• Main class: cloudvm.jsleep.profile.JSleepProfile– Every JVC can have one of these– Defines the two CORE methods:

• checkSuspend( VirtualContainerMXBean )• checkResume( VirtualContainerMXBean )

– Is an abstract class so anyone can write their own custom sleep/resume profile

– Loaded and configured using JMX (local or remote)


Custom jSleep API Example

• Here’s a custom jSleep profile…public boolean checkSuspend(VirtualContainerMXBean paramVirtualContainerMXBean) { Date currentDate = new Date(); if(currentDate.after(sleepDate) && currentDate.before(resumeDate)) { return true; } else { return false; }}

public boolean checkResume(VirtualContainerMXBean paramVirtualContainerMXBean) { Date currentDate = new Date(); if(currentDate.before(sleepDate) || currentDate.after(resumeDate)) { return true; } else { return false; } }


What happens when…

• …a JVC sleeps?:– The threads of the JVC are suspended on the OS kernel

(but NOT destroyed)– The JVC’s dead objects are asynchronously reclaimed by

the GC– If the JSleepProfile was configured with MemoryType.VARIABLE, the JVCs unused memory quota will be reclaimed for use by other JVCs

– All open file descriptors will be kept open– …all this in <1 second


What happens when…

• …a JVC wakes up?:– The threads of the JVC are resumed on the OS

kernel– If the JSleepProfile was configured with MemoryType.VARIABLE, the JVM will reinstate the prior memory quota of the JVC

– Any file descriptors with received but unread data will now be “readable” by the application

– …all this in <1 second


Why you should care…

• Dev/Test, PaaS, and middleware environments are characterized by LOTS of “application idleness”

• Idle apps today are treated NO DIFFERENTLY than active, busy apps

• Years ago H/W wake-on-LAN was an effective technology for powering down idle servers… but it no longer works in multitenant cloud infrastructure

• jSleep is “wake-on-LAN for JVCs” and can increase the number of Java apps-per-server by up to 10x– …this means a reduction in server sprawl and OpEx up to 90%


Summary

• Recap and links…


Biggest change to Java in yrs

• A hypervisor inside the JVM transforms Java with:– Ultra-liteweight JVCs for near zero-cost tenants– JVC “jails” for Java security and isolation– Mixed-version JVCs for version consolidation and

legacy compliance– jSleep (“Wake-On-LAN for JVCs”) for near-zero

footprint of idle tenants


Links

• General information about the Waratek JVM– www.waratek.com

• jSleep, JVI (Java Virtualization Interface), and JMX APIs– javadoc.waratek.com

• ElastiCat source code– github.com/waratek/elastic-driver

• Download ElastiCat and try it out– www.elasticat.com

http://www.waratek.com/?utm_source=Slideshare&utm_medium=JMpresentation&utm_campaign=JAXLondon

http://javadoc.waratek.com/?utm_source=Slideshare&utm_medium=JMpresentation&utm_campaign=JAXLondon

https://github.com/waratek/elasticat-driver?utm_source=Slideshare&utm_medium=JMpresentation&utm_campaign=JAXLondon

http://www.elasticat.com/?utm_source=Slideshare&utm_medium=JMpresentation&utm_campaign=JAXLondon

put your java apps to sleep? find out how

Technology

jvm copyright waratek

java copyright waratek

jvm thwarts memory

jvm design

server java

jvcs copyright waratek

jvm engineering

server jvms