virtualizing java for the cloud using a jvm hypervisor
DESCRIPTION
John Matthew Holt, founder and CTO of Waratek looks at why you should virtualize Java inside the JVM and explores case studies of virtualizing Java for the Cloud. He also looks at issues associated with performance, legacy applications and security.TRANSCRIPT
© Copyright Waratek 2013
!
Virtualizing Java for the Cloud… John Ma(hew Holt | NY JavaSIG, November 2013
© Copyright Waratek 2013
John Matthew Holt!• CTO and co-‐Founder of Waratek • Java Virtual Machine (JVM) engineer for over 10 years
• Invented “Java VirtualizaEon” to transform how JVMs operate in virtualized and cloud-‐compute environments
• Lead the design of the first Type-‐1 VMM (hypervisor) for a JVM
© Copyright Waratek 2013
What will we cover?!• First some background…
– Why virtualize INSIDE the JVM?
• Virtualizing Java for the Cloud – Think “VMware for Java” – Case study 1: Java’s (bad) legacy – Case study 2: Java’s (in)security
© Copyright Waratek 2013
First some background…!
• Why virtualize INSIDE the JVM?
© Copyright Waratek 2013
Because…!• Because JVMs make bad bedfellows with HW hypervisors (HV)
• A JVM thwarts HW hypervisor features like: – The Garbage Collector (GC) in a JVM thwarts memory-‐overcommit in a HW HV
– The GC’s heap memory, and the Just-‐In-‐Time (JIT) permgen/codecache, thwarts ‘transparent page sharing’ (memory de-‐dup) in a HW HV
– Dedicated GC and JIT for EVERY Java applicaEon wastes lots and lots of CPU Eme and memory capacity doing the same thing over, and over, and over, and …
© Copyright Waratek 2013
Todayʼs JVMs are OLD!• JVMs were designed ~20 years ago, and haven’t changed design since then
• JVM design PREDATES the major 2 trends of last decade: virtualizaEon and cloud-‐compuEng
• Only 2 innovaEons in JVMs in 20 years: – Becer JIT compilers – Concurrent GCs – ….all blindly focused on single-‐tenant performance at the expense of mulE-‐tenant efficiency
© Copyright Waratek 2013
Server Java has problems!• JVMs are grossly inefficient users of CPU and memory
– A JVM assumes it can use all available CPU and memory capacity for itself and its single applicaEon, eg:
– 1 JBoss can consume up to 500MB RSS with no apps – 1 Tomcat can consume up to 250MB RSS with no webapps – Running mulEple JVMs on a server rapidly exhausts physically
memory while CPU uElizaEon averages 5% • >100 different JVM versions makes compliance/support a
NIGHTMARE • Security vulnerabiliEes and compromises are EVERYWHERE
inside Java
© Copyright Waratek 2013
Existing virtualization does not virtualize Java well!
• Real world stats using hardware virtualizaEon: – >80% of Server JVMs have <=2GB of heap memory
• Yet actual applicaEon heap uElizaEon averages <20% • …so that means 1.6GB of heap memory per app is wasted
– A 2GB JVM heap becomes ~3GB of OS RSS memory, which in turn becomes ~4GB of hypervisor tenant memory
• …reducing actual applicaEon memory uElizaEon to <10%
– CPU uElizaEon running Java on hardware virtualizaEon averages <7% • Yet physical memory is exhausted because running mulEple JVMs has
consumed all of RAM
– …Hardware VirtualizaEon SoluEons (like Vmware, KVM, XEN) were meant to solve exactly this kind of server underuElizaEon in 2002!
• …but the problem sEll exists for Java in 2013
© Copyright Waratek 2013
Java is a compliance nightmare!• Real world stats:
– 93% of organizaEons are running versions of Java at least 5 years old
– The average organizaEon has 51 disEnct versions of Java installed
– <1% of installed Java is upgraded to the latest version • Recurring enterprise conversaEon:
– QuesEon: “what version of Java do you run?” – Answer: “every version except the latest version!”
© Copyright Waratek 2013
Java is a security nightmare!• Real world stats:
– In last 12 months, 250+ CVE alerts issued by US-‐CERT/NIST for Java
– One version of Java alone has 96 “perfect 10” vulnerabiliEes
– “There are virtually no modern versions of Java without any known severe vulnerabiliEes” (source: Bit9)
• CriEcal Security Patch Updates are released so oxen that developers and admins can’t keep up
© Copyright Waratek 2013
Virtualization is the answer…!• …but HIGHER up the Server Stack • VirtualizaEon is creeping up the Server Stack already – x86 HW got virtualized in 1999 (VMware et al) – x86 OS got virtualized in 2006 (OpenVZ, LXC, et al) – …now it’s Java’s turn in 2013
• Virtualizing Java means virtualizing at the TOP of the Server Stack, whereas exisEng hardware virtualizaEon, virtualizes at the BOTTOM
© Copyright Waratek 2013
Cost-per-tentant to zero !• The higher up virtualizaEon goes up the Server Stack, the cheaper a “tenant” becomes
• Virtualizing INSIDE the JVM gets close to “zero-‐cost tenants”: – Helloworld in <1MB of tenant memory – Examples.war (Tomcat demo webapps) in <5MB of tenant memory
– …Java becomes CHEAPER than python/ruby/perl when the JVM itself gets virtualized!
© Copyright Waratek 2013
Virtualization == Security!• Virtualizing the JVM makes Server Java SECURE and COMPLIANT – Virtualized Java Apps run as “tenants” in a shared JVM – Tenants run in isolated Java Virtual Container (JVC) “jails” – JVCs “lock-‐down” Java’s security vulnerabiliEes – JVCs can run any Java version
• Doesn’t macer what “Java version” a tenant uses, it gets the latest host JVM
• Means a “Java version” becomes just a tenant API for the convenience of developers
• Admins control the REAL version at the host JVM level so that tenants (developers) can stop caring about “versions” altogether
© Copyright Waratek 2013
Java Virtualization!
• Think “VMWare for Java”
© Copyright Waratek 2013
A hypervisor INSIDE a JVM!• Waratek has wricen a type-‐1 VMM that runs INSIDE the
Oracle HotSpot and OpenJDK JVMs – >300,000 lines of Java code, <5K lines of C-‐code – Can easily run on any OS/CPU pair (only Linux/x86 supported so
far) – Does EVERYTHING VMware does, but INSIDE the JVM
• Virtual FS • Virtual networks/IPs/NICs • CPU pinning and quotas • Memory overcommit, elasEc memory, memory quotas • I/O QoS, rate-‐limiEng • Etc…
– …all with ZERO code change to Java applicaEons
© Copyright Waratek 2013
A real VMM inside the JVM!
© Copyright Waratek 2013
JVC
Java Code
C/C++ Code
VMWare for Java!
Waratek JVM
java.* APIs
SecurityManager
Waratek Java Hypervisor
Java App
• Waratek virtualizes Java exactly like VMware virtualizes HW – Waratek inserts a “Java
Hypervisor” between the JVM and the Java App + java.* APIs + SecurityManager
– The Java app PLUS java.* APIs and SecurityManager run inside a Java Virtual Container (JVC) “jail”
– For a Java App to go outside of its JVC “jail” it has to “trap” to the Java Hypervisor
JNI Libs
Process
© Copyright Waratek 2013
JVC
Java Code
C/C++ Code
VMWare for Java!
Waratek JVM
java.* APIs
SecurityManager
• Every incoming and outgoing access by a JVC is intercepted and checked by the Java Hypervisor, e.g: – Class loading intercepted and checked – JNI invocaEon intercepted – Parameter and return value checks – File access (delete/copy/move) checks – Network operaEon checks – IO rate-‐limiEng and flow-‐control – CPU Eme control – Memory consumpEon control – Buffer overflow checks – Null pointer checks – Type safety checks – OS access checks – JNI Libraries isolated – …
Waratek Java Hypervisor
Java App
JNI Libs
Process
© Copyright Waratek 2013
Slots into existing VM tools!• You can manage the Java hypervisor exactly like you manage
exisEng HW hypervisors: – SSH CLI – RedHat LibVirt/virsh integraEon (beta) – VMWare vCAC integraEon shortly – OpenStack integraEon shortly
• Plus other management interfaces – HTTP REST API – Local and remote JMX APIs – /proc/PID/* pseudo-‐filesystem for JVM and each JVC – Vmware CloudFoundry integraEon (beta) – RedHat OpenShix integraEon soon
© Copyright Waratek 2013
A HelloWorld JVC in LibVirt XML!
© Copyright Waratek 2013
“Come in, weʼre open”!• The Java Hypervisor has an open API for unlimited customizaEon and extensibility
• Anyone can write their own PURE-‐JAVA plugins to the Java hypervisor to change ANYTHING about JVCs, eg: – Make a DropBox.com account the virtual filesystem for a JVC
– Create a virtual LAN between distributed JVCs anywhere in the world
– Clone live network I/O of an acEve JVC to a test JVC for side-‐by-‐side applicaEon tesEng
© Copyright Waratek 2013
Example hypervisor plugin!• Waratek wrote ElasECat as one example of a Java Hypervisor plugin
• ElasECat is open source, is <1000 lines of Java code, and loads into the Java hypervisor at startup
• ElasECat virtualizes Apache Tomcat so it becomes the world’s first mulE-‐tenant JavaEE container: – Tomcat runs in one JVC and each webapp runs in it’s own dedicated JVC
– If one webapp crashes it won’t crash any other webapp – …all this achieved with ZERO code change to Tomcat or webapps, and without them knowing they’ve been virtualized!
© Copyright Waratek 2013
© Copyright Waratek 2013
Java Virtualization!
• Case study 1: Java’s (bad) legacy
© Copyright Waratek 2013
Java (non)compliance!• Real world stats:
– 93% of organizaEons are running versions of Java at least 5 years old
– The average organizaEon has 51 disEnct versions of Java installed
– <1% of installed Java is upgraded to the latest version • Recurring enterprise conversaEon:
– QuesEon: “what version of Java do you run?” – Answer: “every version except the latest version!”
© Copyright Waratek 2013
Case Study 1 Profile!• A Global InsEtuEon • Approx 40,000 Java apps on approx 20,000 Linux Servers – Average app-‐to-‐server raEo of 2:1 – Approx 25,000 apps running Java SE 6 – Approx 10,000 apps running Java SE 5 – Approx 3,000 apps running Java SE 4 – Less than 40 apps running Java SE 7 – 2 apps sEll running Java 1.0.0 – … this is NOT unusual!
© Copyright Waratek 2013
How old is that old app?!• Java 7 is already 2 years old! • Java 6 is 7 years old! • Java 5 is 9 nears old! • Java 4 is 11 years old! • …that old app you wrote 5 years ago could sEll be running somewhere!
• …and that NEW app you’re wriEng NOW could sEll be running in 5 years Eme!
© Copyright Waratek 2013
Case Study: Before Waratek!• With <40 apps on Java 7, 99.9% of their Java estate is unsupported and EOL – Those Java CriEcal Patch Updates are precy criEcal axer all
• Before adopEng Waratek, they were going to pay external consultants to migrate all of their legacy apps to Java 7 at an enormous cost-‐per-‐app – And it would have taken 1 year or more
© Copyright Waratek 2013
Why did they care?!• Here’s one really simply example why:
– Did you know that Eme zones in the world are not staEc and change frequently?
– In general Eme zone updates can be released 4-‐6 Emes a year!
• So if you don’t upgrade your old JVMs then you’re running a broken API – …this can be a Ecking Eme-‐bomb (no pun intended) which only gets discovered when something goes really wrong and it’s too late!
© Copyright Waratek 2013
Hereʼs another reason…!• Every version of Java has “perfect 10” security vulnerabiliEes
• One version of Java 6 alone has 96 “perfect 10” vulnerabiliEes from US-‐CERT/NIST!
• Those old apps your running have a whole host of security vulnerabiliEes RIGHT NOW:
• Remote code injecEon and inclusion • DoS • Buffer overflows and trigger-‐able SIGSEGV faults
© Copyright Waratek 2013
Java legacy is EXPENSIVE!• The cost of supporEng old soxware increases over Eme… – …and so does the risk and damages of that old soxware failing
• The real costs are hidden though -‐ an old Java app is the Ep of a very expensive iceberg: – That old Java app is running on an old JVM – That old JVM is running on an old OS – That old OS is running on an old server/CPU – …so the legacy cost is 3x more than JUST the app!
© Copyright Waratek 2013
Case Study: With Waratek!• With Waratek, hundreds of old apps can be migrated to an
UP-‐TO-‐DATE JVM in hours not years – Skips/lowers tesEng requirements because the Java API hasn’t
changed – Things like serializaEon/persistence keep running flawlessly
because version API is unchanged
• Every app keeps running with the Java API it was tested with – Means Java versions become a programming convenience, not
an administraEve nightmare – Whatever version of Java you boot in a JVC, the host JVM is
brand new and shiny and up-‐to-‐date and supported
© Copyright Waratek 2013
Old Java in JVCs == savings!• The more old apps, and the older those apps:
– The greater the cost of app-‐failure – The greater the operaEng and support costs
• Ge}ng old apps off old JVMs and onto JVCs saves lots of $$$ because – Much smaller footprint (tenant cost) – Much faster performance – Secure and quaranEned – Compliant and supported – Legacy OS/hardware can be reEred
© Copyright Waratek 2013
In a customerʼs words!
• “[With Waratek] applicaKons on older versions of Java can be immediately migrated to a secure, resource-‐contained and isolated JVC without any code changes or applicaKon transformaKon”
© Copyright Waratek 2013
Case Studies!
• Case Study 2: Java (in)security
© Copyright Waratek 2013
Java (in)security!• Real world stats:
– In last 12 months 250+ CVE alerts issued by US-‐CERT/NIST for Java
– One version of Java alone has 96 “perfect 10” vulnerabiliEes
– “There are virtually no modern versions of Java without any known severe vulnerabiliEes” (source: Bit9)
• CriEcal Security Patch Updates are released so oxen that developers and admins can’t keep up
© Copyright Waratek 2013
Did you know?!• Server Java has the same root security vulnerabiliEes as desktop Java
• …but acack vectors are slightly different: – Remote and local code-‐injecEon and inclusion – DoS – Buffer overflows/segfaults – Logic bombs / Eme bombs / trojans – …and biggest of all: employee/consultant sabotage!!
© Copyright Waratek 2013
Java Code
C/C++ Code
Java is insecure at multiple levels!
JVM JNI Libs
java.* APIs
SecurityManager
Java App
• Java’s security vulnerabiliEes fall into 3 acack vectors: – Acacks to take control of java.*
APIs • E.g. to delete/copy/move files
– Acacks to take control of Java’s SecurityManager
• E.g. insert malicious code anywhere inside the JVM
– Acacks on JNI Libraries • E.g. trigger buffer overflows, null-‐pointer
segmentaEon faults, unsafe type access, and so on
© Copyright Waratek 2013
Java Code
C/C++ Code
JVMs are insecure by design!
JVM JNI Libs
java.* APIs
SecurityManager
• By Java’s design, a JVM performs no security checks of its own: – The JVM relies on security
enforcement in the java.* API and SecurityManager levels
– If the SecurityManager or java.* APIs are compromised, then the JVM can be instructed to do anything like:
• Insert malicious code • Load C/C++ JNI Libraries • Fork OS processes • Write directly to heap memory • Open network connecEons • And anything else…
Java App
© Copyright Waratek 2013
Unsafe JARs == unsafe JVM!• It’s not just the Java APIs that you need to worry about • Unlike the Java APIs, security vulnerabiliEes in third-‐party
class libraries (JARs) are poorly researched and documented and are EVERYWHERE – Some isolated examples: code-‐injecEon vulnerabiliEes exist in all
of Spring, JIRA, Struts, Webwork2, RMI, JBoss, WebLogic, and many more
• Third-‐party JARs are: – full of security vulnerabiliEes – very poorly audited/tested/understood – not patched frequently enough
© Copyright Waratek 2013
Unsafe Java == unsafe OS!• A compromised JVM becomes an acacker’s puppet, and can
be made to do things like: – Give valuable intelligence about OS, network, and neighbors – “Poison” JAR files on disk, inserEng Trojans – Hide “logic bombs” and “Eme bombs” that explode when an
employee is fired, some anniversary passes, etc – Change startup params for the JVM, like open remote RMI access – Fork new OS processes – Manipulate file descriptors in unintended ways – Launch remote or local DoS acacks – Hijack servers for malicious “bot” purposes
© Copyright Waratek 2013
Case Study 2 Profile!• A Fortune 500 company • Running a DIY “Java PaaS” pla�orm
– Many JVMs per OS – …with poor isolaEon between each JVM – CriEcal Security Patch Updates for Java are released so oxen, they can’t keep up
• In the customer’s words: – “[We have many] security vulnerabiliKes that leave us open to a(ack across the estate”
© Copyright Waratek 2013
Many apps, little oversight!• With so many apps there’s limited oversight for each
individual app • No fine-‐grained tenant control because many JVMs are
packed into a single VMware guest OS – …so VMware’s can’t help them this high up the SW stack
• Their nightmare today: any app could have a trojan and nobody knows
• Biggest problem: the JVM itself provides no security, and once compromised, a JVM is an acacker’s puppet
© Copyright Waratek 2013
Customerʼs wish!• Capability to define per-‐app quaranEne rules AND able to be
set by the app-‐owner DIRECTLY and IMMEDIATELY • …but not possible today because:
– ExisEng JVMs don’t have any means to define applicaEon permissions outside of the (highly-‐compromised) Java API
– …because of this, per-‐app security rules must be defined at the underlying OS or hypervisor or network Eers
– …but the OS/hypervisor/network Eers are managed by different teams/departments from the app-‐owners
– An app-‐owner wanEng to apply or change an app’s security policy has to go to the OS/net/VMM teams who are backlogged doing other things and are slow, distracted and unresponsive
© Copyright Waratek 2013
JVC
Java Code
C/C++ Code
Case Study: with Waratek!
Waratek JVM
java.* APIs
SecurityManager
• By running Java apps inside of JVCs, compromises to a Java app or Java.* APIs will NOT effect the OS or Java Hypervisor or JVM or other JVCs
• JNI Libraries are isolatable in their own OS process
• Thus a compromised app remains under quaranEne and control:
– Cannot insert malicious code outside its JVC – Cannot delete/move/copy files outside its VROOT – Cannot trigger buffer overflows – Cannot do null-‐pointer segmentaEon faults – Cannot perform unsafe heap memory access – Cannot fork OS processes – etc…
Waratek Java Hypervisor
Java App
JNI Libs
Process
© Copyright Waratek 2013
JVCs quarantine Java!• Waratek Java hypervisor can “lock down” Java API acack vectors regardless
of the Java API version running in a JVC, a few examples: – Block class defines by a JVC that does NOT originate from a local registered JAR file – Hide sensiEve OS/network/environment info from a JVC – Do payload scanning of all inbound network IO for a JVC – …and much more
• Means that even a JVC running an old Java API version can sEll be up-‐to-‐date and secured
• Most important of all... the app-‐owner can set and change security policies DIRECTLY and IMMEDIATELY without bothering the OS or infrastructure or network teams/departments: – Can set fine-‐grained per-‐app network rules – Can set fine-‐grained per-‐app CPU/memory/IO quota rules – Can set fine-‐grained per-‐app classloading rules, file rules, …anything!
© Copyright Waratek 2013
In a customerʼs words!
• “[With Waratek] applicaKons can be quaranKned without need of code changes or transformaKon”
• “[Using Waratek] the immediate risks posed by these applicaKons can be miKgated and a route for the upgrade and future sustained management of the applicaKons can be established”
© Copyright Waratek 2013
Summary!
• Recap and links…
© Copyright Waratek 2013
Biggest change to Java in yrs!• A hypervisor inside the JVM transforms Java with: – Ultra-‐lightweight JVCs for near zero-‐cost tenants – Mixed-‐version JVCs for version consolidaEon, legacy compliance, and “virtual upgrades”
– JVC “jails” for Java security, containment and quaranEne
© Copyright Waratek 2013
Links!• General informaEon about the Waratek JVM
– hcp://www.waratek.com
• jSleep, JVI (Java VirtualizaEon Interface), and JMX APIs – hcp://javadoc.waratek.com
• ElasECat source code – hcps://github.com/waratek/elasEcat-‐driver
NOTICE: Waratek, Cloud VM, jSleep, vSleep, jMotion, Jirsh, JVC, JVI, and ElastiCat are TRADEMARKS of Waratek
© Copyright Waratek 2013
!