elastic memory for java

Consolidation Matters: The App Explosion Is Upon Us.1

991 Linus Torvalds

Releases Linux

1990 2011

200

7 iPhoneLaunch

Linux: 40,000 apps.

Mobile: 500,000 apps.

What does this have to do with virtualization?

More and more applications are moving to the datacenter.(Think Google Docs)

In Other Words:

The next wave of computing will produce 10x more applications than the last wave.

Most of these will need cloud back-ends.

Your choice:

Get ready to buy lots more servers to support 10x more VMs.

Or raise consolidation ratios higher than ever.

What does this have to do with Java?

“The data collection tier was originally written in Ruby,but was eventually ported over to Java.

The primary driver for this change was performance.This tier currently supports over 180k requests per

minute and responds in around 2.5 millisecondswith plenty of headroom to go.”

-- New Relic

Think Java is the Cobol of the 1990s? Think again.

Many SaaS and mobile backends will select Java.

Introducing Elastic Memory for Java.

Elastic Memory for Java is a balloon driver that runs inside vFabric tc Server.

Elastic Memory for Java lets VMware ESX share memory between VMs running Java.

Two benefits:

• Java Memory Overcommit On ESX, you can allocate more memory to VMs than physically available to ESX.

This means higher consolidation ratios and less hardware to buy, power and cool.

• Java Memory Reservoirs Give Java some extra cushion to protect against workload spikes.

Memory your application doesn’t need is simply shared. No waste.

ESXi: The Industry’s Most Advanced Memory Sharing.

ESXi has many memory sharing techniques:

• Transparent Page Sharing.

• OS Balloon.

• Memory Compression.

• Host Swapping.

• New with ESXi 5: Elastic Memory for Java.

The most advanced memory sharing, the industry’s best consolidation ratios.

Sharing Memory Impacts Performance.

Less Application Impact More Application Impact

TransparentPage

Sharing

GuestBalloon

MemoryCompression

HostSwapping

How ESX Page Sharing Works.

ESX slowly scans memory looking for identical pages.Identical pages are “factored out”.

Works well when many VMs run the same OS or app.Can take weeks to converge.

ESX Ballooning via VMware Tools.

The guest balloon inflates and takes memory pages from the OS.The pages are pinned into memory so the OS cannot reclaim them.The balloon tells ESX the addresses so these pages can be reassigned.

Host Swapping and Memory Compression.

Case (a) + (b): ESX writes memory pages to disk and reuses them.Case (c): Pages are compressed into a compression cache.ESX has to undo the operation when the page is read or written.

Sharing Memory Impacts Performance.

Less Application Impact More Application Impact

TransparentPage

Sharing

GuestBalloon

MemoryCompression

HostSwapping

Who Decides What Memory To Share?

Less Application Impact More Application Impact

TransparentPage

Sharing

GuestBalloon

MemoryCompression

HostSwapping

OS ESX

OS LRU ESX Working Set Estimate

The OS understands memory needs better than ESX.It can make better decisions memory to give up.

But the OS has no understanding of Java memory.

VM3VM2

VM1

Available ESX Memory

Usage Spikein VM2

Usage Spikein VM3

Danger!Low Memory!

Balloon

How Ballooning Works

14 Confidential

Tenured Space

Eden Young

Guest balloon can cause garbage collection pauses.

JavaHeap

GuestBalloon

OS swapspages thatseem to be

unused

Memorypressure

Garbagecollectionswaps it

all back inat great

cost

Why swapping objects in hurts so much.

Disk seeks are 100,000 times slower than memory reads!(10,000,000 ns = 10 ms)

When using shared storage latency may be even higher.

Latencies approaching 100ms are possible.Source: Scalable Storage Performance, VMware Whitepaper

Java on ESX: Today’s Best Practices.

Today’s Java Best Practices Summarized.

# Best Practice

1 Size heap memory to fit within the VM.

2 Enable large pages in the OS and JVM.

3 Guarantee the VM always has 100% of its memory.

4 Match Java threads and virtual CPUs.

5 Ensure disk latencies are low.

6 Watch out for timekeeping impact.

Java Can Exhibit “Catastrophic Collapse” When Swapped.

Source: IBM Whitepaper

How Swap Kills Java Garbage Collection.

Situation Pause Time

In Memory 0.02s

Local Storage (10ms) 1m 30s

Slow Shared Storage (100ms) 14m

Assumption Value

Total Memory in the Heap 1 GB

Average Object Size 5000 bytes

Total Number of Objects 200,000

% Swapped Out 20%

% Objects Read Per Access 5%

Total Unique Accesses 8,000

Sharing Memory Impacts Performance.

Less Application Impact More Application Impact

TransparentPage

Sharing

GuestBalloon

MemoryCompression

HostSwapping

Introducing Elastic Memory for Java.

Less Application Impact More Application Impact

TransparentPage

Sharing

GuestBalloon

MemoryCompression

HostSwapping

OS ESX

Elastic Memoryfor JavaBalloon

Inside the Java heap.Able to intelligently optimize Java memory.

23 Confidential

Tenured Space

Eden Young

EM4J Works Directly In The Java Heap

JavaHeap

EM4JBalloon

EM4J reservesheap spaceand givesit back ifneeded

No expensivedisk seeks

24 Confidential

Tenured Space

Eden Young

EM4J Can Clean Garbage From The Heap

JavaHeap

EM4JBalloon

If memory isvery low,

EM4J mayneed to doeven more.

Experimental Results Using Modified Spring Travel

Spring Travel simulates a hotel booking site. Modified to have more virtual hotels. 500 simulated users, simultaneous access.

0 15 250%1%2%3%4%5%6%7%8%

Overhead as Function of Overcommit %

EM4J

Experimental Results Using

vFabric GemFire: A Java-based in-memory database. Test Environment: Multiple VMs running a GemFire app, load

applied to one VM at a time. At 15% overcommit, almost no overhead with EM4J. Overhead grows predictably compared to guest balloon.

15 25 30 350%

100%

200%

300%

400%

500%

Overhead as Function of Overcommit %

GuestEM4J

Pieces That Make Elastic Memory for Java Work.

Balloon Driver (Java Agent)

ESX 5+ Virtual API Interface

ESXESX Resource Management

tc Server

VMware Tools

Your Java App

FeedbackLoop

DeterminesBalloonTarget

What Is tc Server Anyway?

Efficient, lean, fit-to-purpose Java application server

Lower cost and complexity

Enterprise capabilities on Tomcat-compatible base

Best platform to run Spring applications

Includes Spring Insight and Elastic Memory for Java

Enabling EM4J At The VM.

Enabling EM4J Within tc Server.

Step 1: Create a tc Server instance with the elastic-memory template.

Step 2: Add your application WAR file to the template.

Consolidation In The vRAM Era.

Utilizations remain low, even after consolidation.

The average ESX host uses only half of its memory.

The average VM has 42% more memory than it needs.

Source: VMware Customer Survey

Overcommit and Run Fewer Physical Servers.

AssumptionsServer Cost CapEX ($) $10,000 per serverServer Cost OpEX ($) $2,000 per server per yearSoftware License Cost Per Server ($) $8,000 per serverAverage Server Memory Size (GB) 32 GB of physical memoryAverage VM Size (GB) 4 GB of allocated memoryNumber of Workload VMs (#) 300 VMsOvercommit Ratio (%) 30 % of allocated memory overcommitted

ResultsVMs per server without EM4J 8 VMs per serverNumber of servers without EM4J 38 vSphere host serversVMs per server with EM4J 10 VMs per serverNumber of servers with EM4J 30 vSphere host serversReduction in Number of Servers Needed 8 vSphere host serversTotal Server CapEx Cost Savings $80,000.00Total Server OpEx Cost Savings $16,000.00 per yearTotal Software License Savings $64,000.00Total Value of EM4J Per VM $533.33 per VM for year one (CapEx + OpEx)

EM4J and tc Server:Join us on the journey to greater

efficiency.

Download Today http://www.vmware.com/products/vfabric-tcserver/

Questions?

EM4J and tc Server:Maximum Consolidation for Java on ESX

Download Today http://www.vmware.com/products/vfabric-tcserver/