the kvm weather report - · pdf filethe kvm weather report ... principles for vcs 2006 libvirt...
Post on 30-Mar-2018
238 Views
Preview:
TRANSCRIPT
1
The KVM Weather Report
Jes Sorensen <Jes.Sorensen@redhat.com>LinuxCon Japan, June 1st 2011
2
Agenda
Foundation for Cloud
Future
Virtualization Overview
3
Virtualization Overview: History
1967 IBM CP-40 1972 Goldberg Arch Principles for VCS
2006 libvirt launched 2007 KVM integrated into Linux 2.6.20
2011 KVM goes to 110
2
4
6
8
10
12
KVM goes to 11
4
Virtualization Overview: History
5
Virtualization Overview: KVM TodayNative vs. Virtualized: 1% Overhead in KVM
SAP Linux Certification SuiteCPU intensive reports phaseRHEL 6 + KVM and Intel E7-8800
6
KVM: The Kernel-based Virtual Machine
7
KVM features
VT-x/AMD-V
EPT/NPT
SMP Guests
Live Migration
OvercommitCPU, Memory, I/O
Hotplug
High Performance I/O
SR-IOV
VT-d/IOMMUKSM
NUMASPICE
Power Management
Massive scalability
sVirtSVVP & WHQL
PCI Device Assignment
Live snapshot
TransparentHuge Pages
Native Windowsdrivers
8
Why reinvent the wheel?
Focus on virtualization.
9
KVM Architecture
10
KVM Architecture
11
KVM Architecture
12
KVM in the Real World
● RAS
● Security Isolation
● Resource management
● Performance improvements● CPU, Block, Net, Memory
● SPECvirt
13
RAS
● timekeeping
● QMP, qdev, VMState, SeaBIOS
● vmchannel
● stable guest HW ABI● machine type● stable PCI topology
● PCI Device Assignment improvements
● Live migration with large memory guests
14
Isolation: sVirt Security (or similar w/ AppArmour)
● Applying security labels to individual guest virtual machines and their resources
● Guest Isolation achieved withSELinux Mandatory AccessControls (MAC)
● Protect against untrustedGuest VM
● Protect against Hostmisconfiguration
● Prevents unauthorizedaccess of Guests/Host
● Builds on existing, provensecurity mechanisms & controls
15
Isolation: Resource Management
Virt Guest A50% CPU50% Mem
Virt Guest B25% CPU25% Mem
Application Z25% CPU25% Mem
Network40% net 40% net 20% net
60% 20%20% I/O
Storage
● Control Group (Cgroups) for CPU/Memory/Network/Disk
● Benefit: guarantee Quality of Service
● Ideal for: Virtualization/Cloud deployments
16
● Scale to 64 vcpus! And same pcpu scaling as Linux
● Guest spin lock-holder preemption sol'n
● KVM efficiency● User return notifiers...get lazier
● x2apic● Use MSR access to limit mmio accesses to the irq chip
KVM: CPU Performance
17
Performance: SMP Scalability
1-cpu 4-cpu 8-cpu 16-cpu 32-cpu 64-cpu0
10000
20000
30000
40000
50000
60000
70000
80000
64 CPU Scalability - Stream BenchmarkIntel EX 64-cpu, 128GB, FC
RHEL 6 PhysicalRHEL 6 Virtual
Mem
ory
BW
(M
B/s
ec)
Excellent, linear scalability; minimal virtualization overhead
Note: With this h/w, at 32 CPUs socket bandwidth is saturated
18
KVM Block Architecture
19
KVM: Block Performance
● qemu vectored I/O (preadv/pwritev) and native AIO
● virtio● flush + fua● MSI support● ioeventfd
● qcow2 cache
20
KVM Performance – RHEL6 aio=native Win2k8 Intel 24cpu, 64GB, FC IOmeter
2k 4k 8K 16K 32K 64K0
50
100
150
200
250
300
350
400
450
IO - Sequential Reads
r5kvm r6aio_native metal
MB
/se
c
2k 4k 8K 16K 32K 64K0
10
20
30
40
50
60
70
80
90
100
CPU - Sequential Reads
r5kvm r6aio_native metal
%C
PU
21
KVM Performance – RHEL6 aio=native Win2k8 Intel 24cpu, 64GB, FC IOmeter
2k 4k 8K 16K 32K 64K0
10
20
30
40
50
60
70
80
90
100
CPU - Sequential Writes
r5kvm r6aio_native metal
%C
PU
2k 4k 8K 16K 32K 64K0
50
100
150
200
250
300
350
400
IO - Sequential Writes
r5kvm r6aio_native metal
MB
/se
c
22
KVM Network Architecture
23
Performance: Networking● vhost-net
● virtio backend in kernel● transparent to guest● reduce latency● better throughput● lower CPU consumption
● SR-IOV for near native performance
● VEPA/VNLink
24
virtio network architecture – detail
25
vhost_net Efficiency
32 64128
256512
10242048
40928192
1638432768
655070
50
100
150
200
250
300
350
400
8 Guest Scale Out RX Vhost vs Virtio - % Host CPU
Mbit per % CPU netperf TCP_STREAM
VhostVirtio
Message Size (Bytes)
Mb
it /
% C
PU
(b
igge
r is
be
tte
r)
26
Latency comparison – RHEL 6
1 64765
61478 21 35125
195384
10272048
409312288
2457332771
655360
50
100
150
200
250
300
350
400
Network Latency by guest interface method
Guest Receive (Lower is better)
host RX virtio RX vhost RX SR-IOV RXMessage Size (Bytes)
La
ten
cy (
use
cs)
27
Performance: Memory
● Transparent Huge Pages● Benefit for bare metal apps (4.2%)● Huge benefit for KVM VMs (25%)
● KSM working with THP
28
Performance
What does it add up to?
29
SPECvirt
30
SPECvirt_sc2010 results*
RHEL 5.5 (KVM) IBM X3650 M3 (12 cores, 144GB, 12 tiles)
VMware ESX 4.1 HP DL380 G7 (12 cores, 192GB, 13 tiles)
RHEL 6 (KVM) IBM HS22V (12 cores, 288GB, 14 tiles)
RHEL 5.5 (KVM) IBM x3690 X5 (16 cores, 512GB, 14 tiles)
RHEL 6 (KVM) IBM x3690 X5 (16 cores, 1TB, 18 tiles)
VMware ESX 4.1 U1 HP BL620c G7 (20 cores, 512GB, 19 tiles)
RHEL 6.1 (KVM) HP BL620c G7 (20 cores, 512GB, 19 tiles)
VMware ESXi 4.1 Bull SAS (32 cores, 512GB, 28 tiles)
VMware ESXi 4.1 IBM x3850 x5 (32 cores, 1TB, 28 tiles)
VMware ESX 4.1 HP DL580 G7 (40 cores, 1TB, 38 tiles)
RHEL 6 (KVM) IBM x3850 X5 (64 cores, 2TB, 56 tiles)
RHEL 6 (KVM) IBM x3850 X5 (80 cores, 2TB, 72 tiles)
0
1000
2000
3000
4000
5000
6000
7000
8000
1169 1221 1367 13691763 1811 1820
2721 2742
3723
5466
7067
*As of May 4, 2011. See http://www.spec.org/virt_sc2010 for details
31
Agenda
Foundation for Cloud
Future
Virtualization Overview
32
Cloud
● Computing as a Service (Anything as a Service)
● Utility model
● Self provisioning
● Dynamic resources
33
KVM in Cloud
● Increase density
● Improve efficiency
● Multi-tenancy (security, isolation)
34
KVM Is The Cloud
35
Agenda
Foundation for Cloud
Future
Virtualization Overview
36
Roadmap
● Scaling and performance
● NUMA
● Lock holder preemption
● Zero copy networking
● Multiqueue virtio-net
● Debugging and profiling
● Further cgroup integration
● virt-agent (guest agents)
● QCOW2 performance
● Multi-tenant network isolation
● Memory management (async page faults, hinting, compcache)
● USB 2.0 support
● Performance Monitoring Unit (PMU) support
Weather forecast...
37
KVM development activity
2.6.20 2.6.21 2.6.22 2.6.23 2.6.24 2.6.25 2.6.26 2.6.27 2.6.28 2.6.29 2.6.30 2.6.31 2.6.32 2.6.33 2.6.34 2.6.35 2.6.36 2.6.37 2.6.38 2.6.390
50
100
150
200
250
300
350
KVM commits per release
Release
Co
mm
its
38
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 240
100
200
300
400
500
600
QEMU commits per month
Months since start
Co
mm
its
QEMU development activity
39
Cloudy with a chance of
total world domination
top related