aix virtual user group july 31, 2014 powerkvm overview · aix virtual user group july 31, 2014...

© 2009 IBM Corporation

AIX Virtual User Group

July 31, 2014

Erwin Earley ([email protected]) - IBM STG Lab Services & Training

31 July 2014

PowerKVM Overview

mailto:[email protected]

© 2014 IBM Corporation2

Why Are We Talking About Linux & PowerKVM?

Linux is the world's fastest growing Operating System

Over 90% of world's fastest supercomputers, including top 10 in TOP500 list, run on Linux

8 of the world's top 10 websites, including Google, YouTube, Yahoo, Facebook, and Twitter run on Linux

80% of all Stock Exchanges in the world rely on Linux

95% of the servers used by Hollywood studios for animation films run on Linux

U.S. Department of Defense is the “single biggest install base for Red Hat Linux” in the world.


POWER8 Scale-out Systems

Power Systems S822LPower Systems

S812L

Power Systems S822

1 & 2 Sockets

Power Systems S814

Power Systems S824

Power Systems S824L

•1-socket, 2U•POWER8 processor•Linux only•CAPI support (1)•2H14

•2-socket, 2U•POWER8 processor•Up to 24 cores•1 TB memory•9 PCI Gen3 slot•Linux only•CAPI support (2)•PowerVM & PowerKVM

•2-socket, 2U•Up to 20 cores•1 TB memory•9 PCIe Gen 3•AIX & Linux•CAPI support (2)•PowerVM

•1-socket, 4U•Up to 8 cores•512 GB memory•7 PCIe Gen 3•AIX, IBM i, Linux•CAPI support (1)•PowerVM

•2-socket, 4U•Up to 24 cores•1 TB memory•11 PCIe Gen 3•AIX, IBM i, Linux•CAPI support (2)•PowerVM

•2-socket, 4U•Up to 24 cores•Linux•NVIDIA GPU•2H14

POWER8 roll-out is leading with scale-out (1-2S) systems Expanded Linux focus: Ubuntu, KVM, and Open Stack Scale-up POWER8 (>2S) systems will be rolled out over time PCI Gen3 right out of POWER8 processor OpenPOWER Innovations


What is KVM

KVM delivers server virtualization based on open source Kernel-based Virtual Machine (KVM) Linux technology

KVM enables the sharing of real compute, memory, and I/O resources through server virtualization

KVM-based server virtualization enables optimization and the commitment of resources like CPU and memory


What the heck is KVM?

KVM = Kernel Virtual Machine

Consists of a number of different components Primarily, a kernel module: kvm.ko

Brings core virtualization and hypervisor features to the Linux kernel A userspace program/facility: QEmu

Provides emulation and virtual devices + control mechanisms A standard interface library: libvirt

Standard library used to manage virtual machines Provides an API

These pieces convert a Linux kernel into a hypervisor Existing Linux scheduler and facilities leveraged Virtual machines exists as userspace processes to the kernel/hypervisor This Linux kernel is designated as the “Host” Virtual Machines are called “Guests”

KVM runs on just about every platform that Linux has been ported to.Now it works on Power!


KVM – At A Glance

• KVM (Kernel-based Virtual machine) – Linux kernel module that turns Linux into a hypervisor

• Requires hardware virtualization extensions• Including paravirtualization where applicable

• Supports multiple architectures including PowerPC• Competitive performance and feature set• Advanced memory management• Tightly integrated into Linux

Paravirtualization – a virtualization technique that presents a software interface to virtual machines (VM) that is similar but not identical to that of the underlying hardware


The KVM Approach to Virtualization

• A hypervisor needs• A scheduler and memory management• An I/O stack

• Device drivers

• A management stack

• Networking

• Platform Support Code

• Linux has support for all of the above

• KVM reuses as much of the Linux-base code as possible

• KVM's focus is on virtualization, leaves other components to respective developers

• KVM benefits (and will continue to benefit) from related advances in Linux


What the heck is PowerKVM?

PowerKVM is an IBM product

Embedded Linux built out, with all KVM modules and programs “Appliance” Full shell (bash) provided Full access to libvirt Many built in tools and monitoring solutions

Kimchi Nagios Ganglia

Easy repository-based updates Fully compliant libvirt Installation options:

Shipped pre-installed Optical media based install Network based install

Install media can also upgrade This appliance Linux OS is the hypervisor/Host


What the heck is QEMU?

A rather amazing open source hardware emulation project● Can emulate 9 target architectures on 13 host architectures! Provides full system emulation supporting ~200 distinct devices Very sophisticated and complete command line interface (CLI) Pronounced: “Q – eem - yoo”

QEMU is used by KVM● Device model for KVM Provides management interface Provides device emulation Provides paravirtual IO backends

PowerKVM does not use QEMU for CPU instruction emulation● Provides a similar function in PowerKVM as VIOS in PowerVM

● Except there is a QEMU instance for each guest, not one large appliance guest● On Power, no “Full” virtualization / emulated CPU or binary translation

● Too slow!


What is libvirt?

A hypervisor management library

Provides a stable, cross-platform interface for higher-level management tools

Used to manage guests, virtual networks and storage on the KVM host

Provides APIs for management

The configuration of each guest is stored in an XML file.

Allows remote management of guests–Encryption, certificates (TLS), authentication (SASL)

Communication between libvirt and tools management is done via a daemon called libvirtd

–Check status: “systemctl status libvirtd”


KVM Terminology

KVM PowerVM

Integrated Management Module (IMM) FSP

Host, Hypervisor Hypervisor

Unified extensible firmware interface (UEFI) and the basic input/output firmware interface (BIOS)

PowerVM hypervisor driver (pHyp) firmware

KVM host userspace (qemu) Virtual I/O server (VIOS)

Host userspace tools based on the libvirt API, including virsh

Integrated Virtualization Manager (IVM) Hardware Management Console (HMC)

KIMCHI or virt-manager Integrated Virtualization MangerHardware Management Console

Command-line message-based hardware management interface to manage IPMI-enabled devices on remote host with impitool

Integrated Virtualization Manager (IVM)Hardware Management Console (HMC)


How the heck does it work? First let's review...

PowerVM Hypervisor

Director /VMControl or PowerVC

Existing Stack

Hypervisor /System Firmware

Sys MgmtSoftware

Smart Cloud CloudSoftware

OperatingSystem

FSP

Partition FirmwareOpenFirmware

IaaS

Various physical Networks

System Firmware

OpenFirmware

Hardware Management

Console (HMC)

VIO

S


Virtualization and the POWER architecture

● The Power platform consists of a vertical integration of hardware, firmware and software components that provide unmatched

● Virtualization features ● Flexibility● Performance

● The platform standards, guidelines and specifications established by a governing bodypower.org

● Power.org defines ● Processor ISA● Memory management● Architecture platform reference specifications

●POWER Architecture Platform Reference (PAPR)

● PAPR describes the environment in which a general purpose operating system will run,● bootstrap● runtime● shutdown function● virtualization operation

● Virtualization standards for the platform must be implemented using a combination of● hardware, firmware and software.


Power Systems Software Stack

PowerVM HypervisorHypervisor /

System Firmware

OperatingSystem

FSP


System Firmware

OpenFirmware

VIO

SPOWER7 Hardware

[PAPR] Platform interfaces


Virtualization and the POWER architecture

● Virtualization on POWER means the cooperation of● hardware, firmware and software.

● This allows for efficient management of privileged hardware resources. ● The hardware includes 3 privilege levels:

● Hypervisor● Supervisor● User

● The Hypervisor state includes partitioning/virtualization facilities via Special Purpose RegistersThese control:

● MMU hash table access● Interrupt control (which ones go to VM, which ones go to Hypervisor)

● Entire platform designed for cooperation or Paravirtualization● Some aspects of the machine cannot be emulated or spoofed● Operating systems have some virtualization responsibilities

● OS calls directly into the hypervisor for some things (hcalls)


Always Paravirtualized

● Hypervisor runs in Hypervisor mode (highest privilege level)● Has access to all memory and system resources●

● Operating Systems in guests/VMs/LPARs run in supervisor mode● Virtualized Operating Systems must conform to the PAPR interfaces

●AIX, IBM i, and ppc64 Linux kernel● PAPR conformance gives knowledge of when to call into the hypervisor● No need to trap and emulate privileged instructions● Runs at full hardware speed

● Hypervisor and VMs each have their own MMU hash tables

● Result = Fast!●High performance, very low overhead virtualization


The POWER Hypervisor (pHyp)

● The only software that runs in Hypervisor mode on the processor.● Responsibilities:

● Managing CPU● Managing memory● Routing interrupts ● Some simple transports● Scheduling of virtual machines● Some platform management● Error recovery

●The pHyp provides interfaces for management, but does not allow a direct log in.

●Deliberately is kept as simple as possible, but has added functions over the years● Manages Non-Uniform Memory Architecture (NUMA) layouts● Processor affinity● Routing of virtualized networking between virtual machines on the same physical server

● The hypervisor does not handle the virtualization of input and output devices


Power Systems Software Stack with KVM

PowerKVM

Hypervisor

OperatingSystem

FSP

Partition FirmwareSLOF

OPAL Firmware

SLOF

POWER8 Hardware


System Firmware

qemu qemu


Power Systems Software Stack PowerVM vs PowerKVM

PowerKVM

Hypervisor

OperatingSystem

FSP

Partition FirmwareSLOF

SAPPHIRE Firmware

SLOF

POWER8 Hardware


System Firmware

qemu qemuPowerVM HypervisorHypervisor /

System Firmware

OperatingSystem

FSP


System Firmware

OpenFirmware

VIO

S

POWER7/8 Hardware



The PowerKVM Hypervisor

● The Host OS runs in Hypervisor mode on the processor● Guest kernels run in supervisor mode● Host has access to all memory and machine resources● Host does not trap or emulate privileged instructions from guests

● Special firmware required● Allows access to hypervisor mode● Disables pHyp

● KVM guests are paravirtualized using the PAPR interfaces● Same interfaces as PowerVM● Existing Linux distributions for Power will work (SLES, RHEL)

NOTE: This may not be true for PowerKVM support distributions due to “Endianness” issues



Changes had to be made!● Qemu

● New machine type added (“pseries”)

● Linux kernel● New KVM “flavor”: book3s_hv

● book3s_pr was the previous KVM on powerpc, ●uses emulation, guest in usermode

● New platform type “powernv” (non-virtualized)● Allows Linux to run truly “bare metal”

● Partition firmware● Open source SLOF (Slim-Line Open Firmware)


Power Virtualization Options

PowerKVM

PowerVM

PowerVM: Provides virtualization of Processors, Memory, Storage, & Networking for AIX, IBM i, and Linux environments on Power Systems.

Initial Offering: 2004

Initial Offering: Q2 2014

PowerKVM: Open Source option for virtualization on Power Systems for Linux workloads.

For clients that have Linux centric admins. (RHEL 6.5 & SLES 11.3)


PowerVM & PowerKVM Unique Features

PowerVM Unique Features not in PowerKVM

Dedicated Processors Shared Processor Pools Shared Dedicated Processors Guaranteed minimum

entitlement Hard Capping of VMs Capacity on Demand IFLs

Compute

Security vTPM Existing Security Certifications* Firmware based hypervisor

I/O NPIV* SR-IOV* Dedicated I/O devices* Redundant I/O

virtualization(Dual VIOS)

Configuration DLPAR* Support for AIX and IBM i VMs System Pools

Ubuntu support No HMC needed Exploits POWER8 Micro-Threading NFS storage support iSCSI storage support

PowerKVM Unique Features not in PowerVM

*PowerKVM functionality planned


PowerVM vs KVM Out of Box Experience

Planning and Sizing Infrastructure

Initial Server Configuration

Virtualization Setup Initial VM Creation Advanced Virtualization Management

Serviceability

Workload Estimator(WLE)

Score request for certified storage

ASM/HMC

Power Control Network Config

Connection to management consoles

HMC / IVM

Install VIOS & Configure

FC Storage, Internal Disk

Network definition

HMC / IVM

Firmware maintenance HMC

Phone Home

PowerVM

HMC / IVM PowerVC

VMControl

Planning and Sizing Infrastructure

Initial Server Configuration

Virtualization Setup Initial VM Creation Advanced Virtualization Management

Serviceability

Workload Estimator(WLE)

ASM: Setup FSP IP address, if no DHCP available

IPMI: Remote Power Control and remote console

Host OS: IP, timezone and root password (if defaults do not apply)

ESA Agent Config

KVM pre-loaded with reasonable defaults for storage, network and logging

Point browser to Kimchi-ginger for further Host OS configuration

Linux cmd line available

Error logs exposed through KVM/Linux

Phone Home ESA Agent

Firmware Maintenance through Linux

PowerKVM

Virsh command line

Kimchi (Web)

PowerVC

Or SmartCloud


What is Different with KVM on Power?

Let's Compare

A couple of things to keep in mind:

KVM is open source

Companies (e.g., Red Hat) offer commercial KVM hypervisor products

On x86,it's also possible to enable KVM on an existing Linux installation– Turns that Linux OS into a hypervisor

Not all companies/distributors/solutions officially support both usage models


What is different with KVM on Power?Some internal differences

● No “full virtualization” on Power● PowerKVM implements PAPR● No full CPU emulation

● Qemu device models● Disk

● virtio-scsi● virtio● spapr-vscsi● No IDE

● Network● virtio● E1000 (intel)● Rtl (realtek)● spapr-vlan

● Graphics● vga (VNC backend only)● No Spice (coming later)


Linux on Power enables open source virtualization with KVM

FirmwarePowerVM

Linux-based KVM

IBM Mgmt SW

Director / VMControl(PowerVM)

Existing Stack Additional New Stack

Hypervisor /Firmware

Sys MgmtSoftware

Smart CloudSmart Cloud CloudSoftware

OperatingSystem

XCAT

Preliminary KVM details:a) Virtualizes selected systems – Scale-Out models, Linux-onlyb) Extends Power virtualization to lightweight, x86-like solutionsc) Executes directly on hardware, not nested virtualization in an LPARd) Supports system “migration” to PowerVM via early boot-time selections

(configurable)e) Runs without an HMC, IVM, or VIOSf) Embraces opensource clouds and other virtualization SW through standard

interfaces like oVirt (VDSM) and OpenStackg) Holds potential to reduce number of hypervisors in the datacenter


What Linux Distributions in various Power Environments?

Linux Release Endian Dedicated LPAR

PowerVM Guest

PowerKVMGuest

Redhat 5.10 Big

Redhat 6.4 Big

Redhat 6.5 Big

SUSE 11 SP3 Big

Ubuntu* 14.04 Little

1. Select the applications you want to run on Linux on Power2. Then look at the Linux distributions that are available for those

apps3. Pick your Linux distribution of choice

*Exploits P8


PowerKVM Exploits POWER8 Micro-Threading

VM1

Traditional PowerVM and PowerKVM Dispatches the complete core to the VM

CPU Core

PowerKVM with Micro-Threading Dispatches Multiple VMs on a single core at the same time.

SMT1-2

CPU Core 4/1 Division

SMT1-8

VM1 VM2 VM3 VM4

Good for many small VMs / Workloads. Enabled with the PowerKVM ppc64_cpu command. 4/1 Division is only option initially.

© Copyright IBM Corporation 2014

Storage ManagementTwo types: Volumes and Pools

Storage Volume

A storage device that can be assigned to a guest–Virtual disk–CD/DVD/floppy image

Can be backed by–Block device (disk partition, LV, LUN)–File

Storage Pool

Pools store volumes–Directory (for file-backed volumes)–Complete physical disk or partition–Libvirt mounted Filesystem (local or remote)–ISCSI target–LVM Volume group (Lvs as volumes)–SCSI host adapter (LUNs as volumes)

32

Managing Storage with Kimchi

All storage management function resides on the Storage tab

Storage ManagementWeb Interface (Kimchi)

33


All storage management function resides on the Storage tab

Storage ManagementWeb Based (Kimchi)

34


Create a Storage Pool

On the storage tab, click on the +. Give the pool a name,a pool type and a location.

35


Display volumes in a storage pool

On the storage tab, click the arrow to display the contents of the pool.

36


Storage volume creation

Kimchi does not provide a function to explicitly create a storage volume● A volume is created at guest creation time out of the selected storage pool● Volume size is defined in the guest template

37

Managing Storage with virsh

File-backed system image tools

Qemu provides a command qemu-img that provides image file management. Used to create, convert and check system image files Uses subcommands to invoke various functions

create – create a new disk image check – checks an existing disk image convert – convert a disk image to a different format info – displays info about the disk image snapshot – manages snapshots of existing images commit – apply changes to an existing image rebase – creates a new base image resize – change the size of an image

Create a new file image

qemu-img create -f FORMAT -o OPTIONS PATH SIZE

Convert a file image to a different type

qemu-img convert

38

Managing Storage with virsh

File-backed system image tools

Convert a file image to a different type

qemu-img convert -c -f SRC_FMT -O DST_FMT -o OPTIONS SRC_PATH DST_PATH

-c : applies compression to the target disk.SRC_FMT: The format of the source diskDST_FMT: The format of the destination diskOPTIONS: additional optionsSRC_PATH: path to the source disk to be convertedDST_PATH: path to the destination disk

Note 1: only raw format disks can be resized.Note 2: only qcow disk formats can be compressed.

Management Tools

• There are multiple tools for managing a KVM environment:

• Kimchi – Web based / open source driven

• Intended for small environments / POCs

• Open Stack – community driven

• Intended for enterprise level management

• PowerVC / SCE – IBM product

• Intended for enterprise level management

Kimchi – Host Page

• Provides a view of the overall KVM environment, including:

• System statistics

• O/S information

• Debug Reports (currently not working in PowerKVM)

Kimchi – Guests Page

• Shows currently defined guests and their running state

• Includes Live tiles showing currently console display

• Shows currently resource utilization of each guests

• Guests can be stoped/started/rebooted

• New guests can be created based on existing templates

• VNC sessions can be started from the Guests page

Kimchi Templates Page

• A template defines the resource characteristics of a guest

• Processor

• Memory

• Disk

• Storage Pool

• Network

• Installation Source

Kimchi – Storage Page

• Provides view of existing storage pools including

• Size

• Utilization

• New Storage Pools can be created. Storgae can be

• DIR – local file backed

• NFS – Remote file backed

• ISCSI – Physical Device connection

• Logical

Kimchi – Network Page

Provides display of currently defined networks

Additional networks can be defined:

Isolated – no connection to a physical network

NAT – Outbound network connection using Network Address Translation

Bridged – Network connection direclty to a physical network

• Provides a shell interface for working with KVM functions

• Common commands:

• 'help' – provide of all virsh commands

• 'console' – provide a console interface to a guest

• 'list –all' – list all guests and their current state

• There are commands for working with:

• Domains

• Host and Hypervisors

• Interfaces

• Network Filtering

• Networking

• Node Devices

virsh

•Snapshots

•Storage Pools

•Storage Volume


Q&A (from Jeff Scheel's developerWorks Blog)

When KVM be available on Power?– The outlook for general availability is next year (2014). However, IBM has already

started releasing patches to various KVM communities to support the POWER platform.

On what systems does IBM intend to support KVM?– IBM intends to initially support KVM on a limited set of models, targeted at the entry end

of the system servers. This strategy supports IBM's efforts to capture the largest growing market, x86 Linux servers In the 2-socket and smaller space.

How does IBM plan to position KVM against PowerVM?– IBM remains committed to POWERVM being the premier enterprise virtualization

software in the industry. With KVM on Power, IBM will be targeting x86 customers on entry servers but will offer both KVM and PowerVM to meet the varying virtualization needs of PowerLinux customers. However, KVM virtualization technology represents an opportunity to simplify customer's virtualization infrastructure with a single hypervisor and management software across multiple platforms.



What Linux versions from Red Hat and SuSE will provide KVM hosts support on Power?

–The decision to provide KVM on PowerLinux will be made by Red Hat and SuSE. IBM will be working with them in the months to come and would welcome their support

What management and cloud software will support KVM on Power?–For KVM node management, IBM intends to work with multiple

vendors, including Red Hat and SuSE to certify KVM on Power into their system management software offerings. Additionally, IBM plans to contribute any patches necessary to OpenStack to extend the KVM driver to Power. Using this foundation, additional KVM and third software should provide a diverse set of management software



What will software providers need to do to support KVM on Power?–Most software providers have become comfortable with some form of

virtualization such as PowerVM, VMWare, and KVM. Just like with applications in Linux, software providers should find that applications in the KVM environment behave similarly on x86 and Power platforms. As such, each vendor should understand any challenge KVM on Power would provide.

What operating systems will be supported as guests in KVM on Power?–Given that KVM is initially targeted to be released on Linux-only

servers, only Linux is planned at this time. IBM plans to certify the latest updates of RHEL 6 and SLES 11 as KVM guests.



How will KVM run on the Power Systems?– The design goal of KVM on Power is to be just another hardware platform

supporting KVM. As such, the KVM on Power will be true to the KVM design point of a KVM host image that supports one or more guests. PowerVM constructs such as the HMC, IVM, and VIOS will not exist in KVM. Management and virtualization will occur through the KVM host image.

Will VM run in a PowerVM logical partition (LPAR)?– While KVM supports a user-mode virtualization that can run on any Linux

operating system, KVM on Power is being developed to run natively on the system, not nested in PowerVM. This is done to enable KVM to run optimally using the POWER processor Hypervisor Mode. As such, the system will make a decision very early in the boot process to run KVM or PowerVM. This is envisioned as a selectable option managed by the Service Processor (FSP).



Will it be possible to migrate from KVM on Power to PowerVM or vice versa?– While the virtualization mode will be selectable on systems, the process of

migrating from KVM and PowerVM will require additional steps such that frequent migrations will be unlikely. However, in the case when a customer wishes to upgrade to PowerVM to acquire advanced virtualization capabilities, this migration should be supported. Steps to backup and restore the VM image will be quired when migrating in either direction.

Will AIX and IVM I run in KVM on Power?– Given that KVM initially runs on Linux-only platforms, support for non-Linux

operating systems has not been planned at this time.

Will Windows run in KVM on Power?– Windows does not run on Power Systems. As such, supporting it in a KVM

guest VM will not work.


PowerKVM Demo

Thank You

MerciGrazie

Gracias

Obrigado

Danke

Japanese

English

French

Russian

German

Italian

Spanish

Brazilian PortugueseArabic

Traditional Chinese

Simplified Chinese

Hindi

Tamil

Thai

Korean

Questions [email protected]

aix virtual user group july 31, 2014 powerkvm overview · aix virtual user group july 31, 2014...

Documents