vio updates - ibm · pdf file• ability to roll out updates to vio servers without...

Technical University/Symposia materials may not be reproduced

in whole or in part without the prior written permission of IBM. 9.0

VIO UpdatesFeatures You May Have Missed

James Nash

Power Systems Strategic Initiatives

James Nash

jjnash@us.ibm.comPower Systems Strategic Initiatives

(formerly known as ATS)

Special Thanks to

Benoit Creau

Steve Knudson

Steve Nasypany

Mark Olson

Steve Pittman

Session Objectives What’s relatively new

PowerVM Editions and Versions

VIO Server 2.2.3

― Simplified SEA failover configuration

― Shared Storage Pool enhancements

― Updates to ‘part’

― NPIV Statistics

― Useful but oft forgotten

Live Partition Mobility

Power Enterprise Pools

Power Integrated Facility for Linux

VIO Alternatives

―SR-IOV

―RoCE

―PowerKVM5

PowerVM Editions

Virtual I/O Server

VIO Server

POWER Hypervisor (PHYP)

Client

VIO Versions

• 2.2.1.5-FP25 SP03 December 2012

• 2.2.1.6-FP25 SP04 April 2013

• 2.2.1.7-FP25 SP05 August 2013

• 2.2.1.8-FP25 SP08 February 2014

• 2.2.1.9-FP25 SP09 August 2014

• 2.2.2.0 December 2012

• 2.2.2.1-FP26 March 2013

• 2.2.2.2 FP26 SP01 February 2013

• 2.2.2.3 FP26 SP02 August 2013

• 2.2.2.4-FP26 SP04 February 2014

• 2.2.2.5-FP26 SP05 August 2014

• 2.2.2.6 January 2015

• 2.2.3.1-FP27 SP01 November 2013

• 2.2.3.2-FP27 SP02 April 2014

• 2.2.3.3-FP27 SP03 August 2014

• 2.2.3.4-FP27 SP04 November 2014

Latest Levels as of August 2014

Keep it current

• 2.2.1.9-FP25 SP09 August 2014

• 2.2.2.6 January 2015

• 2.2.3.4-FP27 SP04 November 2014

HMC 8.8.2 SP1 – New view

HMC 8 requires:

Rack Mount CR5 or newer

Desk-side C08 or newer

At least 2 GB of memory

NIB and SEA Failover

SEA Failover with Load Balance

Simplify SEA Failover

• No Need for a Control Channel

• Requirements

– VIO server 2.2.3 or higher

– HMC 7.7.8 or higher

– System Firmware 780

• Note – Function is not available on “B” model servers

– IBM Power 770 (9117-MMB)

– IBM Power 780 (9179-MHB)

– VLAN ID 4095 not in use

VIO Server 1

Ethernet

Switch 1

Client

LPARClient

VIO Server 2

ent0ent1

11ent3

12ent1

Simplify SEA Failover – VIO 2.2.3

Ethernet

Switch 2

padmin@vio1$ entstat –all ent1 | grep –iE “Priority|Port VLAN ID”

Priority: 1 Active: False

Port VLAN ID: 11

Port VLAN ID: 12

Port VLAN ID: 13

• Verify VLAN IDs and Trunk Priorities

Port VLAN ID: 11

Port VLAN ID: 12

Port VLAN ID: 13

padmin@vio1$ mkvdev –sea ent0 –vadapter ent1 ent2 ent3

–default ent1 –defaultid 11 –attr ha_mode=sharing

ent4 Available

• Create your Shared Ethernet Adapter

padmin@vio2$ mkvdev –sea ent0 –vadapter ent1 ent2 ent3

–default ent1 –defaultid 11 –attr ha_mode=sharing

ent4 Available

ctl_chan=ent#

padmin@vio1$ lsdev -dev ent4 –attr

attribute value description user_settable

accounting disabled Enable per-client accounting of network statistics True

adapter_reset yes Reset real adapter on HA takeover True

ctl_chan Control Channel adapter for SEA failover True

gvrp no Enable GARP VLAN Registration Protocol (GVRP) True

ha_mode sharing High Availability Mode True

hash_algo 0 Hash algorithm used to select a SEA thread True

jumbo_frames no Enable Gigabit Ethernet Jumbo Frames True

large_receive yes Enable receive TCP segment aggregation True

largesend 1 Enable Hardware Transmit TCP Resegmentation True

lldpsvc no Enable IEEE 802.1qbg services True

netaddr 0 Address to ping True

nthreads 7 Number of SEA threads in Thread mode True

pvid 15 PVID to use for the SEA device True

pvid_adapter ent1 Default virtual adapter for non-VLAN-tagged packets True

qos_mode disabled N/A True

queue_size 8192 Queue size for a SEA thread True

real_adapter ent0 Physical adapter associated with the SEA True

send_RARP yes Transmit Reverse ARP after HA takeover True

thread 1 Thread mode enabled (1) or disabled (0) True

virt_adapters ent1,ent2,ent3 List of virtual adapters associated with the SEA True

• Verify VLAN IDs and Trunk Priorities

padmin@vio1$ entstat –all ent4 | grep –i “Control Channel PVID”

Control Channel PVID: 4095

padmin@vio1$ entstat –all ent4 | grep –i state

State: PRIMARY_SH

• Where is the Control Channel

State: BACKUP_SH

VIO Server 1

Ethernet

Switch 1

Client

LPARClient

VIO Server 2

ent0ent1

12ent1

VLAN 4095

Ethernet

Switch 2

Remove Existing Control Channel

padmin@vio1$ oem_setup_env

root@vio1# rmdev –l ent6

ent6 Defined

padmin@vio1$ rmdev –dev ent6 -ucfg

ent6 Defined

• Requires both SEA adapters in a defined state (outage)

padmin@vio2$ oem_setup_env

root@vio2# rmdev –l ent6

ent6 Defined

padmin@vio2$ rmdev –dev ent6 -ucfg

ent6 Defined

root@vio1# lsattr –El ent6 –a ctl_chan

ctl_chan ent4 Control Channel adapter for SEA failover True

root@vio1# chdev –l ent6 –a ctl_chan=“”

ent6 changed

padmin@vio1$ lsdev –dev ent6 –attr ctl_chan

padmin@vio1$ chdev –dev ent6 –attr ctl_chan=“”

ent6 changed

• Blank out the control channel

root@vio2# lsattr –El ent6 –a ctl_chan

root@vio2# chdev –l ent6 –a ctl_chan=“”

ent6 changed

padmin@vio2$ lsdev –dev ent6 –attr ctl_chan

padmin@vio2$ chdev –dev ent6 –attr ctl_chan=“”

ent6 changed

root@vio1# mkdev –l ent6

ent6 Available

padmin@vio1$ cfgdev –dev ent6

ent6 Available

• Requires both SEA adapters in a defined state (outage)

root@vio2# mkdev –l ent6

ent6 Available

padmin@vio2$ cfgdev –dev ent6

ent6 Available

State: PRIMARY_SH

• Where is the Control Channel

State: BACKUP_SH

VIO Server 1

Ethernet

Switch 1

Client

LPARClient

VIO Server 2

ent0ent1

12ent1

VLAN 4095

Ethernet

Switch 2

Simplification Summary

Simplified SEA Failover setup

If you do not specify a control channel (ctl_chan) when

running mkvdev –sea … The system will “discover” SEA

Failover “partner” adapter in the other VIO

Discovery protocol uses VLAN id 4095. If this is one of your

actual tagged VLANs, you must continue to use control

channel on both sides

Multiple SEA pairs in the machine can share VLAN 4095 for

discovery

This is still SEA failover, and we still set priority 1 or 2 on the

trunked virtual adapter in the SEA

VIO server 2.2.3, HMC 7.7.8, Firmware 780. Not supported on

MMB or MHB at this time.

Perhaps we stay consistent with our current Power7 practices,

and use this for new Power8 machines.

Shared Storage Pools

Shared Storage Pool

VIOS VIOS

VIO 2.2.3

Shared Storage Pool Enhancements

– Pool Resiliency – mirror the storage pool (two failover groups)

– Pool shrink by removing physical volume

– Dynamic disk growth in the pool

– Scaling – more clients, larger physical volumes

– New lu, pv, failgrp commands

– Cluster wide operations performed concurrently

– VIO 2.2.2 added

• ability to roll out updates to VIO servers without stopping the cluster

• In case of failed repository disk, cluster remains active, no changes,

until disk is replaced

• VLAN tagging supported in the cluster

developerWorks articles to get started

Shared Storage Pool Limits

Feature Min Max

• Number of VIOS Nodes in Cluster 1 16

• Number of Physical Disks in Pool 1 1024

• Number of Virtual Disks (LUs) Mappings in Pool 1 8192

• Number of Client LPARs per VIOS node 1 200

• Capacity of Physical Disks in Pool 10GB 16TB

• Storage Capacity of Storage Pool 10GB 512TB

• Capacity of a Virtual Disk (LU) in Pool 1GB 4TB

• Number of Repository Disks 1 1

• Capacity of Repository Disk 512MB 1TB

Recommended VIO server requirements – per SSP node:

• Minimum CPU: 1 CPU of guaranteed entitlement

• Minimum memory: 4GB

• Storage requirements per SSP cluster (minimum): – One fiber-channel attached disk for cluster repository, 1 GB

– At least one fiber channel attached disk for data, 10GB

New command: lu

$ lu -list

POOL_NAME: test_pool

TIER_NAME: SYSTEM

LU_NAME SIZE(MB) UNUSED(MB) UDID

aix2-rootvg 10240 0 ad4f5d...

aix3-rootvg 20480 18460 242eac...

aix4-rootvg 102400 0 56e22b...

aix5-rootvg 10240 0 1a4758...

aix6-rootvg 40960 0 bfa955...

aix7-rootvg 51200 0 599ad4...

New command: pv

$ lspv

NAME PVID VG STATUS

hdisk0 00f629dbc192abd7 rootvg active

hdisk1 00f629dbbe759aa5 caavg_private active

hdisk2 00f629dbbe7c9b29 None

hdisk3 00f629dbbd739cbe None

$ pv -list

POOL_NAME: test_pool

TIER_NAME: SYSTEM

FG_NAME: Default

PV_NAME SIZE(MB) STATE UDID

hdisk2 1048576 ONLINE 3C231IBM FlashSystem

VIO Sizing Recommendations

http://www.redbooks.ibm.com/redbooks/pdfs/sg248062.pdf32

VIO Sizing – System Planning Tool

http://www-947.ibm.com/systems/support/tools/systemplanningtool/33

System Planning Tool

VIO Sizing – Workload Estimator

• Updated with information on POWER8 and PowerKVM

http://www-947.ibm.com/systems/support/tools/estimator/35

Workload Estimator

Other options

• IBM Sales Representative

• IBM Client Representative

• IBM Architect

• IBM Client Technical Specialist

• IBM Business Partner

1-866-883-8901

http://www-304.ibm.com/partnerworld/wps/bplocator/landing

• IBM Techline

After you have the box…

Performance Advisors

• Run advisors on test or production systems.

• Advisors will evaluate the environment for performance

optimization opportunities

– Gives guidance on how to make the necessary changes.

• Three advisors available….

– PowerVM Virtualization

– Java (JPA)

– VIOS & Virtual Ethernet (built-in VIO with v2.2.2.1 via ‘part’)

• “Built in Smarts” to detect some of the most common

problems that are encountered

• Available on Developer Works

– FREE OF CHARGE

• https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power+Systems/page/Performance+Tools

Virtualization Performance Advisor

• Diagnoses performance issues within PowerVM partitions.

• Runs within an AIX partition (dedicated or shared)

• Analyzes performance characteristics

• Provides a health check report.

• Report includes system configuration (available resources, hardware & software levels, etc), partition (type, mode, available resources, s/w levels), partition settings and recommendations to improve performance (e.g. placement, SMT, processor folding, processor, and memory pool utilization).

Java Performance Advisor

VIO 2.2.2.1 – part command

Fibre Channel Statistics

Color-coded Icons

Severe Performance Problem

Investigate

part - Components Monitored

The following are the types of advisory reports that

are generated by the VIOS Performance Advisor

tool: System configuration advisory report

–CPU (central processing unit) advisory report

–Memory advisory report

–Disk advisory report

–Disk adapter advisory report

– I/O activities (disk and network) advisory report

https://www.ibm.com/developerworks/wikis/display/WikiPtype/VIOS+Advisor

An option for VIO servers below v2.2.2.1

Reminder – viosbr (added at v2.1.2)

$ viosbr -backup file backup -frequency daily numfiles 5

Creates a backup file once a day, prefixed with

“backup” in /home/padmin/cfgbackups

– backup.01.tar.gz

$ viosbr -restore file

/home/padmin/cfgbackups/backup.03.tar.gz

http://www.ibm.com/partnerw

orld/gsd/searchprofile.do?na

me=VIOS_Recognized_List

For VIOS Recognized applications:http://www.ibm.com/partnerworld/gsd

Click on "Search solutions" link on the left

side of the page

Click on "Validations" towards the top

middle right of the page

Type "VIOS Recognized" in the search field

and click “Search”

This will generate a current list of officially

tested applications that are supported on

the VIO Server.

Third party software - updated

Live Partition Mobility

Evacuate option

migrlapr –o m –m source_server –t target_server –all

Performance suggestions

– large_send, large_receive on physical adapters in VIO

(these SHOULD already be on)

– Suggesting tcp_sendspace, tcp_recvspace 524288 on

VIO interface

– Jumbo frames if your environment supports it

Power Enterprise Pools allow mobile activations of CPU and

memory to be shared among a group of pool of Power Systems

All previous activations are static and do not move between

servers

New systems can be added to the pool

Pool information can be viewed, including pool resource

assignments, compliance, history logs

Up to 75% of system memory can be mobile

In a 770 pool, up to 60 mobile activations allowed (feature EP22)

In a 780-795 pool, up to 120 mobile activations for 780, up to 192

for 795 (feature EP23)

For 770, 780, up to 30 mobile memory activations of 100GB

(EMA4)

For 795, up to 120 mobile memory activations of 100GB (EMA4)

HMC 7.7.8, with 2GB of memory required

770-780 D models (9117-MMD, 9179-MHD) and 795 (9119-FHB)

Firmware 780 only (FW780, AH780)

770, 780 must have at least 4 static core activations, 25% of

memory static activations

795 must have 24 static core activations, or 25% of installed

cores, whichever is higher

In all servers, 25% of memory must have static activations

GA Dec 6 2013 for 795

GA April 25 2014 for 770, 780

Two type of pool – 770 pool, and a high end pool (780, 795)

“Master” HMC, a redundant HMC acts as “backup master”

Configuration file obtained from IBM CoD to create a pool, add server to pool,

add mobile activations to pool

Power Integrated Facility for Linux (IFL)

One Power IFL feature is 4 processor, and 32GB

activations, with 4 PowerVM EE entitlement and SWMA,

feature ELJ0 (zero not O)

Lower price activation than for AIX or IBM i, for Linux

Power customer agrees to segregate AIX and IBM i

partitions into a separate virtual shared processor pool,

away from Power IFL cores.

Any IBM Software Group (SWG) products run on Linux

on Power are also discounted - priced at 70 Processor

Value Units (PVU)

770, 780 B, C, D models, and 795

Firmware 780, HMC 7.7.8

Power Integrated Facility for Linux (IFL)

How do we keep AIX and IBM i from running on the less

expensive Linux cores? AIX, i, and VIO go in a sub pool, where

we can set a maximum CPU limit. Lower cost Linux cores are

used in the default pool

System

Topology

DRILL DOWN

Partition

DRILL DOWN

V1.1.2 support for POWER8

PowerVP Performance Monitor

Servers supported

― POWER7

― POWER7+

― POWER8

Firmware version 770, 780, 810 or higher.

System and partition-level agents

Minimum OS levels:

― AIX 6.1, AIX 7.1

― RedHat Enterprise Linux 6.4

― SUSE Linux Enterprise Server 11 SP3

― IBM i 7.1 with TR6

Do I need a VIO Server?

• RoCE or IB

• SR-IOV

• PowerKVM

• PowerNV

RDMA over Converged Ethernet

• PCIe3 2-port 40 GbE NIC / RoCE QSFP+ Adapter

– FC EC3B; CCIN 57BD

http://www-01.ibm.com/support/knowledgecenter/POWER8/p8hcd/fcec3b.htm?cp=POWER8%2F4-3-10-1-1-1661

SR-IOV

NIC SR-IOV for POWER7+ 770/780 with

Specific 10GE Ethernet adapters #EN0K and #EN0H in Gen2 slots

Or with newest IMFC #EN10 and #EN11

SR-IOV

Single Root I/O Virtualization

Runs “closer to the silicon” potentially offering some

performance efficiencies

Doesn’t require VIOS as a pre-req and thus can do simple

virtualization under PowerVM without VIOS …. BUT …. VIOS

continues to offer many additional advanced functions

Architecturally can virtualize a resource like an Ethernet

adapter and allocate/provide a user-defined minimum level of

bandwidth to a partition … Quality of Service (QoS)

Ethernet NIC announced.

Could use VIOS & SR-IOV together

LPAR #2

LPAR #3

LPAR #1

Additional Linux Options

Power S822L8247-22L

Power S824L

Power S812L8247-12L

The Redbook

http://www.redbooks.ibm.com/abstracts/sg248198.html?Open

VIO Command Reference (v2.2.3.3)

http://public.dhe.ibm.com/systems/power/docs/hw/p8/p8hcg.pdf

Appendix

PowerVM Shared Ethernet Adapter simplification:

Get rid of Control Channel Adapterhttp://chmod666.org/index.php/powervm-shared-ethernet-apdater-simplification-get-rid-of-control-channel-adapter/

PowerVM developerWorks - wikihttps://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/PowerVM

Virtual IO and virtualization developerWorks - wikihttps://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/Virtual%20I

O%20and%20virtualization

Virtualization Best Practices developerWorks – wikihttps://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/Virtualizatio

n%20best%20practices

HMC 8.8.1 SP1https://www.ibm.com/developerworks/community/blogs/brian/entry/hmc_version_8_release_8_8_1_highlights?lang

VIO and IVM Command Referencehttp://public.dhe.ibm.com/systems/power/docs/hw/p8/p8hcg.pdf

VIO 2.2.3.3 Release Noteshttp://www-01.ibm.com/support/knowledgecenter/POWER8/p8eeo/p8eeo.pdf

James Nash

Power Systems

Strategic Initiatives

Formerly known as

1177 S. Beltline RoadCoppell, TX 75019

jjnash@us.ibm.com

vio updates - ibm · pdf file• ability to roll out updates to vio servers without...

Documents

plu vio metros

veritas cluster server release notes -...

secure updates › wp-content › uploads › 2019 › 11...

npiv vio presentation

install vio on p710

datasheet vio l212 - dbtechnologies · vio l212 datasheet...

veritas cluster server...

national and local updates senco cluster meetings autumn...

electrosurgery vio - rabindra surgicals · electrosurgery...

vio enhanced model api reference manual - apcvest · vio...

draft multi cluster assessment mission report di khan,...

vio server & ivm commands

vio creating a vhost

hh.ns ‘vio - cimav

indictment ·vio: u.s.c

vio laptop

ibm vio server

vio technology

user's manual vio-100/200 mx100 series ·...

red hat cluster server to veritas cluster server migration...