configuration of vio on power6
TRANSCRIPT
© 2008 IBM Corporation
Session Title: Configuration of Virtual I/O on POWER6
Speaker Name: Janel Barfield
Session ID: pVI08
IBM Power Systems Technical Universityfeaturing IBM AIX and Linux
September 8 – 12, 2008 – Chicago, IL
IBM UNIX Technical Support Educationemail: [email protected]
IBM Training
© 2008 IBM Corporation
AgendaVirtual I/O configuration concepts
Describe and configure virtual Ethernet
Describe the new Integrated Virtual Ethernet (IVE) adapter– Architecture– Multi-Core Scaling
Configure external network access using IVE
Describe and configure external network access using Shared Ethernet adapter (SEA)
– With Link Aggregation– With SEA Failover
Describe and configure virtual SCSI– Configure new file-backed virtual devices
Conclusion
IBM Training
© 2008 IBM Corporation
Virtual I/O Configuration Concepts
On POWER5 and POWER6, virtual I/O devices include virtual Ethernet, Integrated Virtual Ethernet Adapter, Shared Ethernet
Adapter, and virtual SCSI devices
Virtual SCSI and Shared Ethernet adapters require the Advanced POWER Virtualization feature and a configured
Virtual I/O Server
Integrated Virtual Ethernet is a new POWER6 feature
The Virtual I/O Server is a dedicated special appliance partition
IBM Training
© 2008 IBM Corporation
Virtual Ethernet: POWER5 Hypervisor Ethernet SwitchThe POWER Hypervisor provides the Ethernet switch functionality for virtual Ethernet adapters on POWER5 and POWER6 systems:
– Implements a software Ethernet switch– Supports up to 4094 networks (VLAN IDs)– Copies packets between partitions (memory to memory)– Consistent with VLAN IEEE 802.1Q standard– The ports on this “switch” are configured via the HMC and are part of the
partition’s configuration
VLAN 1 VLAN 2
VLAN 3
LPAR 1 LPAR 2 LPAR 3 LPAR 4 LPAR 5
POWER Hypervisor (switch)
IBM Training
© 2008 IBM Corporation
Virtual Ethernet AdaptersVirtual Ethernet adapters are configured using the HMC or
IVM, in the partition’s profile or dynamically with DLPAR
Select the virtual slot number and PVID
• Configures the port for this slot on the
Hypervisor switch
Select additional VLANs• Allows the adapter to
communicate on multiple VLANS (up to 20 plus
PVID)• Requires additional
configuration in the LPAR
Virtual Ethernet interfaces are configured in AIX just like physical adapter interfaces
IBM Training
© 2008 IBM Corporation
Virtual Ethernet Demonstration
Configure and test a virtual network between two AIX LPARs
LPAR1 LPAR2
POWER Hypervisor
POWER6 System
ent0 ent0
en0 en0
VLAN 1
192.168.1.1 192.168.1.2
PVID=1ieee_virtual_eth=0
PVID=1ieee_virtual_eth=0
VIOS
IBM Training
© 2008 IBM Corporation
Integrated Virtual Ethernet (IVE)
X2
External Layer 2 or Layer 3 switch
Port Group 110 Gb or
1 Gb physicalport
1 Gb physical port
LPARs
Logical Switch Logical Switch
Three IVE models:• Dual-port Gigabit: 1 port group, 2 physical 1 Gb ports• Dual-port 10 Gigabit: 2 port groups, 1 physical 10 Gb port per port group• Quad-port Gigabit: 2 port groups, 2 physical 1 Gb ports per port group
IBM Training
© 2008 IBM Corporation
lheaent
OS
IVE PortsLogical ports are associated with a specific physical port
Port group:– Set of 16 logical ports on 1 or 2 physical ports:
• Logical ports can be split evenly between the 2 physical ports in a port group, or unevenly
– 1 or 2 port groups per Host Ethernet Adapter (HEA), depending on model– 1 or 2 physical ports per port group, depending on model, each with own Layer
2 switch
Port Group
External Switch
LogicalPort
PhysicalPort
Logical Switch HEA
IVE
Logical devices as they appear in AIX
LPAR
IBM Training
© 2008 IBM Corporation
Multi-Core Scaling (MCS)The MCS value sets the maximum number of logical ports for a port group
– Set per port group (regardless of number of physical ports)Use MCS to specify number of ports you need to support LPARs
– Default value is 4, so only 4 logical ports per port group available– Best IVE adapter performance when MCS value equal to # of processors
Must power off/on managed system to change MCS
Port Group
Switch
PhysicalSwitch
Port Group
Switch
PhysicalSwitch
MCS = 1 MCS = 4
4 ports16 ports
IBM Training
© 2008 IBM Corporation
Managed System IVE Configuration (1 of 2)1. Select the managed system, and choose:
– Hardware Information -> Adapters -> Host EthernetPhysical port
settings(actual)
LPARs which are configured for logical ports using this physical port
2. Choose which IVE adapter
3. Choose physical port
4. Click Configurebutton to configure
physical port
IBM Training
© 2008 IBM Corporation
A Virtual I/O Server partition must be listed as the promiscuous LPAR if it is using a port on the IVE as part of a Shared Ethernet adapter device
Managed System IVE Configuration (2 of 2)4. button is used to configure physical port settings:
MCS
Set to None or to a partition that will take entire physical port
Enabled means HMC
will attempt to negotiate flow
control for physical port
IBM Training
© 2008 IBM Corporation
LPAR IVE Configuration
Configure logical ports
from the LPAR profile or with
DLPAR
IBM Training
© 2008 IBM Corporation
IVE DemonstrationConfigure an IVE logical port on the VIOS LPAR
1. Check the configuration of the physical adapter2. Add a logical port for the VIOS LPAR
LPAR1 LPAR2
POWER Hypervisor
POWER6 System
ent0 ent0
en0 en0
VLAN 1
192.168.1.1 192.168.1.2
PVID=1ieee_virtual_eth=0
PVID=1ieee_virtual_eth=0
VIOS
HEA
P0 P1
lhea0
ent0
EthernetSwitch
IBM Training
© 2008 IBM Corporation
Shared Ethernet Adapter (SEA)Shared Ethernet adapter bridges external networks to internal VLANs:
– Forwards frames at OSI Layer 2 and is transparent to IP layer
Virtual I/O Server Partition
Layer 2 Bridge (Shared Ethernet adapter)
Device Driver Device Driver Device Driver
PhysicalAdapter
VirtualAdapter
VirtualAdapter
ExternalLANs
1-16 virtual adapters which connect to VLANs (and client
partitions)
IBM Training
© 2008 IBM Corporation
SEA Example
Shared Ethernet adapter is used to extend VLAN 2 and 3 using the physical Ethernet in the Virtual I/O Server
SharedEthernet
VLAN 3VLAN 3
VLAN 2VLAN 2
Virtual I/O Server
Managed systemEthernet Switch
Standalone serversLPAR 1, 2, 3
IBM Training
© 2008 IBM Corporation
Configure Virtual Ethernet Adapter for SEA
• Virtual adapters that will be associated with an SEA on the VIO Server must have the “Access external network”checkbox selected
• Specify a PVID and optionally any VLAN IDs that the SEA will extend to an external network
- PVIDs are stripped from frames in the SEA, so use a unique VLAN for PVID when supporting access to VLANs on an external network
Virtual Ethernet adapters and Hypervisor switch ports are configured here on the HMC/IVM
IBM Training
© 2008 IBM Corporation
Configure SEA Device in VIO ServerCommand to create Shared Ethernet Adapter:
$ mkvdev –sea ent0 –vadapter ent1 \–default ent1 –defaultid 1
Creates device (ent2 in this case):
$ lsdev –virtualname status description
ent1 Available Virtual I/O Ethernet Adapter (l-lan)
ent2 Available Shared Ethernet Adapter
PVID 1ent0 ent1
ent2 (SEA)
VIOSswitch
physicalHypervisor
virtual
IBM Training
© 2008 IBM Corporation
SEA DemonstrationConfigure an SEA adapter on the VIOS
1. Create a virtual Ethernet adapter on the VIOS LPAR that can access external networks
2. Create the SEA on the VIOS LPAR using the lhea port and the virtual adapter
EthernetSwitch
LPAR1 LPAR2
POWER Hypervisor
ent0 ent0
en0 en0
VLAN 1
192.168.1.1 192.168.1.2
PVID=1ieee_virtual_eth=0
VIOS
HEA
P0 P1
lhea0
ent0
ent1
PVID=1ieee_virtual_eth=0
POWER6 System
SEA
PVID=1ieee_virtual_eth=0
IBM Training
© 2008 IBM Corporation
SEA with Link AggregationCreate link aggregation Ethernet adapter:$ mkvdev –lnagg ent0 ent1
ent3 shows up as:$ lsdev –adapter | grep ent3ent3 Available EtherChannel / IEEE 802.3ad Link Aggregation
Create Shared Ethernet adapter:$ mkvdev –sea ent3 –vadapter ent2 \
–default ent2 –defaultid 1
ent2 PVID 1
(Physical)ent4 (SEA)
ent0
ent1
ent3
(Aggregate)
Virtual I/O Server
switch Hypervisor(Virtual)
IBM Training
© 2008 IBM Corporation
Virtual SCSI Overview
Physical Storage
S S S
PHYVirtual I/O Server
C
Client
C
Client
C
Client
VTDVTD
Hypervisor
PHY
PHY
PHY Physical Adapter
SVSCSI ServerVirtual Adapter
CVSCSI ClientVirtual Adapter
VTD Virtual Target Device
C
VTD
S
VTD
The red connections show two clients accessing the same physical storage (A) via two different server adapters (B) and virtual target devices (D)
VTD
PHY
The blue connection shows multiple target devices (D) attached to a single server adapter (B)
IBM Training
© 2008 IBM Corporation
Virtual SCSI Configuration (1 of 3)1) Define virtual SCSI server in VIO Server partition
and client adapter in AIX or Linux partition
2) Check availability of virtual SCSI server adapters on VIO Server:$ lsdev -virtualname status descriptionvasi0 Available Virtual Asynchronous Services
Interface (VASI)vhost0 Available Virtual SCSI Server Adaptervsa0 Available LPAR Virtual Serial Adapter
IBM Training
© 2008 IBM Corporation
Create vSCSI Adapters DemonstrationCreate virtual SCSI adapters on the VIOS and AIX LPARs
LPAR1
POWER Hypervisor
POWER6 System
VIOS
vscsi0vhost0
FC cardSAN
fcs0 fcs1
hdisk6
hdisk7
sas0
…hdisk5
hdisk0
Internal storage Optical device
cd0
IBM Training
© 2008 IBM Corporation
Virtual SCSI Configuration (2 of 3)3) On the VIO Server, define storage resources
To create a volume group:$ mkvg [ -f ][ -vg VolumeGroup ] PhysicalVolume ...
To create a logical volume:$ mklv [ -lv NewLogicalVolume | -prefix Prefix ] VolumeGroup Size [PhysicalVolume ... ]
To create a storage pool:$ mksp [-f] StoragePool PhysicalVolume ...
To create a backing device from available space in a storage pool:$ mkbdsp [-sp StoragePool] Size [-bd BackingDevice]
-vadapter ServerVirtualSCSIAdapter
IBM Training
© 2008 IBM Corporation
Virtual SCSI Configuration (3 of 3)4) On the VIO Server, define virtual target devices
5) Boot the client or run cfgmgr to use new virtual devices
$ mkvdev -vdev TargetDevice -vadapter VirtualServerAdapter [ -dev DeviceName ]For example:$ mkvdev –vdev hdisk3 –vadapter vhost0vtscsi0 Available$ mkvdev –vdev lv10 –vadapter vhost0vtscsi1 Available$ mkvdev –vdev cd0 –vadapter vhost0vtopt0 Available
Check the target devices with lsdev:$ lsdev -virtualname status descriptionvtscsi0 Available Virtual Target Device - Diskvtscsi1 Available Virtual Target Device - Logical Volumevtopt0 Available Virtual Target Device – Optical Media
IBM Training
© 2008 IBM Corporation
Use lsmap from the VIO Server to verify mapping of virtual targets:$ lsmap -vadapter vhost0
SVSA Physloc Client Partition ID
--------------- ----------------------------------- -----------------
vhost0 U9111.520.10F191F-V3-C6 0x00000003
VTD vtscsi0
LUN 0x8100000000000000
Backing device hdisk3
Physloc U787A.001.DNZ00G0-P1-T10-L8-L0
VTD vtscsi1
LUN 0x8200000000000000
Backing device lv10
Physloc
VTD vtopt0
LUN 0x8300000000000000
Backing device cd0
Physloc
View Configuration with lsmap
Client LPAR ID
Physical location code
Server slot ID
LUN ID
IBM Training
© 2008 IBM Corporation
View Configuration with lshwresUse lshwres from the HMC to see system-wide view of virtual I/O configuration (or view from HMC GUI)hscroot@skylab-hmc:~> lshwres -r virtualio --rsubtype scsi -m skylab
lpar_name=VIOS,lpar_id=1,slot_num=7,state=1,is_required=1,adapter_type=server remote_lpar_id=4,remote_lpar_name=node3,remote_slot_num=6, "backing_devices=drc_name=U787F.001.DPM0ZFL-P1-T10-L4-L0/log_unit_num=0x8100000000000000/ device_name=hdisk1,drc_name=U787F.001.DPM0ZFL-P1-T10-L5-L0/log_unit_num=0x820000000000000/“lpar_name=node3,lpar_id=4,slot_num=6,state=1,is_required=1,adapter_type=client,remote_lpar_id=1,remote_lpar_name=VIOS,remote_slot_num=7,backing_devices=none
IBM Training
© 2008 IBM Corporation
Virtual Target Device DemonstrationConfigure virtual target devices for the vSCSI adapter with different physical backing devices
clientVG
LPAR1
POWER Hypervisor
POWER6 System
VIOS
vscsi0vhost0
FC cardSAN
fcs0 fcs1
hdisk6
hdisk7
sas0
…
hdisk5
hdisk0
Internal storage Optical device
cd0
vtscsi1cl_lv
vtopt0
vtscsi0
cd0
hdisk0
hdisk1
IBM Training
© 2008 IBM Corporation
File-Backed Virtual DevicesFile-back (FB) virtual device types:
– File-backed disk devices• Files created in storage pools can be used as hdisk on client
– File-backed optical media devices
• Create a Virtual Media Repository which can be stocked with DVD-ROM/RAM media
• Clients can use images stored in repository as cd0 devices with media
FB virtual device characteristics:– Read-only FB devices can be shared by multiple clients
– Bootable FB devices appear in SMS
– Reside in FB storage pools
• Mount Directory = /var/vio/storagepools/<FBSP_Name>• LV_NAME = <FBSP_Name>
– Granularity as small as 1MB or as large as parent Logical Volume
FB virtual devices are new as of Virtual I/O Server V1.5
IBM Training
© 2008 IBM Corporation
Creating File-Backed Virtual DisksFiles on the virtual I/O Server can be used as backing storage:
1. Create a volume group (mkvg) or storage pool (mksp -f)
2. Create a FB disk storage pool (mksp -fb) inside volume group/storage pool
3. Create a device in the pool (mkbdsp) and map to a vadapter
4. The client associated with that vadapter sees new FB device as an hdisk
Volume Group/Storage Pool - contains hdisk(s)
FB Disk Storage Pool (contains FB virtual disks)
Target dev Target dev Target dev
IBM Training
© 2008 IBM Corporation
Create FB Virtual Disks Example (1 of 2)Create new volume group/logical volume storage pool:$ mkvg -vg newvg hdisk1 OR mksp -f newvg hdisk1
Create new FB storage pool in the logical volume storage pool:$ mksp -fb fbpool -sp newvg -size 10g
fbpool
File system created successfully.
10444276 kilobytes total disk space.
New File System size is 20971520
Create new file device with a certain size, create the VTD, and map to vhost adapter:$ mkbdsp -sp fbpool 30m -bd fb_disk1 -vadapter vhost3Creating file "fb_disk1" in storage pool "fbpool".
Assigning file "fb_disk1" as a backing device.
vtscsi3 Available
fb_disk1
New storage pool (newvg)
New FB storage pool (fbpool) that is 10 GB
inside of newvg
Create new 30 MB file called fb_disk1
Resulting VTD is named vtscsi3and is mapped to vhost3
IBM Training
© 2008 IBM Corporation
Create FB Virtual Disks Example (2 of 2)View mapping with new backing device:
$ lsmap -vadapter vhost3SVSA Physloc Client Partition
ID
--------------- ----------------------------- ------------------
vhost3 U8203.E4A.10CD1F1-V1-C15 0x00000000
VTD vtscsi3
Status Available
LUN 0x8100000000000000
Backing device /var/vio/storagepools/fbpool/fb_disk1
Physloc
IBM Training
© 2008 IBM Corporation
Create FB Virtual Optical Device (1 of 2)Create volume group/logical volume storage pool:$ mkvg -vg medrep hdisk4 OR mksp -f medrep hdisk1
Create 10 GB Virtual Media Repository in the LV pool:$ mkrep -sp medrep -size 10G
Virtual Media Repository Created
Repository created within "VMLibrary_LV" logical volume
Create media (aixopt1) in repository from a file:– Media could be blank, loaded from cd# device, or a file$ mkvopt -name aixopt1 -file dvd.product.iso -ro
New storage pool (medrep)
IBM Training
© 2008 IBM Corporation
Create FB Virtual Optical Device (2 of 2)View repository and its contents:
$ lsrepSize(mb) Free(mb) Parent Pool Parent Size Parent
Free
10198 6532 medrep 69888 59648
Name File Size Optical Access
aixopt1 3666 None ro
Create FB virtual optical device and map to vhost adapter:
$ mkvdev -fbo -vadapter vhost4vtopt0 Available
Load the image into the media device:– Use the unloadopt command to unload
$ loadopt -vtd vtopt0 -disk aixopt1
New VTD name
IBM Training
© 2008 IBM Corporation
Viewing FB Configuration from the HMC
HMC command line example:hmc:~> lshwres -m hurston -r virtualio --rsubtype scsi
lpar_name=VIOS,lpar_id=1,slot_num=16,state=1,is_required=0,adapter_type=server,remote_lpar_id=any,remote_lpar_name=,remote_slot_num=any,"backing_devices=""0x8100000000000000//""""/var/vio/VMLibrary/aixopt1""""""". . .
IBM Training
© 2008 IBM Corporation
FB Device Command Examples (1 of 2)List the repository and any contents:$ lsrep
Size(mb) Free(mb) Parent Pool Parent Size Parent Free
10198 6532 medrep 69888 59648
Name File Size Optical Access
aixopt1 3666 vtopt0 ro
List the storage pools:– Notice both LVPOOL and FBPOOL types:$ lssp
Pool Size(mb) Free(mb) Alloc Size(mb) BDs Type
rootvg 69888 44544 128 1 LVPOOL
NewVG 69888 59648 64 0 LVPOOL
medrep 69888 59648 64 0 LVPOOL
fbpool 10199 6072 64 2 FBPOOL
List out volume groups/storage pools (LVPOOL type only):$ lsvg
rootvg
NewVG
medrep
IBM Training
© 2008 IBM Corporation
FB Device Command Examples (2 of 2)List LVPOOL details:$ lssp -detail -sp NewVGName PVID Size(mb)hdisk3 000cd1f195f987df 69888
List FBPOOL details:$ lssp -bd -sp fbpoolName Size(mb) VTD SVSAfb_disk1 30 vtscsi3 vhost3fb_disk2 4096 vtscsi4 vhost3
Show all mounts including FB devices:$ mountnode mounted mounted over vfs date options
-------- --------------- --------------- ------ ------------ ---------------/dev/hd4 / jfs2 Apr 18 13:01 rw,log=/dev/hd8/dev/hd2 /usr jfs2 Apr 18 13:01 rw,log=/dev/hd8/dev/hd9var /var jfs2 Apr 18 13:01 rw,log=/dev/hd8/dev/hd3 /tmp jfs2 Apr 18 13:01 rw,log=/dev/hd8/dev/hd1 /home jfs2 Apr 18 13:01 rw,log=/dev/hd8/proc /proc procfs Apr 18 13:01 rw/dev/hd10opt /opt jfs2 Apr 18 13:01 rw,log=/dev/hd8/dev/fbpool /var/vio/storagepools/fbpool jfs2 Apr 28 12:04 rw,log=INLINE/dev/VMLibrary_LV /var/vio/VMLibrary jfs2 Apr 28 14:36 rw,log=INLINE
IBM Training
© 2008 IBM Corporation
File-Backed Virtual Devices DemonstrationConfigure a file-backed virtual disk and file-backed virtual optical device
VIOS LPAR1
vhost1 vscsi1
rootvg
hdisk0
medrep(Virtual Media Repository)
AIX53_isoAIX61_iso
cl_mksysb
hdisk1
stpool1(LV storage pool)
fbpool1(FB storagepool)
fb_disk1fb_disk2
vtscsi2
vtopt1
POWER Hypervisor
hdisk2
cd1
IBM Training
© 2008 IBM Corporation
ConclusionVirtual I/O devices on POWER5 and POWER6 systems include virtual Ethernet, Integrated Virtual Ethernet (IVE), Shared Ethernet (SEA), and virtual SCSI
Virtual Ethernet and IVE, are part of the base feature and do not require PowerVM (or Advanced Power Virtualization for POWER5)
– Devices are defined in the HMC– Configuration in AIX is same as for physical devices
Virtual SCSI and SEA devices extend the use of physical devices to multiple clients using the Virtual I/O Server and require the PowerVM feature
– Create virtual devices with mkvdev on VIO Server command line to bridge virtual Ethernet adapters (mkvdev –sea) and to associate virtual SCSI adapters with physical storage
File-backed virtual disks and optical devices provide even more flexibility in storage allocation
IBM Training
© 2008 IBM Corporation
39
TrademarksThe following are trademarks of the International Business Machines Corporation in the United States, other countries, or both.
The following are trademarks or registered trademarks of other companies.
* All other products may be trademarks or registered trademarks of their respective companies.
Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workloadprocessed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.
Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce.
For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml:
*, AS/400®, e business(logo)®, DBE, ESCO, eServer, FICON, IBM®, IBM (logo)®, iSeries®, MVS, OS/390®, pSeries®, RS/6000®, S/30, VM/ESA®, VSE/ESA, WebSphere®, xSeries®, z/OS®, zSeries®, z/VM®, System i, System i5, System p, System p5, System x, System z, System z9®, BladeCenter®
Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not actively marketed or is not significant within its relevant market.
Those trademarks followed by ® are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States.