implementing bladecenter boot disk system solution
TRANSCRIPT
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 1
Implementing the IBM BladeCenter
Boot Disk System Solution
Khalid M. Ansari Advanced Technical Support [email protected] Document Version January 2, 2008
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 2
Note:
Copyright © 2007 by IBM Corporation All rights reserved. Information has been obtained from sources assumed to be reliable and reflect conclusions at the time. Although the document may utilize publicly available material from various vendors, including IBM, it does not necessarily reflect the positions of such vendors on the issues addressed in this document. Material contained and conclusions presented in this document are subject to change without notice. All warranties as to the accuracy, completeness or adequacy of such material are disclaimed. There shall be no liability for errors, omissions or inadequacies in the material contained in this document or for interpretations thereof. Trademarks included in this document are the property of their respective owners.
IBM, the IBM logo, the e-business logo, BladeCenter, Boot Disk System, DS8300, and xSeries are trademarks of IBM Corporation in the United States and/or other countries. Microsoft, Windows, Windows 2000, Widows 2003, and the Windows logo are trademarks or registered trademarks of Microsoft Corporation. Linux is a trademark of Linus Torvalds in the U.S. and other countries. Other company, product, and service names may be trademarks or service marks of others.
This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 3
Table of Contents
Table of Contents ........................................................................................................ 3
Introduction.................................................................................................................. 7
Serial Attached SCSI overview.................................................................................... 7
Overview of IBM BladeCenter Boot Disk System....................................................... 8
Cost reduction ............................................................................................................. 8
High availability............................................................................................................ 8
Centralized management............................................................................................. 8
Redundancy ................................................................................................................ 8
Performance ................................................................................................................ 9
Implementing Boot from SAS .................................................................................... 10
SAS zoning................................................................................................................ 11
Why zoning................................................................................................................ 11
BladeCenter Boot Disk System Management............................................................ 12
Supported Hardware and software configuration....................................................... 12
Blade Boot from Boot Disk System SAS implementation ....................................... 14
Important considerations ........................................................................................... 14 Boot device selection ......................................................................................14
SAS zoning................................................................................................................ 15 Connecting the BladeCenter Boot Disk System to SAS connectivity modules..........15
Configuration checklist................................................. Error! Bookmark not defined.
IBM BladeCenter Boot Disk Solution Configuration Overview ................................... 17
BladeCenter Boot Disk system configuration for Windows 2003 and RHEL 4 U4 ..... 19 Configuring the primary boot path from the LS21 blade BIOS ..............................19
Configure the LS21 blade server LSI SAS BIOS ....................................................... 20 Configuring the SAS connectivity module ..........................................................22
Zone configuration with the SAS connectivity module ............................................... 23
Boot Disk System storage configuration .................................................................... 26 Create logical drives .......................................................................................28 Configure host access (manual) .......................................................................34 Create host-to-logical drive mappings...............................................................41
Installing the operating system .................................................................................. 45 Installing the Windows 2003 operating system ..................................................45
Add the secondary path from the host to the boot LUN............................................. 51
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 4
Enable failover for the Boot LUN ............................................................................... 51 SAS connectivity module configuration..............................................................51 Boot Disk System storage configuration ............................................................53 Host configuration ..........................................................................................56 Failover verification ........................................................................................61
Installing RHEL 4 Update 4 operating system ........................................................... 63 Failover verification ........................................................................................73
Appendix: Useful links ............................................................................................... 76
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 5
Table of Figures Figure 1: Blade Boot from Boot Disk System SAS Architecture .................................... 10 Figure 2: NVRAM Host Type Definitions and AVT Status ............................................. 11 Figure 3: BladeCenter Boot Disk System Connectivity to SAS connectivity modules in
Bays 3 & 4.............................................................................................................. 16 Figure 4 Disable Planar SAS......................................................................................... 19 Figure 5 Disable Planar SCSI ....................................................................................... 20 Figure 6: LSI MPT SAS BIOS ....................................................................................... 21 Figure 7: SAS Adapter BIOS......................................................................................... 21 Figure 8: SAS Host Bus Adapter BIOS Menu ............................................................... 22 Figure 9: BladeCenter Advance Management Module � I/O Module Tasks Menu ...... 23 Figure 10: Zone Configuration Management................................................................. 24 Figure 11: Active Zone Configuration............................................................................ 25 Figure 12: SAN Fabric Layout for Primary Path ............................................................ 26 Figure 13 Boot Disk System Storage Manager GUI...................................................... 27 Figure 14: Boot Disk System Summary View............................................................... 28 Figure 15: Boot Disk System Configure Storage Menu................................................. 29 Figure 16: No Hosts Configured Warning Message ...................................................... 29 Figure 17: Create Logical Drives � Select Capacity Type ........................................... 30 Figure 18: Create Logical Drives � Automatic Physical Drive...................................... 30 Figure 19: Create Logical Drives � Select RAID level ................................................. 31 Figure 20: Create Logical Drives � Specify Drive Size ................................................ 32 Figure 21: Create Logical Drives �Assign LUN ID =0.................................................. 33 Figure 22: Create Logical Drives � Complete.............................................................. 33 Figure 23: Summary View of the Boot Disk System Boot Device................................. 34 Figure 24: Configure Host Access................................................................................. 35 Figure 25: Configure Host Access � Specify Host Name & Type ................................ 35 Figure 26: Configure Host Access � Specify Host Port ............................................... 36 Figure 27: Configure Host Access � Specify Alias for Host WWPN ............................ 36 Figure 28: Configure Host Access Process................................................................... 37 Figure 29: Non-Clustered Host...................................................................................... 37 Figure 30: Configure Host Access � Confirm Host Definition ...................................... 38 Figure 31: Configure Host Access � Complete............................................................ 39 Figure 32: Create Host Group ....................................................................................... 40 Figure 33: Create Host Group � Add Host................................................................... 40 Figure 34: Create Host Group � Complete .................................................................. 41 Figure 35: Create Host-to-Logical Drive Mappings Option............................................ 42 Figure 36: Create Host-to-Logical Drive Mapping –Select Host .................................... 42 Figure 37: Create Host-to-Logical Drive Mapping � Select Logical Drives .................. 43 Figure 38: Create Host-to-Logical Drive Mapping � Completed .................................. 44 Figure 39: Create the Create Host-to-Logical Drive Mapping � Complete .................. 44 Figure 40: Create Host-to-Logical Drive Mapping ......................................................... 45 Figure 41: AMM� Configure Blade Boot Sequence..................................................... 46 Figure 42 LSI MPT SAS Banner ................................................................................... 47 Figure 43: Windows Server Initial Install Screen ........................................................... 47 Figure 44: Windows Server Select SCSI Adapter Screen............................................. 48
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 6
Figure 45: Windows Setup is Loading SAS Device Driver ............................................ 49 Figure 46: Windows Setup Loads MPT SAS Driver ...................................................... 50 Figure 47: Windows 2003 Setup Completed Successfully ............................................ 50 Figure 48: Dual Paths from Blade to the Boot Disk System .......................................... 51 Figure 49: Monitor SAS Module � Blade 13 ................................................................ 52 Figure 50: Active Zone Configuration............................................................................ 53 Figure 51: Edit Host Topology....................................................................................... 54 Figure 52: Edit Host Topology � Add HBA Option....................................................... 54 Figure 53: Edit Host Topology � WWPN, Host Type ................................................... 55 Figure 54: Edit Host Topology � Add Option ............................................................... 55 Figure 55: Edit Host Topology Complete....................................................................... 56 Figure 56: Windows 2003 Computer Management � Disk Management..................... 57 Figure 57: the Microsoft MPIO DSM Install ................................................................... 58 Figure 58: the Microsoft MPIO DSM Install Folder ........................................................ 58 Figure 59: RDAC Install Completed Successfully ......................................................... 59 Figure 60 Windows 2003 Disk Manager........................................................................ 60 Figure 61: Windows 2004 Device Manager � Multipath Disk Device .......................... 60 Figure 62: Verify Current LUN Owner ........................................................................... 61 Figure 63: Boot Disk System Profile � Current LUN Owner ........................................ 62 Figure 64: Rhel4 U4 Install Method............................................................................... 63 Figure 65: Driver Disk Source ....................................................................................... 64 Figure 66: Insert Driver Disk.......................................................................................... 65 Figure 67: Reading Driver Disk ..................................................................................... 65 Figure 68: Automatic Partitioning .................................................................................. 66 Figure 69: Disk Setup.................................................................................................... 67 Figure 70: Software Selection ....................................................................................... 68 Figure 71: LSI SAS BIOS Banner ................................................................................. 73 Figure 72: Verify Current LUN Owner ........................................................................... 74 Figure 73: Verify LUN Failover ...................................................................................... 75
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 7
Introduction
This document illustrates the step-by-step procedure to deploy an IBM® blade server attached to IBM BladeCenter® Boot Disk System for Windows® 2003 Enterprise Server and Red Hat® Enterprise Linux® 4 Update 4 (RHEL4 U4) x64 operating systems.
Serial Attached SCSI overview
Serial Attached SCSI (SAS) is an emerging storage protocol offering entry-level storage fabric capabilities at cost competitive prices. SAS uses serial point-to-point connections through SAS connectivity module called expanders to provide any port to any port connectivity. Moreover, with the advent of SAS zoning, the connectivity can be controlled at the solution management level. SAS is leveraging the historical efforts of ATA (AT Attachment) and SCSI to build an economical storage solution with capabilities beyond the single server paradigm.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 8
I. Overview of IBM BladeCenter Boot Disk System
The following sections discuss the benefits and best practices to implement the BladeCenter Boot Disk System solution The BladeCenter Boot Disk System solution reduces cost, provides high availability, helps consolidate the IT infrastructure, reduces cabling complexity, minimizes the downtime window, and enhances performance.
Cost reduction
The BladeCenter architecture helps cut the equipment costs by reducing the number of moving parts such as fans, power supply, and internal disk drives when compared with the stand-alone servers. The BladeCenter architecture provides the capability to share the resources such as power, cooling, cabling, and I/O paths among all the servers within the chassis plus facilitates centralized management.
High availability
The system downtime is greatly minimized in situations where a critical component such as a processor, memory, or the system planar fails and needs to be replaced. The system administrator needs to swap only the hardware with similar hardware and install the same HBA (SAS Host Bus Adapter) on the blade server -- thus making the system quickly available for production. In case of the HBA failure, the system administrator must replace the failed HBA and modify the zoning to include the WWPN (worldwide port name) of the new HBA in the active zoneset and update the host port definition on the storage subsystem to make the system operational. Additionally, the user must ensure that the latest and compatible firmware and BIOS is installed on the SAS HBA. The system image resides on the RAID-enabled logical drive on the Boot Disk System storage subsystem.
Centralized management
Centralized management can be achieved with a single blade server within the same chassis with management applications such as IBM Director/RDM installed for remote deployment of the OS (operating system) or firmware updates for device peripherals over the local area network (LAN).
Redundancy
End-to-end redundancy is established from the host to the Boot Disk System storage. The dual port HBA, redundant SAS connectivity modules in the chassis,
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 9
and a maximum of three paths on each Boot Disk System storage controller enables load balancing across the multiple paths. The The Microsoft MPIO Device Specific Module (DSM) for Windows and the RDAC multipath driver for Linux and the Auto Volume Transfer feature on the storage subsystems detect path failures and reroute the I/O via the alternate path. In addition to the redundant architecture of the solution, the RAID (redundant array of inexpensive disks) technology -- that is, RAID0 (striping and no mirroring) RAID 10 (mirroring and striping), RAID 3 (striping and all parity stored on one disk per array), and RAID 5 (striping and parity on all disks in an array) -- protect against data loss if only one disk per array has failed. Thus, RAID 5 plus a spare drive per array provides adequate protection against data loss and drive failures.
Performance
Enhanced performance can be achieved by exploiting the existing SAS bandwidth --a maximum speed of 3 Gb/s end to end -- in conjunction with load balancing using the Microsoft MPIO Device Specific Module (DSM) for Windows and the RDAC multipath driver for Linux operating systems. The Boot Disk System storage subsystem operates in an active/passive configuration at the LUN level, which means a LUN is always available via single or multiple paths on the active controller (preferred owner of that LUN). If the primary path fails, then the LUN is moved to the backup path, regardless of the failing point -- i.e., the host, switch, or storage.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 10
II. Implementing Boot from SAS
The following sections briefly cover the workings of the BladeCenter Boot Disk System solution and the best practice procedures that ensure you can seamlessly integrate various components in addition to cleanly deploying the solution.
Figure 1: Blade Boot from Boot Disk System SAS Arch itecture
The architectural view in Figure 1 illustrates the most common software stack installed on the host for successful boot from the BladeCenter Boot Disk System. The SAS BIOS is enabled on the HBA by default, thus no configuration is required in the HBA BIOS. After the connectivity from the host to the storage is established and the logical drive (boot LUN) is mapped to the host, the SAS BIOS sees the boot LUN and configures it as the first boot device, only if the local disk controller is disabled. If there are local disks installed then the SAS HBA should be selected as the first device in the boot order. The PCI Device Boot Priority is configured from LSI MPT SAS BIOS by invoking the LSI configuration menu using the Crtl+C keys. After you successfully install the operating system via the single and unique path, you can add additional paths from the host to the storage. The Microsoft MPIO Device Specific Module (DSM) or Redundant Disk Array Controller (RDAC) multipath driver for Linux is then installed to provide failover and multipath capability to the boot LUN. The MPIO driver maintains the path information, and if a link failure is detected on the active path, it notifies the storage controller to move the LUN(s) to the alternate path.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 11
Detecting a failed link and reconfiguring the new path occurs with minimum delay before the I/O is rerouted through the alternate path and the system becomes available to the user. If the link failure is detected during system initialization (i.e., before the SAS and MPIO driver is initialized on the host), then the Auto Volume Transfer (AVT) function or Auto Disk Transfer (ADT) if enabled on the boot LUN will move the LUN to the alternate path. This enablement is based on the host type selection used by the BladeCenter Boot Disk System controller to move the LUN to the alternate path -- after it detects the link failure on the preferred path. The following Figure 2 shows the default NVRAM settings (AVT Status) on the BladeCenter Boot Disk System:
Figure 2: NVRAM Host Type Definitions and AVT Statu s
1. SAS zoning
SAS fabric zoning allows the server administrator to create a virtual SAS fabric within the physical SAS fabric. These virtual SAS fabrics, similar to Ethernet VLANs, allow the server administrator to restrict the SAS device visibility on a per port or device level.
2. Why zoning
Zoning segregates devices at the fabric level by creating smaller virtual domains within the fabric. Zoning prohibits access between devices within the same logical fabric. For a host (initiator) to gain access to the storage subsystem (target), the initiator HBA WWPN or the switch port to which it is connected must be zoned with the corresponding target WWPN, or the switch port and this zone should be a member of the active zoneset. Thus, although zoning is a tool to permit or deny access to the devices
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 12
connected to the fabric, it does not have the intelligence to apply controls beyond the fabric, i.e., to present or hide the LUN to certain hosts.
3. BladeCenter Boot Disk System Management
The BladeCenter Boot Disk System can be managed directly through the out of band Ethernet connections on each controller. The following of the network configurations options are can be used:
� DHCP server � Static IP Note: The Default Ethernet port IP Address for
o controller A: 192.168.128.101 o controller B: 192.168.128.102 o SubNet Mask: 255.255.255.0
Establish the management connections to the Boot Disk System via the default IP address and use the Configure Ethernet Management Port option under the Tools tab of the subsystem management window to change the controller default IP addresses to static IP addresses. For more information, refer to the IBM BladeCenter Boot Disk System Installation, User's and Maintenance Guide or the IBM System Storage System Storage DS3000 Storage Manager Version 2 Installation and Support Guide for Microsoft Windows Server 2003, Linux, Novell NetWare, and VMware ESX Server Guide.
4. Supported Hardware and software configuration
The following table lists the hardware and the minimum software levels required to successfully implement the BladeCenter Boot Disk System environment.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 13
Hardware configuration Minimum Software levels required
BladeCenter HS20 type 7981 BIOS: 10A BMC: 08A
BladeCenter HS20 type 8843 BIOS: 28A BMC: 35A
BladeCenter HS21 type 8853 BIOS: 21A BMC: 36A
BladeCenter HS21 type 7995 BIOS: 12A BMC: 13A
BladeCenter LS20 type 8850 BIOS: 26A BMC: 28A
BladeCenter LS21 type 7991 BIOS: 26A BMC: 36A
BladeCenter JS21 type 8844 246_060_009
BladeCenter advance management module Firmware: v32B
SAS connectivity module Firmware: v3.27
Boot Disk System Storage Manager Firmware: 06.30.01.00IOC: 11.116.00 (included in the firmware) NVSRAM: N1726D320R230V03
System Storage™ DS3000 Storage Manager Client (to manage the boot disk system)
Version: 02.70.x5.05, where x is the version of a particular operating system
BladeCenter SAS Expansion Card Firmware:1.23.03, BIOS: 6.18.01, Fcode : 1.00.47
Windows 2003 Device driver: 1.25.10.07
Windows MPIO/DSM for Multipathing MPIO DSM is 1.01.35.52
Windows 2003 x64 Device driver: 1.25.10.07 x64
Red Hat 4 Update 4 and SUSE Linux Enterprise Server 9 SP3
Device driver: 3.12.19.00-1
SUSE Linux Enterprise Server 10 Device driver: 4.00.13.10-1
Linux RDAC for all Linux systems (32 bit, x64, LoP)
09.01.B5.77
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 14
III. Blade Boot from Boot Disk System SAS implementation
Important considerations
Following are some important configuration considerations with Boot Device selection, SAS storage cabling, and SAS zoning before you proceed to implement the solution.
Boot device selection
Because the SAS HBA does not require Boot Device selection from the LSI BIOS, it is important to understand how to configure the boot sequence in scenarios where either controller A or controller B is the preferred owner of the boot LUN and for environments with drives installed on the blade for page file or data storage, etc. The following configurations A & B detail the scan and boot order.
Configuration A – No local SAS disks on the HS21(8853) blade server and Planar SAS is
Disabled – LSI SAS HBA mapped to single 15 GB boot LUN on Boot Disk System
• Boot LUN is mapped to both SAS ports of the HBA – LSI SAS HBA is the only device listed in the boot order
� Scan order – LSI 1064 SAS HBA
• Finds 15 GB LUN via path A (HBA port 1 to switch in bay 3 to controller A on external port 3)
Finds 15 GB LUN via path B (HBA port 2 to switch in bay4 to controller B on external port 3)
� Blade boot order
– If controller A is the preferred and current LUN Owner • Set the boot sequence to Floppy, CD, HD0, HD1
– If controller B is the preferred and current LUN Owner • Set the boot sequence to floppy, CD, HD1, HD0
– Permits boot failover via AVT (Automatic Volume Transfer) support on Boot Disk System boot model
• If primary path is not available, the blade will try the alternate path
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 15
Configuration B – No local SAS disks on the HS21(8853) blade server and Planar SAS is
Disabled – LSI SAS HBA mapped to LUN0 & LUN1 on the Boot Disk System, where
LUN0 MUST be the boot LUN and LUN1 is used for Paging. • LUN0 & LUN1 are mapped to both SAS ports of the HBA
– LSI SAS HBA has the higher boot order priority
� Scan order – LSI 1064 SAS HBA
• Finds LUN1 & LUN2 via path A (HBA port 1 to switch in bay 3 to controller A on external port 3)
• Finds 1 LUN1 & LUN2 via path B (port 2 to switch 4 to controller B) – Total of 4 disks found in the above order
� Blade boot order – If LUNs are owned by controller A
• Change to floppy, CD, HD0, HD1 – If LUN is owned by controller B
• Change to floppy, CD, HD2, HD3 – Permits boot failover via AVT (Automatic Volume Transfer) support on
Boot Disk System boot model • If primary path is not available, the blade will try the backup path.
SAS zoning
The following zone configuration rules are applied on the SAS connectivity modules installed in all BladeCenter chassis except the BC-S chassis.
� The default zone is active by default. � Default zone details
– Each internal blade port can exclusively see all external ports. – All external ports can see all other external ports. – Each internal blade port is isolated with the other internal blade ports.
� For all diskless boot configurations, the default z one is recommended. – Custom zones can be created, but are not required.
Connecting the BladeCenter Boot Disk System to SAS connectivity modules
• Connect the cables from the BladeCenter Boot Disk System controllers A and B to the external port# 3 of the two SAS connectivity module modules as shown in the following Figure 3:
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 16
Figure 3: BladeCenter Boot Disk System Connectivity to SAS connectivity modules in Bays 3 & 4
Important: At this time, only external port# 3 is supported for BladeCenter Boot Disk
system connectivity to the SAS connectivity modules. External Ports 1 & 2 are reserved for future use. External Port 4 is used to connect to SAS connectivity module on another BCH chassis.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 17
IV. IBM BladeCenter Boot Disk Solution Configuration Overview
Accessing the IBM BladeCenter Advanced Management M odule: 1. Open a Web browser from the management station and log into the IBM
BladeCenter AMM. 2. Open the storage I/O Module configuration window, then complete the following
tasks: a. For the SAS Connectivity Module in I/O Module in bay 3, enable external
ports to connect to the BDS Controller A. b. For the SAS Connectivity Module in I/O Module in bay 4, complete
configurations steps 3 through 6, then perform step 7. Accessing the IBM Boot Disk System: 3. Using the IBM DS3000™ SMC program, create a logical drive that is assigned to the
blade server as the boot operating system disk. If the logical drive is created using a new RAID array, you must define the RAID level and the number of drives that will make up the RAID array. Otherwise, you can define the logical drives by using the free capacity in the existing RAID array. For automation help, please refer to resources in the table.
4. Using the DS3000 SMC program, complete the following steps for each blade server that has a boot operating system LUN defined in the boot disk system:
a. Define the host. b. Define the host port. c. Specify the applicable operating system for each host.
Note that at this point, only one host port is defined. Accessing the designated IBM BladeCenter Blades: 5. Restart the blade server and press F1 to enable configuration. Complete the
following tasks: a. Disable the integrated SAS controller. b. Set the boot order for the blade server-based BDS system controller if you
have only one logical drive from the boot disk system mapped to the blade server:
i. For blade servers using controller A: usb, HDD0, HDD1 ii. For blade servers using controller B: usb, HDD1, HDD0
c. Save the configuration changes and exit.
6. Repeat steps 5.1 through 5.3 for each blade server that has a boot operating system LUN defined in the boot disk system. Note: If you have more than one logical drive mapped to the blade server, refer to the
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 18
a. Restart the blade server and start the operating system installation using the following guidelines:
b. Use the applicable operating system installation procedures. c. Depending on the operating system, you might need the SAS expansion
card device driver diskette. d. Install the SAS multipath driver after the operating system installation is
completed.
7. Repeat step 6.1 through 6.3 for each blade server that has a boot operating system LUN defined in the boot disk system.
Returning to the IBM BladeCenter Advanced Managemen t Module: 8. From the Storage I/O Module Configuration window, complete the following tasks:
a. For the SAS Connectivity Module in I/O bay 4, enable the external ports for connectivity to the BDS controller B.
Returning to the IBM Boot Disk System: 9. Wait 5 minutes to allow the boot disk system controller B to discover the second
SAS expansion card host port in the blade server through the SAS connectivity module in I/O-module bay 4.
10. Using the DS3000 SMC program, complete the following tasks for each blade server that has a boot operating system LUN defined in the boot disk system:
a. Define the second SAS host port for each blade server. b. Specify the applicable operating system for each host.
Returning to the designated IBM BladeCenter Blades: 11. Restart the blade servers so that the multipath driver recognizes both paths. The
blade servers are now using the boot disk system for booting and operating system activities.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 19
V. BladeCenter Boot Disk system configuration for Windows 2003 and RHEL 4 U4
Configuring the primary boot path from the LS21 bla de BIOS
The following steps illustrate the procedure for booting a BladeCenter LS21 blade from the BladeCenter Boot Disk system using Windows 2003 Advanced Server or Enterprise Server Edition. 1. Connect the BladeCenter Boot Disk System to the SAS connectivity module fabric
as shown in the Boot Disk System and the SAS connectivity module Installation and User Guide.
2. Turn on the blade server and interrupt the boot sequence to enter the LS21 blade system BIOS by pressing the F1 key.
3. Select Devices and I/O ports from the main menu. a. Disable the Planar SAS for the HS21/LS21/LS41 blades as shown in
Figure 4 below: Note : This step is recommended to ease the initial OS installation on the boot LUN via the single path.
Figure 4 Disable Planar SAS
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 20
b. To disable the local SCSI planar for the HS20/LS20 (8843/8850) blade servers, see the following example in Figure 5:
Figure 5 Disable Planar SCSI
4. Press ESC to exit out of this menu, and Save the changes.
Configure the LS21 blade server LSI SAS BIOS
1. Turn on or restart the HS21 blade.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 21
2. Press <Ctrl + C> to enter the MPT SAS BIOS configuration utility as shown in the following Figure 6:
Figure 6: LSI MPT SAS BIOS
3. The following menu is displayed after pressing Crtl + C in the preceding step to
invoke the SAS Configuration Utility Adapter menu as shown in Figure 7.
Figure 7: SAS Adapter BIOS
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 22
4. Press the Enter key to select the SAS adapter internally connected to the SAS
connectivity module in bay 3 as shown in Figure 8 below:
Figure 8: SAS Host Bus Adapter BIOS Menu
5. Record the Worldwide Port Name (WWPN) of the first port on the SAS host bus
adapter. The WWPN is needed for defining host ports on the Boot Disk System Storage subsystem. The WWPN can also be retrieved from the switch Web interface.
Note: In Figure 5, notice that the Boot Support is set to Enabled BIOS & OS . Unlike Fibre Channel or iSCSI, this parameter is Enabled by default.
6. Configuring the SAS connectivity module
• Enable External Ports : From the BladeCenter Management Module GUI ���� IO Module Tasks ���� Admin/Power , ensure that the external ports for the I/O modules in bays 3 and 4 are enabled as shown in the following Figure 9.
Note: If the external ports are set to disabled from the AMM Advanced setup
menu, the external ports on the switch will always initialize as persistent disabled following the switch reboot.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 23
Figure 9: BladeCenter Advance Management Module ���� I/O Module Tasks Menu
Note: Port-based zoning is the only option currently supported on the BladeCenter SAS connectivity module modules. WWPN zoning will be available in the future.
Zone configuration with the SAS connectivity module
Several predefined SAS fabric zone configurations are available from the factory and can be invoked by a simple selection from the Advanced Management Module (AMM). Zoning on the SAS connectivity module can be performed using the AMM I/O module configuration option, Web GUI, SAS, telnet and SAS connectivity module (SCM) application.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 24
1. Pre-defined zone configuration options:
You can select from five pre-defined zone configuration options. In this example, option# 5 “Pre-Defined” zone configuration is currently active (indicated as ' ') as shown in Figure 10. With this option, each server bay is exclusively zoned with all the external ports, thus allowing access to one or more storage controller ports connected to the SAS connectivity module.
This configuration is applicable to BCE, BCH, BCT, and BCHT chassis with the AMM module installed. Each blade bay (port) is exclusively zoned with all the external ports. The following example lists the active zone configuration. Because the blade in slot 13 is used for this test, it belongs to a zone group that also has four external ports in its group.
Figure 10: Zone Configuration Management
The following example in Figure 11 was captured from the SAS connectivity module Web interface. It lists the Basic Zone Permission table for the LS21 blade in slot 13. The blade in slot 13 is zoned with four external ports. Notice that the External port is set to True under the Connected column and Normal under the Status column. This is because the Boot Disk System Controller A port is connected to the external port# 3 for
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 25
this test. There is only one external connection for Controller A to the SAS connectivity module in Bay1. Basic Zone Permission Table
Figure 11: Active Zone Configuration
Note: Notice in Error! Reference source not found. that there is only a single path
active from the Boot Disk System controller to the switch. The second path from BladeCenter Boot Disk System controller B to the switch is purposely disabled for the OS installation process to complete smoothly via the single path to the boot LUN.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 26
You can disable the second path from the host to the boot LUN in one of the
following ways:
a. Disable the switch’s internal port using the CLI as shown below:
MAIN> sasport disable b13 B14 Blade Slot Connection 13 OK - Port disable request has been queued. b. Disable external ports from the BladeCenter Management Module GUI
���� IO Module Tasks ���� Admin/Power . This option is recommended for deploying large quantities of blade servers so that all the external ports can be disabled at once. The AVT-enabled feature on the Boot Disk System will move the LUNs owned by the controller on the disabled path to a non-preferred path.
Figure 12: SAN Fabric Layout for Primary Path
Boot Disk System storage configuration
1. Download and install the latest BladeCenter Boot Disk System Storage Manager client utility. For installation instructions, refer to the BladeCenter Boot Disk System Storage Manager Install Guide.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 27
2. Access the Boot Disk System storage subsystem using the GUI-based Storage Management utility. The following Figure 13 lists the Boot Disk System subsystem accessible from the Storage Manager client GUI:
Figure 13 Boot Disk System Storage Manager GUI
3. Double-click the name of the Boot Disk System subsystem listed in the right pane
of the Storage Management window to open the Boot Disk System Summary view shown in Figure 14. Select the Configure tab to configure the storage subsystem.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 28
Figure 14: Boot Disk System Summary View
Create logical drives
1. Click the Create Logical Drive option shown in Figure 15.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 29
Figure 15: Boot Disk System Configure Storage Menu
2. Select the OK option and ignore the message. The host group and hosts will be
configured after the boot logical drive is created.
Figure 16: No Hosts Configured Warning Message
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 30
3. Select the Unconfigured Capacity radio button and then the Next option as
shown in Figure 17.
BCE
Figure 17: Create Logical Drives ���� Select Capacity Type
4. Select the Automatic radio button (recommended option), then select Next as shown in Figure 18. This option will automatically select the physical drives to create a logical drive.
Figure 18: Create Logical Drives ���� Automatic Physical Drive
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 31
5. Select the RAID level from the drop-down list as shown in Figure 19. In the
example below, RAID5 is selected for the boot LUN.
Figure 19: Create Logical Drives ���� Select RAID level
6. Specify the size for the boot LUN as shown in Figure 20, then select Next .
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 32
Figure 20: Create Logical Drives ���� Specify Drive Size
7. Specify the boot LUN ID =0 and leave the Map Now radio button unselected as
shown in Figure 21.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 33
Figure 21: Create Logical Drives ����Assign LUN ID =0
8. Select option No to the question shown in Figure 22 to complete the boot LUN
creation process. If you need to create multiple logical drives, then you can select Yes. For this test, only one logical drive is required.
Figure 22: Create Logical Drives ���� Complete
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 34
9. The following Figure 23 is the summary view, and it now lists a logical drive created in the preceding steps.
Figure 23: Summary View of the Boot Disk System Bo ot Device
Configure host access (manual)
1. Select the Configure Host Access manual option listed from the Configure menu shown in the following Figure 24
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 35
Figure 24: Configure Host Access
2. Specify the host name and select the host type from the drop down menu as
shown in Figure 25.
Figure 25: Configure Host Access ���� Specify Host Name & Type
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 36
3. Select the WWPN of the first port on the SAS HBA, then select Add as shown in Figure 26.
Figure 26: Configure Host Access ���� Specify Host Port
4. Select the Edit option to specify the Alias for the first WWPN of the SAS HBA
and select OK as shown in Figure 27.
Figure 27: Configure Host Access ���� Specify Alias for Host WWPN
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 37
5. The following figure shows the WWPN identifier for the host ls21blade13 and
host port alias ls21blade14p1. Select Next as shown in Figure 28.
Figure 28: Configure Host Access Process
6. Select the first radio button No: This Host will NOT share access to the same
logical drives with other host as shown in Figure 29.
Figure 29: Non-Clustered Host
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 38
7. The following Figure 30 lists all the parameters “hostname, host type, host wwpn,
etc.” configured in the preceding steps. Re-check to ensure that all the values are unique and select Finish to confirm the parameters.
Figure 30: Configure Host Access ���� Confirm Host Definition
8. Select No to exit the host configuration process as shown in Figure 31.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 39
Figure 31: Configure Host Access ���� Complete
9. Now select the Create Host Group (optional) option from the Configuration menu to define a new Host Group and add the host configured manually in the preceding steps. Note: Creating a Host group for Non-Clustered nodes is not necessary
10. Specify a Host Group name. The host should be listed under the Select hosts to
add menu. Select the host, then select the Add option to add the host to the group as shown in Figure 32.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 40
Figure 32: Create Host Group
11. Select OK to complete the Create Host Group process.
Figure 33: Create Host Group ���� Add Host
12. The following Figure 34 confirms that you successfully added the host to a Host
group.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 41
Figure 34: Create Host Group ���� Complete
Create host-to-logical drive mappings
1. From the Configuration main menu, select the Create Host-to-Logical Drive Mappings option as shown in Figure 35.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 42
Figure 35: Create Host-to-Logical Drive Mappings Op tion
2. Select the host and click Next as shown in Figure 36.
Figure 36: Create Host-to-Logical Drive Mapping –Se lect Host
3. Select the boot LUN to be mapped to the host and assign LUN ID = 0 as shown in
figure 34, then select Finish as shown in Figure 36.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 43
Figure 37: Create Host-to-Logical Drive Mapping ���� Select Logical Drives
4. Select OK to complete the logical drive mapping process as shown in Figure 38.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 44
Figure 38: Create Host-to-Logical Drive Mapping ���� Completed
5. Select No to exit the Create Host-to-Logical Drive Mapping process as shown in
Figure 39.
Figure 39: Create the Create Host-to-Logical Drive Mapping ���� Complete
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 45
6. The following Figure 40 lists the Host-to-Logical Drive Mapping.
Figure 40: Create Host-to-Logical Drive Mapping
At this point, the necessary configuration to prepare the blade to access the boot LUN via the single path is complete. If the zoning and host definitions are performed correctly, then the blade should see the boot LUN from the LSI SAS HBA BIOS utility.
Installing the operating system
The following sections list the procedure to successfully install the Windows 2003 and RHEL4 U4 x64 operating systems. Before installing the operating system, you must complete all the necessary configuration steps shown in the preceding steps. Confirm that the host can access the boot LUN via the single path and that the boot sequence and boot order is selected based on the boot LUN’s preferred owner, etc.
Installing the Windows 2003 operating system
Note : To install RHEL4 U4, refer to the Installing RHEL 4 Update 4 operating system section on page 59. 1. Configure the boot sequence from AMM.
Note: Unlike Fibre Channel, there is no need to select and configure the boot device (the primary and secondary) from LSI SAS HBA BIOS (i.e., controllers). After the
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 46
boot sequence is configured from the AMM or from the blade BIOS menu, the SAS HBA automatically finds the first boot disk selected in the boot order and boots from it. After the boot sequence is configured from the AMM ���� Blade Tasks ���� Configuration ���� Boot Sequence , select Save as shown in the following Figure 41.
Figure 41: AMM ���� Configure Blade Boot Sequence
Note : Download the latest SAS HBA device driver and create a diskette. To download the latest SAS device driver, refer to the Supported Hardware and software configuration section. 2. Insert the Windows 2003 Standard or Enterprise Edition CD in the media tray and
select the media tray for the appropriate blade booting from the Boot Disk System. The blade in slot 13 for this example is selected for media tray access.
3. Power on the blade server and watch for the LSI MPT SAS BIOS banner. 4. The following Figure 42 shows that a 15 GB LUN with LUN ID = 0 on Boot Disk
System (1726-2xx) is seen by the host. This confirms that the host (blade13) has access to the boot LUN via the single path.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 47
Note: For the OS installation to complete successfully on the boot LUN, it is critical that only a single, unique path exists from the blade to the boot LUN.
Figure 42 LSI MPT SAS Banner
5. Boot the blade server using the Windows 2003 Installation CD and prepare to
press F6 immediately as shown in Error! Reference source not found. .
Figure 43: Windows Server Initial Install Screen
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 48
6. Press the Enter key option shown in Error! Reference source not found. to
install the latest SAS STORPort device driver from the diskette.
Figure 44: Windows Server Select SCSI Adapter Screen
7. The following Figure 45 also confirms that the device driver is correctly
recognized and that the Windows Setup is loading the driver.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 49
Figure 45: Windows Setup is Loading SAS Device Driv er
8. Press Enter to continue with Windows 2003 setup as shown in Figure 46.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 50
Figure 46: Windows Setup Loads MPT SAS Driver
9. The following Figure 47 shows that the Windows 2003 setup has completed
successfully. Restart the system and perform the following tasks:
Figure 47: Windows 2003 Setup Completed Successfull y
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 51
a. Install the recommended service packs and reboot. b. Install the Microsoft MPIO Device Specific Module (DSM) driver for
failover and multipathing, Agent/Utilities (optional). Refer to the Boot Disk System Installation and Setup documentation for more information.
c. Turn off the LS21 blade server.
Add the secondary path from the host to the boot LU N
The following topology diagram in Figure 49 shows two paths from the host to the Boot Disk System.
Figure 48: Dual Paths from Blade to the Boot Disk S ystem
Enable failover for the Boot LUN
SAS connectivity module configuration
1. Verify from the SAS connectivity module Web interface � Monitor SAS Module option that the second path from the host to the Boot Disk System is Enabled and Connected as shown in the following Figure 49:
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 52
Figure 49: Monitor SAS Module ���� Blade 13
2. If the path is not connected, then connect the physical cable from Boot Disk
System controller B to the SAS connectivity module installed in bay 4 for the BladeCenter chassis.
3. Verify and confirm that the active zone configuration includes the host and Boot Disk System storage port. The “Predefined Config 01” zone configuration option is for this test as shown in Figure 50 below.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 53
Figure 50: Active Zone Configuration
Boot Disk System storage configuration
1. Access the Boot Disk System Storage subsystem using the storage manager graphical user interface (GUI) and select the Modify ���� Edit Host Topology option as shown in Figure 51.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 54
Figure 51: Edit Host Topology
2. Select the Host � Add HBA option as shown in Figure 52.
Figure 52: Edit Host Topology ���� Add HBA Option
3. Select the WWPN from the drop-down list, host type and specify the HBA host
port alias as shown in Figure 53.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 55
Figure 53: Edit Host Topology ���� WWPN, Host Type
4. Select Add to apply the selected values as shown in Figure 54.
Figure 54: Edit Host Topology ���� Add Option
5. The following Figure 55 lists the WWPNs for the dual port SAS HBA. Select Close to
exit the Boot Disk System Edit Host Topology menu.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 56
Figure 55: Edit Host Topology Complete
Host configuration
1. Verify from the Windows 2003 Device Manager that the second path to the Boot LUN is Enabled as shown in Figure 56.
Note : The second disk labeled “disk1” should disappear after the The Microsoft MPIO Device Specific Module (DSM) is installed on the host, because it ensures a unique path to the boot LUN.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 57
Figure 56: Windows 2003 Computer Management ���� Disk Management
2. Download the latest Microsoft MPIO Device Specific Module (DSM) for Windows contained in the Boot Disk System Storage Manager Client package available at the Boot Disk System download site and double-click to install.
3. Double-clicking the executable file brings up the main installation menu. Select OK
to continue. 4. Select the Host option to install the Microsoft MPIO DSM driver, as shown in Figure
57.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 58
Figure 57: the Microsoft MPIO DSM Install
5. Follow the prompts and accept the agreement (examples not shown). 6. Review the Install Folder and select Install to continue with the Microsoft MPIO
DSM driver installation as shown in Figure 58.
Figure 58: the Microsoft MPIO DSM Install Folder
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 59
7. If the installation completes successfully, the program will ask for system restart.
Select Yes, restart my system � Done to restart the system as shown in Figure 59.
Figure 59: Microsoft MPIO (DSM) for Windows Install Completed Successfully
8. After the reboot, verify from the Windows 2003 Disk Management that the
duplicate boot LUN labeled as “disk1” in figure 53 has disappeared as shown in Figure 60. This confirms that the Microsoft MPIO DSM driver installation succeeded.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 60
Figure 60 Windows 2003 Disk Manager
9. The Windows Device Manager now also lists the IBM 1726-2xx FAStT Multipath
disk device for the driver as the virtual disk, as shown in Figure 61.
Figure 61: Windows 2004 Device Manager ���� Multipath Disk Device
This completes the Microsoft MPIO DSM driver installation process.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 61
Failover verification
1. From the Storage Manager ���� Subsystem Profile ���� Logical Drives menu, verify the current LUN owner for the boot LUN, as shown in the following Figure 62.
Figure 62: Verify Current LUN Owner
2. The failover test for the boot LUN can be performed in couple of ways. You can pull
the SAS cable between the controller and switch or disable the SAS HBA port connected to the controller that currently owns (preferred owner) the boot LUN.
3. To Disable the SAS HBA port connected to the switch module in bay3, from the
command line interface (CLI), disable the port# 13 as shown in the text box below, because the Boot Disk System controller A is the current owner of the boot for blade 13.
=== IBM BladeCenter SAS Connectivity Module ===
Login: USERID
Password: ******** USERID logged on
MAIN> sasport disable b13 B13 Blade Slot Connection 13 OK - Port disable request has been queued.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 62
4. After the SAS HBA port is Disabled, the operating system pauses until the boot LUN is moved to the backup path, i.e., Boot Disk System controller B in this case, as shown in the following Figure 63.
Figure 63: Boot Disk System Profile ���� Current LUN Owner
5. After successfully verifying the failover function by Microsoft MPIO DSM driver,
enable the primary path to Boot Disk System controller A as shown in the following example.
=== IBM BladeCenter SAS Connectivity Module === Login: USERID Password: ******** USERID logged on MAIN> sasport enable b13 B13 Blade Slot Connection 13 OK - Port enable request has been queued. This completes the failover verification process of the blade with Windows 2003 operating system.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 63
Installing RHEL 4 Update 4 operating system
The following sections cover steps to deploy RHEL4 U4 x64 on the HS21 blade server attached to THE BladeCenter Boot Disk system.
1. Download the RHEL4 U4 x64 device driver for the SAS HBA and create a device driver diskette.
2. Verify that CDROM is one of the options configured in the blade boot sequence.
3. Insert the RHEL4 U4 x64 CD1 in the media tray and ensure that the media tray is
selected.
4. Power on or restart the blade.
5. After the blade boots from CD1, enter linux dd as shown in the following Figure 64.
Figure 64: Rhel4 U4 Install Method
6. The Red Hat installation asks for the driver source, select the appropriate media,
then select OK as shown in the following Figure 66.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 64
Figure 65: Driver Disk Source
7. Select OK after inserting the driver diskette in the floppy drive as shown in the
following Figure 66.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 65
Figure 66: Insert Driver Disk
8. The following Figure 67 shows that the Red Hat installation has correctly recognized and loaded the SAS device driver.
Figure 67: Reading Driver Disk
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 66
9. At the Automatic Storage Partitioning menu, select the Remove all partitions on
this system radio button and uncheck the second USB disk check box as shown in the following Figure 68.
Figure 68: Automatic Partitioning
10. The following Figure 69 shows that the boot LUN is the first device “/dev/sda” seen by the operating system. This is critical for successfully installing the OS on the boot LUN, because Linux always uses the first device /dev/sda as the boot device.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 67
Figure 69: Disk Setup
11. Select the necessary software packages. For this test, the install Everything
option is selected as shown in the following Figure 70.
Note: RHEL4 U4 x64 requires CD1-5 if the install Everything option is selected.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 68
Figure 70: Software Selection
12. From this point, follow the prompts and change the CD media, specify a
password for the root user, etc., and complete the OS installation successfully on the BladeCenter Boot Disk system via the single path.
13. Add the second path to the boot LUN. For additional details on adding the
second path to achieve failover, refer to the Add the secondary path from the host to the boot LUN on page 51.
14. The following section covers the procedure to install the RDAC driver to achieve
failover capability for the boot LUN.
Install the RDAC driver
1. Download the latest RDAC driver from the BladeCenter Boot Disk System download site, and refer to the Supported Hardware and software configuration on page 3 for the link to downloading the latest RDAC driver
2. Copy the driver tar file to the host and change to the directory as shown in the
following example:
[root@localhost sasboot]# ls rdac-LINUX-09.01.B5.77-source.tar.gz
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 69
3. Unpack the source package, as shown in the following example:
root@localhost sasboot]# tar -xvzf rdac-LINUX-09.01.B5.77-source.tar.gz
4. The source files will uncompress into the directory name “linuxrdac-09.01.B5.77” as shown in the following example:
[root@localhost sasboot]# ls -l total 320 drwxr-xr-x 5 root root 4096 Sep 25 11:28 linuxrdac-09.01.B5.77 -rw-r--r-- 1 root root 308045 Sep 25 11:11 rdac-LINUX-09.01.B5.77-source.tar.gz [root@localhost sasboot]#
5. Change to the “linuxrdac-09.01.B5.77” directory and list the contents as shown in the following example:
[root@localhost linuxrdac-09.01.B5.77]# ls genfileattributes mppiscsi_umountall mppLnx26_vhbaproc.c genuniqueid.c mpp_linux_headers mppLnx26_vhbatask.c hbaCheck mpp_linux_sys_headers mpp_rcscript.REDHAT License.txt mppLnx26_spinlock_size.c mpp_rcscript.SUSE lsvdev mppLnx26_sysdep.c mppSupport Makefile mppLnx26_upper.c Readme.txt mppCmn_s2tos3.c mppLnx26_vhba.c setupDriver.REDHAT mppCmn_SysInterface.c mppLnx26_vhbaio.c setupDriver.SUSE mpp.conf mppLnx26_vhbalib.c utility MPP_hba.c mppLnx26_vhbamisc.c [root@localhost linuxrdac-09.01.B5.77]#
6. Before you compile the driver, type the make clean command as shown in the following example:
[root@localhost linuxrdac-09.01.B5.77]# make clean make V=1 -C/lib/modules/2.6.9-42.ELsmp/build M=/sasboot/linuxrdac-09.01.B5.77 MODVERDIR=/lib/modules/2.6.9-42.ELsmp/build/.tmp_versions SUBDIRS=/sasboot/linuxrdac-09.01.B5.77 clean make[1]: Entering directory `/usr/src/kernels/2.6.9-42.EL-smp-x86_64' make -f scripts/Makefile.clean obj=/sasboot/linuxrdac-09.01.B5.77 rm -rf /lib/modules/2.6.9-42.ELsmp/build/.tmp_versions make[1]: Leaving directory `/usr/src/kernels/2.6.9-42.EL-smp-x86_64' rm -f ./mpp_linux_sys_headers/mppLnx26_spinlock.h mppLnx_Spinlock_Size [root@localhost linuxrdac-09.01.B5.77]#
7. To compile all driver modules and utilities, type the make command as shown
in the following example:
[root@localhost linuxrdac-09.01.B5.77]# make
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 70
make V=1 -C/lib/modules/2.6.9-42.ELsmp/build M=/sasboot/linuxrdac-09.01.B5.77 MODVERDIR=/lib/modules/2.6.9-42.ELsmp/build/.tmp_versions SUBDIRS=/sasboot/linuxrdac-09.01.B5.77 modules make[1]: Entering directory `/usr/src/kernels/2.6.9-42.EL-smp-x86_64' mkdir -p /lib/modules/2.6.9-42.ELsmp/build/.tmp_versions make -f scripts/Makefile.build obj=/sasboot/linuxrdac-09.01.B5.77
…… ………….. …………….. ………………… ……………………….. ……………………………
gcc -I/sasboot/linuxrdac-09.01.B5.77 -I/sasboot/linuxrdac-09.01.B5.77/mpp_linux_headers/ -I/sasboot/linuxrdac-09.01.B5.77/mpp_linux_sys_headers/ -c ./utility/mppUtil.c -o mppUtil.o /bin/bash ./genfileattributes bld gcc -I/sasboot/linuxrdac-09.01.B5.77 -I/sasboot/linuxrdac-09.01.B5.77/mpp_linux_headers/ -I/sasboot/linuxrdac-09.01.B5.77/mpp_linux_sys_headers/ -c ./utility/mppUtil26_sysdep.c -o mppUtilSysdep.o gcc mppUtil.o mppUtilSysdep.o -o mppUtil gcc -o genuniqueid genuniqueid.c [root@localhost linuxrdac-09.01.B5.77]#
8. Type the make install to perform the following: • Copy the driver modules to the kernel module tree. • Build the new RAMdisk image (mpp-`uname -r`.img) which includes the
RDAC driver modules and all driver modules that are needed at boot time.
[root@localhost linuxrdac-09.01.B5.77]# make install make V=1 -C/lib/modules/2.6.9-42.ELsmp/build M=/sasboot/linuxrdac-09.01.B5.77 MODVERDIR=/lib/modules/2.6.9-42.ELsmp/build/.tmp_versions SUBDIRS=/sasboot/linuxrdac-09.01.B5.77 modules make[1]: Entering directory `/usr/src/kernels/2.6.9-42.EL-smp-x86_64' mkdir -p /lib/modules/2.6.9-42.ELsmp/build/.tmp_versions make -f scripts/Makefile.build obj=/sasboot/linuxrdac-09.01.B5.77 Building modules, stage 2
…… ………….. …………….. ………………… ……………………….. …………………………… gcc mppUtil.o mppUtilSysdep.o -o mppUtil Checking Host Adapter Configuration... Detected 1 LSI Host Adapter Port(s) on the system Please wait while we modify the system configuration files. Your kernel version is 2.6.9-42.ELsmp Preparing to install MPP driver against this kernel version... Generating module dependencies...
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 71
Creating new MPP initrd image... You must now edit your boot loader configuration file, /boot/grub/menu.lst, to add a new boot menu, which uses mpp-2.6.9-42.ELsmp.img as the initrd image. Now Reboot the system for MPP to take effect. The new boot menu entry should look something like this (note that it may vary with different system configuration): ... title Red Hat Linux (2.6.9-42.ELsmp) with MPP support root (hd0,5) kernel /vmlinuz-2.6.9-42.ELsmp ro root=LABEL=RH9 initrd /mpp-2.6.9-42.ELsmp.img ... MPP driver package has been successfully installed on your system. [root@localhost linuxrdac-09.01.B5.77]#
9. Verify that the initrd image named as mpp-2.6.9-42.ELsmp.img is listed in the /boot directory as shown in the following example:
[root@localhost sasboot]# ls /boot config-2.6.9-42.EL lost+found System.map-2.6.9-42.EL config-2.6.9-42.ELsmp memtest86+-1.26 System.map-2.6.9-42.ELsmp grub message vmlinuz-2.6.9-42.EL initrd-2.6.9-42.EL.img message.ja vmlinuz-2.6.9-42.ELsmp initrd-2.6.9-42.ELsmp.img mpp-2.6.9-42.ELsmp.img [root@localhost sasboot]#
10. List the contents of the /boot/grub/menu.lst prior to making changes as shown in the following example: [root@localhost linuxrdac-09.01.B5.77]# more /boot/grub/menu.lst # grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You have a /boot partition. This means that # all kernel and initrd paths are relative to /boot/, eg. # root (hd0,0) # kernel /vmlinuz-version ro root=/dev/VolGroup00/LogVol00 # initrd /initrd-version.img #boot=/dev/sda default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Red Hat Enterprise Linux AS (2.6.9-42.ELsmp) root (hd0,0) kernel /vmlinuz-2.6.9-42.ELsmp ro root=/dev/VolGroup00/LogVol00 rhgb qui et initrd /initrd-2.6.9-42.ELsmp.img title Red Hat Enterprise Linux AS-up (2.6.9-42.EL) root (hd0,0) kernel /vmlinuz-2.6.9-42.EL ro root=/dev/VolGroup00/LogVol00 rhgb quiet initrd /initrd-2.6.9-42.EL.img
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 72
[root@localhost linuxrdac-09.01.B5.77]#
11. Now modify the /boot/grub/menu.lst file to include the “mpp-2.6.9-42.ELsmp.img” inird image as shown in the following example:
[root@localhost sasboot]# more /boot/grub/menu.lst # grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You have a /boot partition. This means that # all kernel and initrd paths are relative to /boot/, eg. # root (hd0,0) # kernel /vmlinuz-version ro root=/dev/VolGroup00/LogVol00 # initrd /initrd-version.img #boot=/dev/sda default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title RHEL4 Update4 i86_64 MPP Kernel (2.6.9-42.ELsmp) root (hd0,0) kernel /vmlinuz-2.6.9-42.ELsmp ro root=/dev/VolGroup00/LogVol00 rhgb qui et initrd /mpp-2.6.9-42.ELsmp.img title Red Hat Enterprise Linux AS (2.6.9-42.ELsmp) root (hd0,0) kernel /vmlinuz-2.6.9-42.ELsmp ro root=/dev/VolGroup00/LogVol00 rhgb qui et initrd /initrd-2.6.9-42.ELsmp.img title Red Hat Enterprise Linux AS-up (2.6.9-42.EL) root (hd0,0) kernel /vmlinuz-2.6.9-42.EL ro root=/dev/VolGroup00/LogVol00 rhgb quiet initrd /initrd-2.6.9-42.EL.img [root@localhost sasboot]#
12. Reboot the system for the new MPP based initrd image to take effect as
shown in the following example: [root@localhost linuxrdac-09.01.B5.77]# reboot Broadcast message from root (pts/2) (Tue Sep 25 11:40:46 2007): The system is going down for reboot NOW! [root@localhost linuxrdac-09.01.B5.77]#
13. During the reboot, confirm that the LSI HBA sees two paths to the boot disk as shown in the following Figure 71.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 73
Figure 71: LSI SAS BIOS Banner
This completes the successful RDAC driver installation on the RHEL4 U4 x64 operating system.
Failover verification
1. From the Storage Manager ���� Subsystem Profile ���� Logical Drives menu, verify the current LUN owner for the boot LUN as shown in the following Figure 72.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 74
Figure 72: Verify Current LUN Owner
2. The failover test for the boot LUN can be performed in couple of ways. You can pull
the SAS cable between the controller and switch or disable the SAS HBA port connected to the controller that currently owns (preferred owner) the boot LUN.
3. To Disable the SAS HBA port connected to the switch module in bay4, from the CLI,
disable the port# 14 as shown in the text box below -- because the Boot Disk System controller B is the current owner of the boot for blade 14.
=== IBM BladeCenter SAS Connectivity Module ===
Login: USERID Password: ********
USERID logged on
MAIN> sasport disable b14 B13 Blade Slot Connection 14 OK - Port disable request has been queued.
4. After the SAS HBA port is disabled, the operating system pauses until the boot LUN
is successfully moved to the alternate path, i.e., Boot Disk System’s controller A in this case as shown in the following Figure 73.
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 75
Figure 73: Verify LUN Failover
6. After successfully verifying the failover function by RDAC, enable the primary path to
Boot Disk System controller A as shown in the following example.
This completes the failover verification process on the blade with Windows 2003 operating system.
=== IBM BladeCenter SAS Connectivity Module === Login: USERID Password: ******** USERID logged on MAIN> sasport enable b14 B14 Blade Slot Connection 14 OK - Port enable request has been queued. MAIN>
Implementing the IBM BladeCenter Boot Disk System Solution
©2007 International Business Machines Corporation 76
Appendix: Useful links
1. IBM BladeCenter SAS Expansion Card and SAS Connectivity module
2. IBM BladeCenter Boot Disk Solution
3. IBM BladeCenter Boot Disk System Installation, User's, and Maintenance Guide
4. IBM System Storage DS3000 Storage Manager Version 2 Installation and Support Guide for Microsoft Windows Server 2003, Linux, Novell NetWare, and VMware ESX Server
5. IBM BladeCenter SAS Connectivity Module Installation and User's Guide
6. IBM BladeCenter SAS Expansion Card (CFFv) Installation and User's Guide