1

How to connect

Linux to a storage unit

PSSC - IBM Customer Center Montpellier

Guillaume CRESTA

Continuous Availability

High Availability

Continuous Operations

2

France

Table of Contents

1 Introduction ........................................................................................................................ 3

Presentation of the infrastructure .......................................................3 2 How to configure................................................................................................................ 4

2.1 Gathering the World Wide Name..............................................4

2.2 Configuring the unit storage......................................................5

2.3 Configuration at the SAN level: zoning....................................9

2.4 The operating system: Linux...................................................12

2.5 Script........................................................................................20

2.6 Dscli .........................................................................................21 3 Special Thanks ................................................................................................................. 23

3

1 Introduction

In order to answer to some of customers’ main concerns and to provide them with answers

based on real testing, we are implementing a two sites long distance disaster recovery solution

between Montpellier (France) and Poughkeepsie (USA) that is 7000 km between the

production site and the backup site. Within the framework of this project, an integration of

System x has been effected. The purpose of this document is to describe the configuration of

the System x with RedHat Enterprise Linux 4.0 and the storage unit DS8000 o DS6000 based

on this real experience.

In order to connect these two technologies, we ran different steps explained all over the

document. The last part of this document is dedicated to the persons who help me.

Presentation of the infrastructure

The main components of the architecture are:

� IBM System p servers

� IBM System x servers

� IBM DS8000 and DS6000 disk subsystems

� MCDATA SAN switches

� MCDATA iFCP router 2027-R16

The hardware environment we are using for this test is dispatched in 2 sites (one site in

Montpellier-France and one site in Poughkeepsie-US).

4

2 How to configure

Different steps have to be done in order to attach a Linux (RedHat Enterprise Linux release

4.0) to a DS8000 storage unit:

� gather the World Wide Name, unique identifier of each fiber card and that permits

the identification of the System x on storage unit

� configure the storage

� make configuration at SAN level

� install sdd service

� to configure Linux in order to show disks

� make starting script and at the end

� install the dscli tool in order to communicate with storage unit.

2.1 Gathering the World Wide Name

First we have to see the compatibility matrix in order to verify if the Linux version installed is

in agreement to the storage unit (http://www-

03.ibm.com/servers/storage/disk/ds8000/pdf/ds8000-matrix.pdf). As far as that goes, we have

to know the Linux version and his kernel:

[root@halinux Desktop]# more /etc/redhat-release Red Hat Enterprise Linux AS release 4 (Nahant Update 3) [root@halinux Desktop]# uname -a Linux halinux 2.6.9-34.ELsmp #1 SMP Fri Feb 24 16:54:53 EST 2006 i686 i686 i386 GNU/Linux

Then, in order to make zoning, we need the WWPN of fiber channel card of the System x:

[root@halinux Desktop]# cd /proc/scsi/qla2xxx/ [root@halinux qla2xxx]# ls 1 [root@halinux qla2xxx]# cat 1 QLogic PCI to Fibre Channel Host Adapter for QLA2310: Firmware version 3.03.18 IPX, Driver version 8.01.02-d4 ISP: ISP2300, Serial# D76762 Request Queue = 0x1bc0000, Response Queue = 0x37a00000 Request Queue count = 2048, Response Queue count = 512 Total number of active commands = 0 Total number of interrupts = 2781 Device queue depth = 0x20 Number of free request entries = 784 Number of mailbox timeouts = 0 Number of ISP aborts = 0 Number of loop resyncs = 0 Number of retries for empty slots = 0 Number of reqs in pending_q= 0, retry_q= 0, done_q= 0, scsi_retry_q= 0 Host adapter:loop state = <READY>, flags = 0x1a03

5

Dpc flags = 0x4000000 MBX flags = 0x0 Link down Timeout = 000 Port down retry = 030 Login retry count = 030 Commands retried with dropped frame(s) = 0 Product ID = 4953 5020 2020 0001 SCSI Device Information: scsi-qla0-adapter-node=200000e08b05babf; scsi-qla0-adapter-port=210000e08b05babf; scsi-qla0-target-0=50050763031080b7; scsi-qla0-target-1=50050763031bc0b7; FC Port Information: scsi-qla0-port-0=5005076303ffc0b7:50050763031080b7:740d13:81; scsi-qla0-port-1=5005076303ffc0b7:50050763031bc0b7:740f13:82; SCSI LUN Information: (Id:Lun) * - indicates lun is not registered with the OS. ( 0: 0): Total reqs 1674, Pending reqs 0, flags 0x0, 0:0:81 00 ( 1: 0): Total reqs 1587, Pending reqs 0, flags 0x0, 0:0:82 00 [root@halinux qla2xxx]#

The LUNs mapped to the server can be verified by checking the contains of the file

/proc/scsi/scsi:

[root@halinux ~]# cat /proc/scsi/scsi Host:SCSI0 Channel:00 Id:00 Lun:00 Vendor: IBM-ESXS Model:CBR036C31210ESFN Rev: DFQD Type: Direct-Access ANSI SCSI revision:04 Host:SCSI0 Channel:00 Id:08 Lun:00 Vendor: IBM Model:FT1V1S2 Rev: 0 Type: Processor ANSI SCSI revision:02 Host:SCSI1 Channel:00 Id:00 Lun:00 Vendor: IBM-ESXS Model:2107900 Rev: 201 Type: Direct-Access ANSI SCSI revision:05 Host:SCSI1 Channel:00 Id:01 Lun:00 Vendor: IBM-ESXS Model:2107900 Rev: 201 Type: Direct-Access ANSI SCSI revision:05

2.2 Configuring the unit storage

The configuration of the unit storage can be done by GUI (web interface) or by CLI

(commands line). I myself prefer working with commands line. The working is the same with

a DS6000 or with a DS8000. The below extracts correspond to the creation on DS8000.

6

To create host on storage unit permitting relation between Linux and storage unit, the World

Wide Port Name mentioned on Linux (210000E08B05BABF), the type of operating system

used (LinuxRHEL) and the DS ports used (all in order to mention all available):

dscli mkhostconnect -wwname 210000E08B05BABF -hosttype LinuxRHEL -ioport all 'HA1Linux' -cfg C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile Date/Time: 4 ao?t 2006 14:25:30 CEST IBM DSCLI Version: 5.1.600.260 DS: IBM.2107-7506551 CMUC00012I mkhostconnect: Host connection 0013 successfully created. Date/Time: 08 04, 2006 14:25:39 Execution time : 12 Seconds.

The configuration made can be verified:

dscli lshostconnect -l -cfg C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile Date/Time: 25 juillet 2006 18:42:57 CEST IBM DSCLI Version: 5.1.600.260 DS: IBM.2107-7506551 Name ID WWPN HostType LBS addrDiscovery Profile portgrp volgrpID atchtopo ESSIOport ========================================================================================================================================================================================================================================================================================================================================================================== BSBI_P53 0000 10000000C93EC6FE pSeries 512 reportLUN IBM pSeries - AIX 1 - FC-AL all […] HA1Linux 0013 210000E08B05BABF LinuxRHEL 512 LUNPolling Intel - Linux RHEL 0 - - all […] SVC_SpeederMan_EW 0042 50050768014003EA SVC 512 reportLUN San Volume Controller 0 V2 - all Date/Time: 07 25, 2006 18:43:06 Execution time : 11 Seconds.

Moreover, a space on storage unit called Volumes is created, specifying the capacity (5 Gb),

the volume name (Vol1Halinux) and a number available (47000) and the extend pool on

which it’s going to be created (P3). Before, we need to verify that the extend pool has

allocated space.

dscli lsextpool -l -cfg C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile Date/Time: 4 ao?t 2006 14:29:36 CEST IBM DSCLI Version: 5.1.600.260 DS: IBM.2107-7506551 Name ID stgtype rankgrp status availstor (2^30B) %allocated available reserved numvols numranks ======================================================================================================== extpool_ha0 P0 fb 0 below 650 16 650 0 8 1 extpool_ha1 P1 fb 1 below 679 12 679 0 4 1 extpool_ha2 P2 fb 0 below 859 5 859 0 2 1

7

extpool_ha3 P3 fb 1 below 729 6 729 0 2 1 […] Date/Time: 08 04, 2006 14:29:51 Execution time : 17 Seconds. dscli mkfbvol -extpool P3 -cap 5 -name 'Vol1Halinux' 4700 -cfg C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile Date/Time: 4 ao?t 2006 14:30:45 CEST IBM DSCLI Version: 5.1.600.260 DS: IBM.2107-7506551 CMUC00025I mkfbvol: FB volume 4700 successfully created. Date/Time: 08 04, 2006 14:30:55 Execution time : 12 Seconds.

When the volumes are established they are grouped in a volume group. In this case, there’s

only one volume. But, in order to connect the host to the volume, a volume group need to be

created. When the volume group is created by command line interface, the type SCSIMASK o

SCSIMap256 can be configured. By default, if you put SCSIMask, the relation between

volume group and host configured as a RedHat EL is impossible in this case. In order to make

a volume group, the type (scsimpa256 in case of a Linux RedHatEL), the volumes number

gathered in the volume group (4700) and volume group name (VolGrpe1HALinux) have to be

specified:

dscli mkvolgrp -type scsimap256 -volume 4700 'Volgrpe1HALinux' -cfg C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile Date/Time: 4 ao?t 2006 14:31:45 CEST IBM DSCLI Version: 5.1.600.260 DS: IBM.2107-7506551 CMUC00030I mkvolgrp: Volume group V4 successfully created. Date/Time: 08 04, 2006 14:31:53 Execution time : 10 Seconds.

The volume establishment by command line can be verified:

dscli lsvolgrp -l -cfg C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile Date/Time: 4 ao?t 2006 14:22:27 CEST IBM DSCLI Version: 5.1.600.260 DS: IBM.2107-7506551 Name ID Type ======================================== FC_OracleCode V0 SCSI Mask HAVG1 V1 SCSI Mask SVCDEMO_EW V2 SCSI Mask MED4PC12 V3 SCSI Map 256 VolGrp1_HALINUX V4 SCSI Map 256 sdvolume V5 SCSI Map 256 VGPC2 V6 SCSI Map 256 blue_aix_VG V7 SCSI Mask Volumes_for_AIX V8 SCSI Mask Date/Time: 08 04, 2006 14:22:36 Execution time : 12 Seconds.

8

The volume establishment can be verified and so that it is associated to the volume group V4:

dscli lsfbvol -l -cfg C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile Date/Time: 17 mai 2006 18:40:59 CEST IBM DSCLI Version: 5.1.600.260 DS: IBM.2107-7506551 Name ID accstate datastate configstate deviceMTM datatype extpool sam captype cap (2^30B) cap (10^9B) cap (blocks) volgrp =========================================================================================================================================== 03000 0300 Online Normal Normal 2107-900 FB 512 P7 Standard DS 2.0 - 4194304 V18 [...] Vol1Halinux 4700 Online Normal Normal 2107-900 FB 512 P3 Standard DS 5.0 - 10485760 V4 [...] volAIXLaurent FB00 Online Normal Normal 2107-900 FB 512 P7 Standard DS 1.0 - 2097152 - Date/Time: 05 17, 2006 18:41:08 Execution time : 12 Seconds.

We need to associate the volume group to the host previously established, specifying the

volume group number (V4) and the host one (0013):

dscli chhostconnect -volgrp V4 0013 -cfg C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile Date/Time: 4 ao?t 2006 14:32:43 CEST IBM DSCLI Version: 5.1.600.260 DS: IBM.2107-7506551 CMUC00013I chhostconnect: Host connection 0013 successfully modified. Date/Time: 08 04, 2006 14:32:51 Execution time : 10 Seconds.

It can be verified that this operation has been realized by watching the host configuration

again:

dscli lshostconnect -l -cfg C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile Date/Time: 25 juillet 2006 18:42:57 CEST IBM DSCLI Version: 5.1.600.260 DS: IBM.2107-7506551 Name ID WWPN HostType LBS addrDiscovery Profile portgrp volgrpID atchtopo ESSIOport ========================================================================================================================================================================================================================================================================================================================================================================== BSBI_P53 0000 10000000C93EC6FE pSeries 512 reportLUN IBM pSeries - AIX 1 - FC-AL all […] HA1Linux 0013 210000E08B05BABF LinuxRHEL 512 LUNPolling Intel - Linux RHEL 0 V4 - all […] SVC_SpeederMan_EW 0042 50050768014003EA SVC 512 reportLUN San Volume Controller 0 V2 - all Date/Time: 07 25, 2006 18:43:06 Execution time : 11 Seconds.

9

2.3 Configuration at the SAN level: zoning

In order to administrate the switches, a web application is available. Thanks to EFCM Basic,

we can configure switches by accessing to the menu Configure > Zoning and the page

figuring next page can be seen.

In the first column, all nodes are listed. The zone name is notified in the second column.

The World Wide Port Names have to be added by using the button >.

10

In the zone S1_LINUX, the System x World Wide Port Names and the unit storage World

Wide Port Names can be verified to be there. When this inspection is done, this zone has to be

joined to the zone set. The button > is used to realized it.

11

Now, the zone appears in the column Zone set. Thus, the modification is saved by the button

Update and applied by Activate.

The created zones are editable using the button < and the steps above remade.

The port used is configured. In this case, System x is connected on switch port 0. The fields

associated to this port are informed.

12

Thereafter, we can access to the list of the switch ports (Product > Port List). Clicking on

port 0, information linked to this port appear, for example le World Wide Port Name and the

World Wide Node Name.

The zoning and the storage unit configuration can be done in parallel. The order done doesn’t

matter.

2.4 The operating system: Linux

� The integration of sdd service

After the DS6/8000 configuration, the operating operation has to be restarted. It’s necessary to

verify that the disks are in relation with the operating system using the fdisk -l command:

login as: root Sent username "root" root@172.31.11.36's password: Last login: Wed Aug 16 11:04:10 2006 from 10.8.1.1 [root@halinux ~]# fdisk -l Disk /dev/sda: 36.4 GB, 36401479680 bytes 255 heads, 63 sectors/track, 4425 cylinders

13

Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 13 104391 83 Linux /dev/sda2 14 4425 35439390 8e Linux LVM Disk /dev/sdb: 5368 MB, 5368709120 bytes 166 heads, 62 sectors/track, 1018 cylinders Units = cylinders of 10292 * 512 = 5269504 bytes Disk /dev/sdb doesn't contain a valid partition table Disk /dev/sdc: 5368 MB, 5368709120 bytes 166 heads, 62 sectors/track, 1018 cylinders Units = cylinders of 10292 * 512 = 5269504 bytes Disk /dev/sdc doesn't contain a valid partition table

If this command doesn’t work, the disks mount through the Fiber Channel card can be forced

by the command line: echo "scsi-qlascan" > /proc/scsi/qla2xxx/1.

The System x linked to the DS8000 has got only one card, and when the zoning is made, two

links are put to the storage unit. The disk also appears in double: /etc/sdb and /etc/sdc above.

Whereas on System x linked to the DS6000 there are two cards each linked to the storage unit

by two fibers, four disks appear as notified below (/etc/sdb, /etc/sdc, /etc/sdd and /etc/sde):

login as: root Sent username "root" root@172.31.11.37's password: Last login: Wed Aug 16 11:05:28 2006 from 10.8.1.1 [root@halinux2 ~]# fdisk -l Disk /dev/sda: 36.4 GB, 36401479680 bytes 255 heads, 63 sectors/track, 4425 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 13 104391 83 Linux /dev/sda2 14 4425 35439390 8e Linux LVM Disk /dev/sdb: 5368 MB, 5368709120 bytes 166 heads, 62 sectors/track, 1018 cylinders Units = cylinders of 10292 * 512 = 5269504 bytes Disk /dev/sdb doesn't contain a valid partition table Disk /dev/sdc: 5368 MB, 5368709120 bytes 166 heads, 62 sectors/track, 1018 cylinders Units = cylinders of 10292 * 512 = 5269504 bytes Disk /dev/sdc doesn't contain a valid partition table Disk /dev/sdd: 5368 MB, 5368709120 bytes 166 heads, 62 sectors/track, 1018 cylinders Units = cylinders of 10292 * 512 = 5269504 bytes Disk /dev/sdd doesn't contain a valid partition table

14

Disk /dev/sde: 5368 MB, 5368709120 bytes 166 heads, 62 sectors/track, 1018 cylinders Units = cylinders of 10292 * 512 = 5269504 bytes Disk /dev/sde doesn't contain a valid partition table

Thus, the file IBM.sdd-1.6.1.0-3.i686.rhel4.rpm is installed.

[root@halinux ~]# rpm -ivh IBMsdd-1.6.0.1-11.1.i686.rhel4.rpm Preparing.... ####################################################[100%] 1:IBMsdd ####################################################[100%] Added following line to /etc/inittab: srv:345:respawn:/opt/IBMsdd/bin/sddsrv > /dev/null 2>&1

In order to verify if it is correctly installed, /etc/inittab can be verified:

[root@halinux ~]# cat /etc/inittab | grep sdd #IBMsdd path recovery daemon: srv:345:respawn:/opt/IBMsdd/bin/sddsrv > /dev/null 2>&1

When done, you can verify the content of the package:

[root@halinux ~]# rpm -ql IBMsdd /etc/cron.hourly/sddrv_log.sh /etc/logrotate.d/sddsrv_log.d /etc/sddsrv.conf /etc/vpath.conf /opt/IBMsdd /opt/IBMsdd/bin /opt/IBMsdd/bin/cfgvpath /opt/IBMsdd/bin/datapath /opt/IBMsdd/bin/lsvpcfg /opt/IBMsdd/bin/make_sddcrs /opt/IBMsdd/bin/pathtest /opt/IBMsdd/bin/rmvpath /opt/IBMsdd/bin/sdd_rcscript /opt/IBMsdd/bin/sddgetdata /opt/IBMsdd/bin/sddsrv /opt/IBMsdd/kernel_list /opt/IBMsdd/sdd-mod.ko-2.6.9-11.ELhugemen /opt/IBMsdd/sdd-mod.ko-2.6.9-11.ELsmp /opt/IBMsdd/sdd-mod.ko-2.6.9-22.0.1.ELhugemen /opt/IBMsdd/sdd-mod.ko-2.6.9-22.0.1.ELsmp /opt/IBMsdd/sdd-mod.ko-2.6.9-22.0.2.ELhugemen /opt/IBMsdd/sdd-mod.ko-2.6.9-22.0.2.ELsmp /opt/IBMsdd/sdd-mod.ko-2.6.9-22.ELhugemen /opt/IBMsdd/sdd-mod.ko-2.6.9-22.ELsmp /opt/IBMsdd/sdd-mod.ko-2.6.9-34.ELhugemen /opt/IBMsdd/sdd-mod.ko-2.6.9-34.ELsmp /usr/share/doc/IBMsdd-1.6.1.0/LICENSE /usr/share/doc/IBMsdd-1.6.1.0/README /usr/share/doc/IBMsdd-1.6.1.0/rd-linux.txt

This file is the last version of sdd for RedHat 4.0. When the file is installed, the status has to

be verified:

15

[root@halinux ~]# sdd status IBMsdd status: [FAILED] You have new mail in /var/spool/mail/root

If the status is FAILED, the sdd service has to be started:

[root@halinux ~]# sdd start Starting IBMsdd driver load: [ OK ] Issuing killall sddsrv to trigger respawn... Starting IBMsdd configuration: [ OK ] [root@halinux ~]# sdd status IBMsdd status: [ OK ] 000 vpatha ( 252, 0) 75065514700 = 6005076303ffc0b70000000000004700 = /dev/sdb /dev/sdc

In order to make multipathing, sdd make a vpath going to take into account the two disks and

manage the traffic between the two links.

� The visualization of the disks

When an operating system Linux and especially sdd is configured, the configuration of the

file /etc/lvm/lvm.conf has to be modified. Indeed, if you don’t modify it, you can’t create a

vpath in order to establish multipathing. In order to resolve this problem two lines in this file

have to be modified:

filter = ["a/.*/"]

types = ["fd", 16]

by the lines:

filter = ["a|/dev/sda|", "r|/dev/sd|", "a/.*/"]

types = ["vpath", 16]

These lines permit Linux server not to filter the other disk as sda (principal disk which is

installed Linux). When this step is done, we can also show that the path is created:

[root@halinux ~]#cfgvpath query /dev/sdb ( 8, 16) host=1 ch=0 id=0 lun=0 vid=IBM pid=2107900 serial=75065514700 lun_id=6005076303ffc0b70000000000004700 ctlr_flag=0 ctlr_nbr=0 df_ctlr=0 /dev/sdc ( 8, 32) host=1 ch=0 id=1 lun=0 vid=IBM pid=2107900 serial=75065514700 lun_id=6005076303ffc0b70000000000004700 ctlr_flag=0 ctlr_nbr=0 df_ctlr=0

When the scan disk is made, we can notice that a new disk is present and correspond to the

path group:

16

[root@halinux ~]# fdisk -l …………………………… Disk /dev/vpatha: 5368 MB, 5368709120 bytes 166 heads, 62 sectors/track, 1018 cylinders Units = cylinders of 10292 * 512 = 5269504 bytes Disk /dev/vpatha doesn't contain a valid partition table

Then, we need to make a partition:

[root@halinux ~]# fdisk /dev/vpatha Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-1018, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-1018, default 1018): Using default value 1018 Command (m for help): t Selected partition 1 Hex code (type L to list codes): l 0 Empty 1e Hidden W95 FAT1 75 PC/IX be Solaris boot 1 FAT12 24 NEC DOS 80 Old Minix bf Solaris 2 XENIX root 39 Plan 9 81 Minix / old Lin c1 DRDOS/sec (FAT- 3 XENIX usr 3c PartitionMagic 82 Linux swap c4 DRDOS/sec (FAT- 4 FAT16 <32M 40 Venix 80286 83 Linux c6 DRDOS/sec (FAT- 5 Extended 41 PPC PReP Boot 84 OS/2 hidden C: c7 Syrinx 6 FAT16 42 SFS 85 Linux extended da Non-FS data 7 HPFS/NTFS 4d QNX4.x 86 NTFS volume set db CP/M / CTOS / . 8 AIX 4e QNX4.x 2nd part 87 NTFS volume set de Dell Utility 9 AIX bootable 4f QNX4.x 3rd part 8e Linux LVM df BootIt a OS/2 Boot Manag 50 OnTrack DM 93 Amoeba e1 DOS access b W95 FAT32 51 OnTrack DM6 Aux 94 Amoeba BBT e3 DOS R/O c W95 FAT32 (LBA) 52 CP/M 9f BSD/OS e4 SpeedStor

e W95 FAT16 (LBA) 53 OnTrack DM6 Aux a0 IBM Thinkpad hi eb BeOS fs

f W95 Ext'd (LBA) 54 OnTrackDM6 a5 FreeBSD ee EFI GPT 10 OPUS 55 EZ-Drive a6 OpenBSD ef EFI (FAT-12/16/

17

11 Hidden FAT12 56 Golden Bow a7 NeXTSTEP f0 Linux/PA-RISC b 12 Compaq diagnost 5c Priam Edisk a8 Darwin UFS f1 SpeedStor 14 Hidden FAT16 <3 61 SpeedStor a9 NetBSD f4 SpeedStor 16 Hidden FAT16 63 GNU HURD or Sys ab Darwin boot f2 DOS secondary 17 Hidden HPFS/NTF 64 Novell Netware b7 BSDI fs fd Linux raid auto 18 AST SmartSleep 65 Novell Netware b8 BSDI swap fe LANstep 1b Hidden W95 FAT3 70 DiskSecure Mult bb Boot Wizard hid ff BBT 1c Hidden W95 FAT3 Hex code (type L to list codes): 8e Changed system type of partition 1 to 8e (Linux LVM) Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.

Thanks to the new sdd service, the new commands are accessible which permit to see the

LUN, the relation between LSS and the volumes:

[root@halinux ~]# datapath query essmap Disk Path P Adapter LUN SN Type Size LSS Vol Rank C/A S Connection Port RaidMode ----- ----- - ------------- ----------- ----------- ---- --- --- ---- --- - ----------- ---- -------- vpatha sdb Host1Channel0 75065514700 IBM 2107900 1.0GB 71 00 0000 17 Y R1-B3-H1-ZC 202 RAID5 vpatha sdc Host1Channel0 75065514700 IBM 2107900 1.0GB 71 00 0000 17 Y R1-B4-H3-ZD 333 RAID5 [root@halinux ~]# datapath query adapter Active Adapters :1 Adpt# Name State Mode Select Errors Paths Active 0 Host1Channel0 NORMAL ACTIVE 136 0 2 0 [root@halinux ~]# datapath query device Total Devices : 1 DEV#: 0 DEVICE NAME: vpatha TYPE: 2107900 POLICY: Optimized Sequential SERIAL: 75065514700 =========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 Host1Channel0/sdb CLOSE NORMAL 53 0 1 Host1Channel0/sdc CLOSE NORMAL 83 0

To create physical volumes, volumes group, logical volumes, a serial of command has to be

made before mounting the disk on /halinux1:

[root@halinux ~]# pvcreate /dev/vpatha1 Physical volume "/dev/vpatha1" successfully created [root@halinux Desktop]# pvscan

18

PV /dev/sda2 VG VolGroup00 lvm2 [33.78 GB / 64.00 MB free] PV /dev/vpatha1 VG VGHALINUX1 lvm2 [4.99 GB / 4.00 MB free] Total: 2 [38.77 GB] / in use: 2 [38.77 GB] / in no VG: 0 [0 ] [root@halinux ~]# vgcreate VGHALINUX1 /dev/vpatha1 Volume group "VGHALINUX1" successfully created [root@halinux Desktop]# vgscan Reading all physical volumes. This may take a while... Found volume group "VolGroup00" using metadata type lvm2 Found volume group "VGHALINUX1" using metadata type lvm2 [root@halinux Desktop]# vgdisplay VGHALINUX1 --- Volume group --- VG Name VGHALINUX1 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 2 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 1 Max PV 0 Cur PV 1 Act PV 1 VG Size 4.99 GB PE Size 4.00 MB Total PE 1278 Alloc PE / Size 1277 / 4.99 GB Free PE / Size 1 / 4.00 MB VG UUID 5P3JKI-ztYt-eQ4p-zzAp-4tVF-jrJW-4ZK7PN [root@halinux ~]# pvscan PV /dev/vpatha1 VG VGHALINUX1 lvm2 [4.99 GB / 4.99 GB free] Total: 1 [4.99 GB] / in use: 1 [4.99 GB] / in no VG: 0 [0 ] [root@halinux ~]# lvcreate VGHALINUX1 -n lv_halinux1 -l 1277 Logical volume "lv_halinux1" created [root@halinux Desktop]# lvscan ACTIVE '/dev/VolGroup00/LogVol00' [31.78 GB] inherit ACTIVE '/dev/VolGroup00/LogVol01' [1.94 GB] inherit ACTIVE '/dev/VGHALINUX1/lv_halinux1' [4.99 GB] inherit [root@halinux /]# mkdir /halinux1 [root@halinux /]# ls bin etc initrd media opt sbin sys usr boot halinux1 lib misc proc selinux tftpboot var dev home lost+found mnt root srv tmp [root@halinux /]# mount /dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw) none on /proc type proc (rw) none on /sys type sysfs (rw)

none on /dev/pts type devpts (rw,gid=5,mode=620)

usbfs on /proc/bus/usb type usbfs (rw) /dev/sda1 on /boot type ext3 (rw) none on /dev/shm type tmpfs (rw)

19

none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) [root@halinux ~]# lvscan ACTIVE '/dev/VolGroup00/LogVol00' [31.78 GB] inherit ACTIVE '/dev/VolGroup00/LogVol01' [1.94 GB] inherit inactive '/dev/VGHALINUX1/lv_halinux1' [4.99 GB] inherit [root@halinux ~]# lvchange /dev/VGHALINUX1/lv_halinux1 -a y [root@halinux ~]# lvscan ACTIVE '/dev/VolGroup00/LogVol00' [31.78 GB] inherit ACTIVE '/dev/VolGroup00/LogVol01' [1.94 GB] inherit ACTIVE '/dev/VGHALINUX1/lv_halinux1' [4.99 GB] inherit [root@halinux ~]# mount /dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw) none on /proc type proc (rw) none on /sys type sysfs (rw) none on /dev/pts type devpts (rw,gid=5,mode=620) usbfs on /proc/bus/usb type usbfs (rw) /dev/sda1 on /boot type ext3 (rw) none on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) [root@halinux ~]# mkfs.ext3 /dev/VGHALINUX1/lv_halinux1 mke2fs 1.35 (28-Feb-2004) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 654080 inodes, 1307648 blocks 65382 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=1342177280 40 block groups 32768 blocks per group, 32768 fragments per group 16352 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Writing inode tables: done Creating journal (8192 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 30 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. [root@halinux ~]# mount /dev/VGHALINUX1/lv_halinux1 /halinux1/ [root@halinux ~]# cd /halinux1 [root@halinux halinux1]# ll total 16 drwx------ 2 root root 16384 Aug 18 19:12 lost+found

20

2.5 Script

When path problem is resolved and Linux and unit storage configured, a starting script has to

be made. If you restart the system without made this step (in order to see the elements which

have been take into account), you can see that sdd service is not started by default. Then, the

administrator’s intervention on the system in order to start it, it is needed. However, the

logical volume created on Linux appeared after each starting with a status Inactive, and

needed a user’s operation in order to give it a status Active and then he had to make a mount.

You can realize a starting script permitting to restart the operating system without reconfigure

and without root’s intervention. Here is the script:

#!/bin/bash # # script_sdd This shell script takes care of starting and stopping # sdd and configurating Linux. # # description: sdd is a service which is the program # that permits to make multipathing ; and lvchange to change # the logical volume status. # processname: sdd # Stop sdd . /etc/init.d/sdd stop # Start sdd . /etc/init.d/sdd start # Source sdd configuration. # Physical volume scan pvscan # Volume Groupe scan vgscan # Logical volume scan lvscan # Change logical volume lv_halinux1 to active lvchange /dev/VGHALINUX1/lv_halinux1 -a y # Logical volume scan lvscan mount /dev/VGHALINUX1/lv_halinux1 /halinux1 exit 0

21

This script is present on the two sites in order to automate configuration and in order the user

not to have to intervene after each restart. Once the script is written, you have to configure

Linux to take it into account each start. That is the procedure to realize: you have to place the

script in the directory /etc/init.d, put it in executable mode (that is to say -rwx-r-xr-x) and then

make links to the different rc.d by the command ln-s (cf. below) in order to permit its taking

into account at each restart.

[root@halinux ~]# [root@halinux ~]# chmod 755 script_sdd [root@halinux ~]# [root@halinux ~]# ln -s ../init.d/script_sdd /etc/rc2.d/S81script_sdd_site1 [root@halinux ~]# ln -s ../init.d/script_sdd /etc/rc3.d/S81script_sdd_site1 [root@halinux ~]# ln -s ../init.d/script_sdd /etc/rc4.d/S81script_sdd_site1 [root@halinux ~]# ln -s ../init.d/script_sdd /etc/rc5.d/S81script_sdd_site1

2.6 Dscli

When the configuration above and the starting script are finished, Linux is integrated on the

platform and copy services can be tested thanks to the service dscli permitting to

communicate with the unit storage.

First you have to access to the contains of the directory you have downloaded:

[root@halinux ~]# cd /opt [root@halinux opt]# mkdir dscli [root@halinux ~]# mount DSCLI-5.1.720.8.iso /opt/dscli -t iso9660 -o loop [root@halinux ~]# [root@halinux ~]#

Then, you have to uninstall old dscli versions before running this install. You need to have

Java Runtime Environment (JRE) installed on your target machine before starting the install.

IBM Java 1.4.2 is the preferred JRE on Linux and strongly recommended, although dscli can

support any Java 1.4.1.

[root@halinux dscli]# rpm -ivh IBMJava2-JRE-1.4.2-0.0.i386.rpm [root@halinux dscli]# [root@halinux dscli]#

Thus you can run setupgenericunix.sh. The installer lets you choose where dscli gets installed.

For instance, on Linux, the default is /opt/ibm/dscli. Then you run setuplinux.bin.

[root@halinux dscli]# ./setupgenericunix.sh [root@halinux dscli]# ./setuplinux.bin

22

In order to have installation in text mode than in GUI, you have to precise -console as

mentioned below:

[root@halinux dscli]# ./setupgenericunix.sh -console [root@halinux dscli]# ./setuplinux.bin -console

The command dscli is only available in the directory /opt/bin/dscli. In order to access it all

over the directory, you need to modify .bashrc:

[root@halinux dscli]# vi ~/.bashrc # .bashrc # User specific aliases and functions alias rm='rm -i' alias cp='cp -i' alias mv='mv -i' # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi PATH=$PATH:.:/opt/ibm/dscli/ [root@halinux dscli]# dscli

23

3 Special Thanks

I would like to thank the many people from PSSC who helped me spending time, energy and

expertise:

� Christian Casanova for the SAN and Network cabling and configuration. � Christophe Majek and Said Kemiche for the DS8000 setup and support. � Alexandre Chabrol for his help on System x.

Special thanks too, for Christine O’Sullivan and Emmanuel Tong-Viet who help me all over

my research.