Last update: Feb 3, 2009Version 2.0

The week before the change:A. Check the system microcode level to ensure it can handle the OS being installed. lsattr -El sys0 | grep fw (ex: for 9113, AIX 5.3 requires microcode SF240_358 or higher) If its not high enough, bring it to your teamlead's attention to remediate prior to the change.

Verify the system isn’t a Regatta, or LPAR. Models 7040-681 and 7038-6M2 are regattas.

The day before the change:B. Ensure that bos.alt_disk_install* is installed on the nim server and the spot which will be used. If not then backup and install. C. The level of the NIM server rootvg, lpp_source and SPOT must be at the same level.

D. Check the nim server:1. verify rsh to client is working (ensure ~root/.rhosts perm is 600) 2. verify NFS export to client is working 3. make sure there are no NFS exports to the NIM client after validation 4. verify name resolution of client via nslookup

E. Check the nim client:1. verify rsh to server is working (ensure ~root/.rhosts perm is 600) 2. verify NFS export to server is working 3. make sure there are no NFS exports to the NIM client after validation 4. verify name resolution of server via nslookup 5. check that /etc/niminfo file exists and is accurate. If not, run smitty nim and select " Configure Network Installation Management Client Fileset to add it6. Run /adm/bin/snapit.ksh to document the filesystems currently mounted7. Run /adm/backup/aix/scripts/recovery.info8. Run /adm/public/sanitychk_aix.ksh and note any possible issues. Any issues, ask your teamlead for assistance.

The change:F. Notification1. Contact Operations to put the server on maintenance. 2. Notify the Basis team that the online server work is beginning.

G. On the nim_client (server to be upgraded)1. Add "exit 0" near the beginning of /usr/local/etc/rc.local startup script 2. Commit all applied software smitty commit3. Unmirror rootvg of the system where the alt disk install will be performed. unmirrorvg -c1 rootvg hdiskx (where x is for altinst_rootvg) 4. If the rootvg contains data on both disks migrate from one disk to the other. migratepv hdiskx hdisky (where x is for altinst_rootvg) 5. Run bosboot on the disk that now contains all the rootvg data bosboot -ad /dev/hdisky (where y is the rootvg disk) 6. Verify that the system will boot off rootvg in case there is a problem

bootlist -om normal hdisky (where y is the rootvg disk) 7. Ensure that the quorum is not active chvg -Qn rootvg 8. Reduce target hdisk from the rootvg that no longer contains rootvg data reducevg rootvg hdiskx (where x is for altinst_rootvg) 9. start nfsd (temporarily) smit nfs

H. On the nim_server: 1. Ensure the client is defined to the nim server (lsnim -l oldtutxxx) or define it. Example: nim -o define -t standalone -a platform=chrp -a netboot_kernel=mp -a if1="168_146_x_x oldtutxxx 0" -a cable_type1=tp oldtutxxx where x specifies the client server hostname and IP address (i.e. oldtut102 and 168.146.19.187). 3. Start the clone and migration (use appropriate values)

Nimlpar# echo "/usr/sbin/nimadm -c'oldtutxxx' -l'aix5308_lppsource' -s'aix5300-06-02_spot’ -d'hdisk1' '-V' '-D' '-Y'" | at now

NOTE: For most 5.2 to 5.3 migration, this takes 5hrs to run.

4. The install can be monitored from the NIM server using tail -f. The log file is in /var/adm/ras/alt_mig. Example: tail -f oldtutxxx_alt_mig.log

I. On the nim_client: 1. Verify the bootlist so that the system will boot off altinst_rootvg (this should have been done by the alt_disk_install )bootlist –m normal –oif its not set correctly, investigate the reason. If ok to proceed, then set it manually:bootlist m normal -o hdiskx (where x is the altinst_rootvg disk) 2. Stop all applications/databases 3. Reboot shutdown –Fr

J. Verification and cleanup 1. Run oslevel –s to see expected result (5300-08-03-0831)2. Run mlcheck and lssecfixes and apply missing filesets, security fixes and hipers as needed until clean.3. Run lppchk –v –m 3

Common error msg:lppchk: The following filesets need to be installed or corrected to bring the system to a consistent state:

devices.common.IBM.mpio.rte 5.2.0.0 (not installed; requisite fileset) devices.common.IBM.mpio.rte 5.2.0.106 (usr: not installed, root: COMMITTED)

To fix:# cd /# tar -cvf /tmp/odm.tar ./etc/objrepos ./usr/lib/objrepos ./usr/share/lib/objreposodmget -q name=devices.common.IBM.mpio.rte lpp

Note the "lpp_id "

lpp: name = "devices.common.IBM.mpio.rte" size = 0 state = 1 cp_flag = 275 group = "" magic_letter = "I" ver = 5 rel = 2 mod = 0 fix = 30 description = "MPIO Disk Path Control Module" lpp_id = 628

odmdelete -q name=devices.common.IBM.mpio.rte -o lppodmdelete -q lpp_name=devices.common.IBM.mpio.rte -o productodmdelete -q lpp_id=<lpp_id> -o inventory odmdelete -q lpp_id=<lpp_id> -o history

Check nothing is showing when u run#odmget -q name=devices.common.IBM.mpio.rte lppexport the ODM dir export ODMDIR=/usr/lib/objreposand follow the same #odmget -q name=devices.common.IBM.mpio.rte lpp

Make sure nothing is listing.If it is, repeat the 4 steps with the new lpp_id.

Export ODMDIR=/etc/objrepos

Install the latest from nimlpar:

#mount nimlpar:/AIXFILESETS /mnt#cd /mnt/AIX5308baseinstall the fileset devices.common.IBM.mpio

#smitty installp

Update the fileset "devices.common.IBM.mpio" to latest version #cd /mnt/AIX5300-08-03_new#smitty update_all (or latest)select above and update Reboot afterwards.

Check that there’s no “APPLY-HOLD” or “COMMIT-HOLD” status:# lslpp -la devices.common.IBM.mpio.rte

4. Check for filesets not upgraded to current level and update as needed instfix -i | grep ML or instfix -i | grep ":-:"NOTE: if additional filesets are needed and should be added to the lppsrc, put them in the appropriate STAGE directory and send an upgrade request to your teamlead - do not update lppsrc or the spot directly.

5. Run oslevel -s to ensure it is reporting the expected level 6. Commit the AIX 5.3 filesets:

# smit commit7. Clear error logs errclear 0 /usr/lib/errstop /usr/lib/errdemon 8. Reconfigure /etc/mail/sendmail.cf (reference teamroom document for sendmail) 9. Update /etc/motd 10. Ensure that aio is enabled (available) on the new image if needed (may require a reboot) lsattr -El aio0 (or smit aio)11. Remove "exit 0" from /usr/local/etc/rc.local startup script if it was added previously.12. Run /adm/bin/snapcomp.ksh and mount missing filesystems.13. Contact application team to start applications and verify everything looks ok.14. Contact Operations to take the server off maintenance.

The next day:L. Do a mksysb of 5.3 to the nim server: 1. Copy the mksysb (AIX 5.2 one) and then on 2. Do a mksysb of the current 5.3 server [# /adm/backup/aix/scripts/mksysb_nfs.ksh ]

After 1 week or the date set by the SAP SDM 1. Remove the old_rootvg alt_disk_install -X old_rootvg 2. Extend rootvg to include the hdisk extendvg -f rootvg hdiskx (where x is where old_rootvg was) 3. Re-mirror the rootvg nohup mirrorvg rootvg hdiskx & (where x is where old_rootvg was) 4. Wait for the logical volumes to be in sync 5. Run bosboot on each rootvg disk bosboot -ad /dev/hdisk0bosboot -ad /dev/hdisk1 6. Verify that the system can boot off each disk in rootvg in case there is a problem bootlist -om normal hdisk0 hdisk1 7. Turn off the quorum on rootvg chvg -Qn rootvg 8. /adm/public/sanitychk_aix.ksh and make sure everything looks ok. If not, notify your teamlead.