lms 6.9 upgrade launch day plan

6.9 SP1 and additional patches05/08/2014

LMS Upgrade Plan

Pre-Implementation GroundworkItems that must be completed before launch date

3

Pre-Implementation Groundwork

Task Responsible StatusDelete all DA Profiles

Warren Complete

Stage software to all servers

Warren Complete

Complete UAT and resolve all issues

Team 80% Complete

Place student center replication on hold

Tina Complete

4


Task Responsible StatusBuild replication job for student center to NTVLMSWPA

Tina Complete

PSWR to hold all batch jobs

Warren Complete

Compose message for splash page fro ANSO and Answers.net

Tina Complete

5


Task Responsible StatusComplete design changes

David In-Progress

Capture snapshot of NTVLMSBPA, NTVLMSWPA and NTVLMSAPA

Nick Complete

Stage config file in turnover directory.

Michael Complete

Update Timeout Setting on Web Servers

Warren In-ProgressDEV/QA complete

6


Task Responsible StatusCreate new Training Materials

L&D New

Launch Day Tasks and Timeline

8

Launch Day Tasks and Timeline1

Task Responsible Start Time DurationShut down all services and clear queue

Warren 10:00 PM 15 min

Post Splash Pages

Tina/Gannon 10:00 pm 15 min

Capture backup of production database

David 10:15 pm 20 min

9


Task Responsible Start Time DurationRestore backup to new Database Server


Update UDLs


Update DSN’s


10


Task Responsible Start Time DurationRun pre-install script


Install 6.9 SP1 to batch server

Warren 11:05 pm 30 min

Initiate Database Upgrade

Warren 11:35 30 min

11


Task Responsible Start Time DurationInstall additional PRNs on batch server

Warren 12:05 am 30 min

Initiate DB upgrade


Install 6.9 SP1 to application server


12


Task Responsible Start Time DurationInstall PRNs to application server

Warren 1:15 30 min

Install 6.9 SP1 to web server

Warren 1:45 30 min

Install PRNs to web server

Warren 2:15 30 min

13


Task Responsible Start Time DurationCopy student centers to replication server

Warren 2:45 10 min

Modify default app pool to .NET 4.0

Tina 2:55 3 min

14


Task Responsible Start Time DurationCreate new app pool for reports and set to .NET 2.0

Tina 2:58 3 min

Move reports to new app pool

Tina 3:01 4 min

15


Task Responsible Start Time DurationDeploy Learning Center

Warren/David/Tina

3:05 am 30 min

Activate replication for web server

Tina 3:35 am 5 min

Install Custom .NET 4.0 files on web server

Warren 3:50 10 min

16


Task Responsible Start Time DurationCheck the physical path of the default web site

Warren 4:00 10 min

Re-start all services

Warren 4:10 15 min

Save calender styles

Warren 4:25 15 min

System Testing & Resolution

Warren 4:40 1 hour

17


Task Responsible Start Time DurationUAT Testing L&D 5:25 am 2 hoursGo/No Go Decision

All 7:25 5 min

Take down Splash Page

Tina/Gannon 7:30 10 min

PSWR to move file and release batch jobs.

Warren 7:30 30 min

Deploy Clones

Nick 7:30 2 hours

Backout Plan

19

Backout Plan

Task Responsible Start Time DurationRestore Batch Server from Snapshot

Nick 7:30

Start Services on application server B

Warren 7:30 15 min

Take web server out of the VIP

Nick

20

Backout Plan

Task Responsible Start Time DurationRestore web server

Nick

Restore application server A

Nick

Restart services on application server A

Warren 5 min

Open Issues

22

Issues Register

Issue Responsible StatusCan’t delete Content Objects

Warren Resolved

Replication Failure

Nick/Tina Resolved

Course not compatible with I.E. 8

Fran In-Progress

Manager Assigning Courses

Warren In-Process

23

Issues Register

Issue Responsible StatusChange Over of Learning Centers

Dave/Warren In-Progress

Unable to Populate Groups

Fran/Warren Resolved

Reporting will be Delayed

Dave Communication

Re-Design of Non-Retail

Dave In-Progress

24

Issue Responsible StatusReporting tables not available

David Mitigation

Loss of LC Changes

Warren In-Progress

Issues Register

25

Results of Forensic InvestigationLoss of LC Settings

Root Cause:• From 2:01 PM to 4:02 PM 05/21/2014 the following message was

found in the web server logs

“Module:w3wp Error: Server time-out: unable to complete request. Source:pscsock.cpp Line:340 Lc:8 Close Session:1”

• What has been done so far

• Restored from On-Line back up on 5/22/2014

• Restore from archived backup from 11:00 pm 05/21/2014

• Data was not found

Mitigation for the future• Installation of Scom alert for this message• Nightly export of Learning Center changes

27

Risks

Risk Responsible StatusStaff Change TEAM See backup chartTraining Material Not ready

L&D New

No Approval for new Design

David New

28

Backup Plan

Task Responsible BackupSoftware Install Warren UnknownServer Configuration

Tina Warren

Database work David WillCloning Nick UnknownTesting Fran KimReplication Tina WarrenExport/Import LCs

Warren Unknown

lms 6.9 upgrade launch day plan

Documents