comdisco confidential

31
WORLD TRADE CENTER And Related Recoveries Len Guddemi Vice President Northeast Operations Comdisco Confidential

Upload: janine

Post on 04-Jan-2016

31 views

Category:

Documents


3 download

DESCRIPTION

WORLD TRADE CENTER And Related Recoveries Len Guddemi Vice President Northeast Operations. Comdisco Confidential. SunGard Worldwide Disaster Support Experience. Recoveries per Year. Environmental 8/2%. Software 10/2%. Telecomm 14/3%. Terrorist Attack 100/19%. Bomb 16/3%. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Comdisco Confidential

WORLD TRADE CENTERAnd Related Recoveries

Len GuddemiVice PresidentNortheast Operations

Comdisco Confidential

Page 2: Comdisco Confidential

0

20

40

60

80

100

'81 '82 '83 '84 '85 '86 '87 '88 '89 '90 '91 '92 '93 '94 '95 '96 '97 '98 '99 '00 '01

SunGard Worldwide Disaster Support Experience

Recoveries per Year

Page 3: Comdisco Confidential

Terrorist Attack 100/19%

Hardware 97/18%

Hurricane 84/16%Power Outage 76/14%

Flood 57/11%

Fire/Explosion 27/5%

Miscellaneous 20/4%

Earthquake 19/4%

Bomb 16/3%

Telecomm 14/3%

Software 10/2%

Environmental 8/2%

Data Center

7/1%

Civil Unrest 2/0%

Disruption Cause: Number/Percentage

SunGard Worldwide Disaster Support Experience

Page 4: Comdisco Confidential

DEC 43/8%

AS/400 56/10%

Sun 16/3%

Item Processing 5/1%

NT Server 4/1%

Trading Floor 19/4%

Advanced Recovery 13/2%

Miscellaneous 26/5%

HP 22/4%

Workarea Recovery 130/24%

Mainframe 138/26%

RS/6000 10/2%

Unisys 2/0%

Mobile Cluster 3/1%

Systems On-site Mobile 39/7%

Tandem 3/1%

Shell 7/1%

Stratus 1/0%

SunGard Worldwide Disaster Support Experience

Recovery Platforms Used: Quantity/Percentage

Page 5: Comdisco Confidential

Terrorist Attack 1154/20%

Flood 691/12%

Hardware Problem 666/12%

Power Outage 370/7%

Miscellaneous 858/15%

Fire/Explosion 359/6%

Hurricane 318/6%

Data Center Move 269/5%

Earthquake 136/2%

Bomb 146/3%Environmental 573/10%

Civil Unrest 3/0%

Telecomm 30/1%

SunGard Worldwide Disaster Support Experience

Days of Occupancy: Days/Percentage

Page 6: Comdisco Confidential

Declaration Information

Number of Events

Overall - 93 Schedules, 47 Customers

Simultaneous High Point - 76 Declarations, 16 Facilities, 14 Platforms

Comdisco Confidential

Reason

Facilities Unavailable

Loss of Power, telecommunications, denied access to buildings,

concerns over building structural safety

Page 7: Comdisco Confidential

Atlanta: Work Area Seats, Remote Sun,

AS400, RS6000

Carlstadt: Voice/Work Area, Business

Recovery/Quick Ship, Trading Floor,

Dedicated Seats, SUN, AS400,

DEC/VAX, RS6000, HP9000, Intel

Based Servers; Dedicated HP9000,

RS6000, NT Servers

Boston: SUN, Work Area

Cypress: Work Area, Remote Tandem

Mobile Recovery: 13 Units/5,000sq.ft., 10

Units/3,000sq.ft. for Work Area

Minnetonka: Work Area

Comdisco Confidential

North Bergen: Voice/Work Area,

Main-Frame, ARS & Networking

Phoenix: Work Area

Queens: Work Area, Trading Floor,

Dedicated, Trading PIT

Rosemont: Work Area

San Ramon: Work Area, Trading Floor

Wood Dale: DEC/VAX, DEC/Alpha,

VAX3100, VAX7620, Filenet, AS400,

Tandem

Wood Cliff Lake: Intel Servers, Work Area

Jersey City, workgroup, AS/400

Philadelphia: Work area, RS6000,

AS/400,Mainframe, Voice

Herndon: Work Area

Warminster: Work Area,Unisys

SunGard Sites & Platforms

Page 8: Comdisco Confidential

Declaration Information - Continued

Under-subscription or additional needs assessment remains an issue Many businesses requested additional needs ATOD

Responded to over 500 special requests for service. Built out with best efforts

This puts recovery & business continuity at risk due to extended recovery timeline.

Personnel & Intellectual Property Keys to Recovery Methodologies, ideas and documented plans utilized. Assigned backup roles and succession planning implemented. Intellectual capital and personnel in alternate locations. Use of professional organizations to augment staff.

Comdisco Confidential

Items of Note Implement Your Plan & Associated Vendor Services with Urgency

14 Declarations in 30 minutes; 24 within 1 hour; 19 in 2nd hour You will be in competition with other business for services & manufacturers

Quickly establish damage assessment & need for services, and implement

Page 9: Comdisco Confidential

Declaration Information - Continued

Air Travel Unavailable Delayed delivery of offsite stored media to recovery location

Identify alternate means of transportation Assess need for electronic journaling of data to alternate location

Expert staff unable to travel Identify options for local access by staff to remote alternate systems (and

possibly home systems) Assess value of ‘turnkey’ type services with vendors

Offsite Storage Critical Evaluate for frequency, completeness Paper records and transactions may be totally lost Evaluate if your offsite storage is far enough away Evaluate need/value of electronic journaling

Comdisco Confidential

Items of Note - Continued

Reluctance to Travel End Users generally won’t travel far from home.

Ensure adequate end user facilities available nearby (about 50 miles)

Page 10: Comdisco Confidential

Declaration Information - Continued

Active Testers More successful at meeting RTO/RPO More realistic understanding of application performance Fewer problems

Very High Demand for Voice Recovery Realistically evaluate needs: line trunking, extensions, call recording, voice

mail, call forwarding Avoid Fast Busy

Comdisco Confidential

Items of Note - Continued High Availability Solutions Extremely Successful

Electronic Journaling, Data Base Mirroring, SRDF, etc. 45 Terabytes ready very quickly Data requiring physical transport delayed

Page 11: Comdisco Confidential
Page 12: Comdisco Confidential
Page 13: Comdisco Confidential
Page 14: Comdisco Confidential
Page 15: Comdisco Confidential
Page 16: Comdisco Confidential
Page 17: Comdisco Confidential
Page 18: Comdisco Confidential
Page 19: Comdisco Confidential
Page 20: Comdisco Confidential
Page 21: Comdisco Confidential
Page 22: Comdisco Confidential

SunGard Challenges & Tactics Disaster Hotline & Notification

Increased Staff handling incoming Disaster calls

Security Immediate increase in security staff on all shifts. Contacted local authorities to assist in access control

Additional Staff Request for additional support staff Mobil Recovery Team

Additional Resource Contacted third party vendors Arranged additional parking with neighbors

Page 23: Comdisco Confidential

SunGard Challenges & Tactics

Implemented Swat Support Teams Established 24/7 Command Center at Corporate HQ with Open Bridge SunGard Canadian employees, Illinois, Minneapolis, and Texas employees

drove to Carlstadt & North Bergen to help SunGard’s Professional Services Consultants, Sales, Sales Support,

Administration and SunGard’s Operations staff worked side by side. Dispatched relief support from other SunGard facilities once air travel restored Activated facilities, equipment procurement, shipping and solutions teams to

respond to special requests Received offers of help from customers

Gratefully declined Network Alternate Routing Critical

Significant damage due to the event Fiber cut after event by construction/demolition crews SunGard alternate routings worked without issue Vendors terrific with T1s, DS3 installations in days

Comdisco Confidential

SunGard Facilities and Staff not directly affected

Page 24: Comdisco Confidential

Customer Challenges & Tactics

Comdisco Confidential

Inaccurate Needs Assessment

Special requests for PCs, & Servers IP addressing Storage Tape Drives Electricity Processing power LAN bridging Televisions & Printers Network capability, Voice services & Routers Faxes Software and more

Page 25: Comdisco Confidential

Customer Challenges & Tactics Logistics

Travel Restriction an Impact - used ground travel for many shipments Shipped over 4,200 PCs along with much other equipment Customer parking, bussing, and catering

Comdisco Confidential

Getting Restoration Media to Alternate Site Transportation challenge Complete set available and labeled Missing or unavailable documentation

Start-up scriptsRun instructions

Page 26: Comdisco Confidential

Learned / Re-Affirmed

Important to Have a Team Leader for Each Shift

Integration of recovery requirements into change control Number of PCs, Positions Network Capabilities Processors, disk, tape capabilities Printers, fax machines PBX features & functionality Platform Technology upgrades, changes, maintenance

Importance of Quick Assessment & Implementation of Plan when appropriate

Comdisco Confidential

Value in Pre-Planning Staff Placement within Rooms End User priorities & usage End User equipment requirements – avoids setting up more than once

Page 27: Comdisco Confidential

Learned / Re-Affirmed - Continued Value of Solid Backup Records Program

Tape, electronic, & paper records foundation of recovery Evaluate right balance for your company

Documented & labeled so vendor(s) can assist Have scripts shipped with restore media or stored at alternate location Frequency of backups Offsite location & transportation dependencies Examine benefit of electronic transmission of offsite storage data

Can significantly improve recovery time and reduce data loss

Comdisco Confidential

Evaluate Strategy of Shared versus Dedicated Resources Guarantees resource access for some number of most critical requirements

Complement with shared resources

PC Image Restoration Tool Effective Ghost, or similar

Page 28: Comdisco Confidential

Learned / Re-Affirmed - Continued Basics are Fundamental

Establish personnel roles & assign backup

Document methodologies, ideas, business plans & recovery plans

Review, update & follow established procedures

Test the way you would recover, recover the way you test

Worthwhile to review how home procedures apply in alternate site production

mode

Offsite Storage, Disk mirroring, Miscellaneous other

Comdisco Confidential

Revisit your Plan & Recovery Requirements Often – Technology & Business

In Times of Need, Help Comes From Many Sources

Page 29: Comdisco Confidential

Customer Comments

Comdisco Confidential

“Although operating in our primary location is preferred, the equipment, professional and courteous staff, and outstanding overall accommodations provided for an effective and successful business recovery. I am glad that we have contracted with your organization!”

“I have in the past worked with a number of outstanding individuals, but never before have I worked with such a highly functional, customer service oriented team. Individually each team member I have had the pleasure to deal with was exceptional; as a team there is none better.”

Page 30: Comdisco Confidential

Customer Comments

“Thanks for all the support and assistance. We’re glad you are here.”

Comdisco Confidential

“I would like you to know of the "more than wonderful" service we are receiving at the SunGard site. ... have gone out of their way to assist us with some special needs. ... have answered numerous questions and assisted --she is great. ... worked with ENS last night, assisted me with set up so I can stay on line (due to the situation in New York and the weather closing down our site) and has been a great help to the IT staff.”

“I would like you to know it is people like this that make SunGard a vendor we choose to depend on in the event of an emergency. Please pass this message along to their managers, it would be greatly appreciated.”

Page 31: Comdisco Confidential

Questions and Answer

Comdisco Confidential