jd edwards & peoplesoft 2 _ mark elley _ real word experiences disaster recovery.pdf
TRANSCRIPT
The most comprehensive Oracle applications & technology content under one roof The most comprehensive Oracle applications & technology content under one roof
Real World Disaster Recovery
Mark Elley
The most comprehensive Oracle applications & technology content under one roof
What is a Disaster? Definition: dis·as·ter noun /diˈzastər/ disasters, plural
• A sudden event, such as an accident or a natural catastrophe, that causes great damage or loss of life- 159 people died in the disaster
• - disaster struck within minutes of takeoff • An event or fact that has unfortunate
consequences- a string of personal disasters • - reduced legal aid could spell financial disaster • A person, act, or thing that is a failure- my perm is
a total disaster
The most comprehensive Oracle applications & technology content under one roof
Disaster
Bad Perm
The most comprehensive Oracle applications & technology content under one roof
Types of Disaster • Natural Disaster
– Earthquake – Tsunami – LiquefacEon
• Other Disaster – Fire – Flood – Infrastructure failure
The most comprehensive Oracle applications & technology content under one roof
SENDAI Japan
March 2011
The most comprehensive Oracle applications & technology content under one roof
Japan -‐ Earthquake & Tsunami
The most comprehensive Oracle applications & technology content under one roof
Building Damage
The most comprehensive Oracle applications & technology content under one roof
Infrastructure Damage
The most comprehensive Oracle applications & technology content under one roof
Tsunami
The most comprehensive Oracle applications & technology content under one roof
Christchurch New Zealand
September 2010
& February 2011
The most comprehensive Oracle applications & technology content under one roof
Christchurch -‐ Earthquakes
The most comprehensive Oracle applications & technology content under one roof
Building Damage
The most comprehensive Oracle applications & technology content under one roof
DestrucEon & Human Tragedy
The most comprehensive Oracle applications & technology content under one roof
Infrastructure Damage
The most comprehensive Oracle applications & technology content under one roof
Brisbane Australia
January 2011
The most comprehensive Oracle applications & technology content under one roof
Inundated
The most comprehensive Oracle applications & technology content under one roof
Stock Recovery
The most comprehensive Oracle applications & technology content under one roof
Server Recovery
The most comprehensive Oracle applications & technology content under one roof
All of the Disasters above have occurred in the last
12 Months
The most comprehensive Oracle applications & technology content under one roof
What about Localised Disasters?
The most comprehensive Oracle applications & technology content under one roof
Disaster -‐ FIRE
The most comprehensive Oracle applications & technology content under one roof
Disaster FLOOD
The most comprehensive Oracle applications & technology content under one roof
Infrastructure Failure
1998 Auckland CBD Three Week Outage Container Ships plugged into the grid to provide some supply…
The most comprehensive Oracle applications & technology content under one roof
Could Your Business Survive A Disaster
The most comprehensive Oracle applications & technology content under one roof
Disaster Recovery
• Focussed Overview of Recent Experience • Predominantly with JD Edwards in mind
– Purpose • Lessons learned from the field • What works best / What doesn’t? • What fits your business expectaEons for:
– Return to OperaEon (RTO) ? – How long can the business survive without this core system
• Is distance a factor?
The most comprehensive Oracle applications & technology content under one roof
Business ConsideraEons • Human Resource Issues
– Staff dealing with personal & family issues – Key staff oeen have to step up beyond the call!
• Site Access – Days to get an engineer to give the OK to site access – Building irreparably damaged – Servers & server room destroyed – Physical constraints – i.e Roading – And Many More!!!
The most comprehensive Oracle applications & technology content under one roof
Provoking Thought • What is in place today if a disaster occurred • Do you have a true DR strategy or HA only? • Could you be confident that your business would conEnue to operate in the event of a disaster?
• Can you put a cost to each hour of outage for your business?
• Can you keep doing business with system users in other ciEes?
• Does your business need up to the Nano-‐second data or point in Eme?
The most comprehensive Oracle applications & technology content under one roof
Typical Infrastructure Test / DR Produc6on
The most comprehensive Oracle applications & technology content under one roof
Scenario One • Earthquake – city wide impact – huge destrucEon – server room within the cordon – liquefacEon in the server room – building structurally unsound.
• Decision made early to switch to DR soluEon in another city.
• But!!!
The most comprehensive Oracle applications & technology content under one roof
Scenario One -‐ LiquefacEon
The most comprehensive Oracle applications & technology content under one roof
Scenario Two • Earthquake – city wide impact – huge destrucEon – server room within the cordon – no power to server room – limited access.
• DR site was in the midst of change – switching to DR might be painful.
• Generator onsite within 24 hours • Decision made to stay with the producEon system located on the customer site.
• But!!!
The most comprehensive Oracle applications & technology content under one roof
Scenario Two – No Electricity
Do you have access to one of these?
The most comprehensive Oracle applications & technology content under one roof
Scenario Three • Earthquake – city wide impact – huge destrucEon – server room within purpose built data centre – limited outage.
• Cross city DR site did not need to be invoked • Within an hour all other sites were working as normal.
• Local staff could focus on family and conEnue to access systems remotely as infrastructure allowed
The most comprehensive Oracle applications & technology content under one roof
Lessons Learned • Do you have split ProducEon & Test / DR infrastructures? – Test / DR ensures that your DR is ready for acEon!
• Network & Infrastructure are constantly used (tested) • Do you have a DR strategy? – GREAT!
– Have you tested it recently? – Did you deploy a JDE full package?
• A ‘Purpose Built’ data centre is best for ProducEon -‐ Why?
• Test your DR strategy regularly
The most comprehensive Oracle applications & technology content under one roof
What Worked – What Didn’t? • Data ReplicaEon – sorted J • Server images or backups in hand?
– Where is the last full backup of this machine? – What do you mean you backed up the structures ONLY!!!
– Where are the JDE installaEon CDs? • Don’t neglect your deployment server • Ensure your client technologies are covered • Any business criEcal 3rd party systems?
The most comprehensive Oracle applications & technology content under one roof
Is Distance a Factor? • Absolutely – you don’t want your DR too close – RIGHT?
• Too far apart – shipping consideraEons – Comms costs consideraEons for ANZ
• Too close and both instances are affected
• 20km+ = Ideal minimum separaEon
The most comprehensive Oracle applications & technology content under one roof
In Summary
The solu6on that I have seen work best is as follows.
• Split ProducEon and Non ProducEon (Test/DR) infrastructures • ProducEon hosted in a purpose built hosted facility where security,
power circuit redundancy, generators and diesel fuel stocks are available to ensure 24x7 security and electricity supply.
• Non producEon either hosted in an alternate facility or on site at the customer’s premises 20+ kms from the producEon data centre.
• 100Mb metropolitan LAN or equivalent available to service communicaEons requirements between the two sites
• Test the soluEon regularly & ensure your backups are appropriate
The most comprehensive Oracle applications & technology content under one roof
Build Your DR SoluEon to match your
Business ExpectaEons
No CEO/CIO/CFO affected by the Christchurch earthquakes has wanted to reduce their disaster
recovery protecEon aeer recent events.
Most have requested a full DR review Many have implemented stronger soluEons.
The most comprehensive Oracle applications & technology content under one roof
Disaster Recovery ARE YOU READY